Scene classification method, device, equipment and storage medium

Abstract

This application discloses a scene classification method, apparatus, device, and storage medium, relating to the fields of artificial intelligence technologies such as computer vision, deep learning, and autonomous driving. A specific implementation of the method includes: acquiring vehicle driving scene data; extracting features from the vehicle driving scene data using self-supervision technology in deep learning to obtain vehicle driving scene features; and classifying the vehicle driving scene features to obtain the scene category to which the vehicle driving scene data belongs. In the process of scene classification, this implementation utilizes self-supervision technology in deep learning to extract features, replacing manually defined features, ensuring the objectivity of the features, and thereby improving the accuracy of scene classification.

Description

Technical Field

[0001] The embodiments of the present application relate to the field of computer technology, specifically to the field of artificial intelligence technologies such as computer vision, deep learning, and autonomous driving, and especially to scene classification methods, devices, equipment, and storage media. Background Art

[0002] Currently, autonomous driving development utilizes large amounts of data collected on the road. In particular, autonomous driving simulation systems use this data to generate simulation test sets during algorithm testing. To ensure the integrity and authenticity of these test sets and minimize computational complexity, the collected data must be categorized according to the scenarios in which they occur.

[0003] The existing scene classification method extracts features from the collected data through artificially defined features, compares the data for similarity based on the extracted features, and classifies data with similar features into one category. Summary of the Invention

[0004] The embodiments of the present application provide a scene classification method, apparatus, device, and storage medium.

[0005] In the first aspect, an embodiment of the present application proposes a scene classification method, including: obtaining vehicle driving scene data; using self-supervision technology in deep learning to extract features of the vehicle driving scene data to obtain vehicle driving scene features; classifying the vehicle driving scene features to obtain the scene category to which the vehicle driving scene data belongs.

[0006] On the second aspect, an embodiment of the present application proposes a scene classification device, including: an acquisition module, configured to acquire vehicle driving scene data; an extraction module, configured to use self-supervision technology in deep learning to extract features from vehicle driving scene data to obtain vehicle driving scene features; a classification module, configured to classify the vehicle driving scene features to obtain the scene category to which the vehicle driving scene data belongs.

[0007] In a third aspect, an embodiment of the present application proposes an electronic device comprising: at least one processor; and a memory communicatively connected to the at least one processor; wherein the memory stores instructions executable by the at least one processor, and the instructions are executed by the at least one processor so as to enable the at least one processor to execute a method as described in any implementation manner in the first aspect.

[0008] In a fourth aspect, an embodiment of the present application proposes a non-transitory computer-readable storage medium storing computer instructions, where the computer instructions are used to enable a computer to execute the method described in any implementation manner in the first aspect.

[0009] In a fifth aspect, an embodiment of the present application provides a computer program product, including a computer program, which, when executed by a processor, implements the method described in any implementation manner in the first aspect.

[0010] The scene classification method, apparatus, device, and storage medium provided in the embodiments of the present application first acquire vehicle driving scene data; then, using self-supervised deep learning techniques, extract features from the vehicle driving scene data to obtain vehicle driving scene features; and finally, classify the vehicle driving scene features to determine the scene category to which the vehicle driving scene data belongs. During the scene classification process, self-supervised deep learning techniques are used for feature extraction, replacing manually defined features. This ensures the objectivity of the features and thereby improves the accuracy of scene classification.

[0011] It should be understood that the content described in this section is not intended to identify the key or important features of the embodiments of the present disclosure, nor is it intended to limit the scope of the present disclosure. Other features of the present disclosure will become easily understood through the following description. BRIEF DESCRIPTION OF THE DRAWINGS

[0012] Other features, objects, and advantages of the present application will become more apparent upon reading the detailed description of the non-limiting embodiments made with reference to the following drawings. The drawings are provided for a better understanding of the present invention and do not constitute a limitation of the present application. Among them:

[0013] Figure 1 is an exemplary system architecture to which the present application may be applied;

[0014] Figure 2 is a flowchart of an embodiment of a scene classification method according to the present application;

[0015] Figure 3 is a flowchart of another embodiment of the scene classification method according to the present application;

[0016] Figure 4 This is an application scenario diagram of the scenario classification method according to an embodiment of the present application;

[0017] Figure 5 is a structural diagram of an embodiment of a scene classification device according to the present application;

[0018] Figure 6 It is a block diagram of an electronic device used to implement the scene classification method of an embodiment of the present application. DETAILED DESCRIPTION

[0019] The following description of exemplary embodiments of the present application is made in conjunction with the accompanying drawings, including various details of the embodiments of the present application to facilitate understanding. These details should be considered as merely exemplary. Therefore, those skilled in the art will recognize that various changes and modifications may be made to the embodiments described herein without departing from the scope and spirit of the present application. Similarly, for the sake of clarity and conciseness, descriptions of well-known functions and structures are omitted in the following description.

[0020] It should be noted that, in the absence of conflict, the embodiments and features of the embodiments in this application can be combined with each other. The present application will be described in detail below with reference to the accompanying drawings and in combination with the embodiments.

[0021] Figure 1 An exemplary system architecture is shown to which an embodiment of the scene classification method or scene classification device of the present application can be applied.

[0022] like Figure 1 As shown, the system architecture may include a vehicle 101, a network 102, and a server 103. The network 102 is used to provide a medium for a communication link between the vehicle 101 and the server 103. The network 102 may include various connection types, such as wired or wireless communication links or fiber optic cables.

[0023] The vehicle 101 may collect vehicle driving scene data during driving, including autonomous driving vehicles and manually driven vehicles.

[0024] The server 103 can provide various services. For example, the server 103 can analyze and process the vehicle driving scene data obtained from the vehicle 101 to generate a processing result (such as the scene category to which the vehicle driving scene data belongs).

[0025] It should be noted that server 103 can be either hardware or software. When server 103 is hardware, it can be implemented as a distributed server cluster consisting of multiple servers, or as a single server. When server 103 is software, it can be implemented as multiple software programs or software modules (for example, to provide distributed services), or as a single software program or software module. This is not specifically limited here.

[0026] It should be noted that the scene classification method provided in the embodiment of the present application is generally executed by the server 103 , and accordingly, the scene classification device is generally set in the server 103 .

[0027] It should be understood that Figure 1 The number of vehicles, networks and servers in the embodiment is only illustrative. Any number of vehicles, networks and servers may be provided as needed.

[0028] Continue to refer Figure 2, which shows a process 200 of an embodiment of a scene classification method according to the present application. The scene classification method includes the following steps:

[0029] Step 201: Acquire vehicle driving scene data.

[0030] In this embodiment, the execution subject of the scene classification method (for example Figure 1 The server 103 shown can acquire vehicle driving scene data. The vehicle driving scene data can be road data collected while the vehicle is driving, including but not limited to vehicle trajectory data, obstacle trajectory data, map data, etc. The vehicle that collects the vehicle driving scene data can be either an autonomous vehicle or a manually driven vehicle.

[0031] Step 202: Use the self-supervision technology in deep learning to extract features from the vehicle driving scene data to obtain vehicle driving scene features.

[0032] In this embodiment, the above-mentioned execution entity can use the self-supervision technology in deep learning to extract features from the vehicle driving scene data to obtain vehicle driving scene features.

[0033] Deep learning (DL) is a new research direction in the field of machine learning (ML). It was introduced to ML to bring it closer to its original goal: artificial intelligence (AI). Deep learning involves learning the inherent patterns and representational hierarchies of sample data. The information gained from this learning process is highly helpful in interpreting data such as text, images, and sound. Its ultimate goal is to enable machines to possess human-like analytical and learning capabilities, enabling them to recognize data such as text, images, and sound. Self-supervised technology can be seen as an ideal state of deep learning. Models learn directly from unlabeled data, eliminating the need for data annotation. Deep learning employs self-supervised feature extraction techniques to replace manual feature definition and ensure objectivity.

[0034] Step 203: classify the vehicle driving scene features to obtain the scene category to which the vehicle driving scene data belongs.

[0035] In this embodiment, the execution entity may classify the vehicle driving scene features to obtain the scene category to which the vehicle driving scene data belongs. The similarity between the vehicle driving scene features of different vehicle driving scene data of the same scene category is generally high. The similarity between the vehicle driving scene features of different vehicle running script scene data of different scene categories is generally low. Optionally, the execution entity may perform a similarity comparison on the vehicle driving scene data using the extracted vehicle driving scene features, and classify vehicle driving scene data with similar vehicle driving scene features into the same scene category during classification.

[0036] The scene classification method provided in the embodiment of the present application first obtains vehicle driving scene data; then uses the self-supervision technology in deep learning to extract features from the vehicle driving scene data to obtain vehicle driving scene features; finally, classifies the vehicle driving scene features to obtain the scene category to which the vehicle driving scene data belongs. In the process of scene classification, the self-supervision technology in deep learning is used to extract features to replace the manually defined features, thereby ensuring the objectivity of the features and improving the accuracy of scene classification. Furthermore, based on the scene classification results, the vehicle driving scene data is labeled with scene categories, and the labeled vehicle driving scene data can help the autonomous driving simulation system to construct a test set. When constructing the test set, it can be collected from the vehicle driving scene data of each type of scene to ensure that the vehicle driving scene data of each type of scene is collected as test data, thereby ensuring a complete test road scene.

[0037] Further references Figure 3 , which shows a process 300 of another embodiment of a scene classification method according to the present application. The scene classification method includes the following steps:

[0038] Step 301: Acquire vehicle driving scene data.

[0039] In this embodiment, the subject executing the scene classification method may obtain vehicle driving scene data. The vehicle driving scene data may be on-road data collected while the vehicle is driving, including vehicle trajectory data and map data. The vehicle collecting the vehicle driving scene data may be either an autonomous vehicle or a manually driven vehicle.

[0040] Step 302 : Render the vehicle trajectory data and the map data simultaneously to obtain a rendered image sequence.

[0041] In this embodiment, the execution entity can simultaneously render vehicle trajectory data and map data to produce a rendered image sequence. The rendered images in the rendered image sequence can be sorted according to the corresponding data acquisition time. Vehicle trajectory data acquired at the same time and the map data corresponding to the vehicle's location at that time can be simultaneously rendered into a single rendered image. This way, the rendered image includes both vehicle trajectory data and map data, preserving complete information about the vehicle's driving scene.

[0042] Step 303 : compress and reconstruct the rendered image sequence through a convolutional neural network-variational autoencoder, introduce a triplet loss function in the reconstruction process, and obtain a first vehicle driving scene feature vector sequence.

[0043] In this embodiment, the execution entity may compress and reconstruct each rendered image frame in the rendered image sequence using a CNN-VAE (Convolutional Neural Networks-Variational Auto Encoder) to obtain a sequence of first vehicle driving scene feature vectors. Each rendered image frame corresponds to a sequence of first vehicle driving scene feature vectors. The first vehicle driving scene feature vectors may be used directly as vehicle driving scene features for scene classification, or may be further processed before scene classification.

[0044] CNN-VAE is a type of unsupervised learning of complex probability distributions. It is built on a standard function approximation unit, namely a neural network, and can also be optimized using stochastic gradient descent. The most significant feature of CNN-VAE is that it mimics the learning and prediction mechanism of an autoencoder, encoding and decoding between measurable functions. For any given probability distribution, there is always a differentiable, measurable function that maps it to another probability distribution, making it arbitrarily close to the target probability distribution.

[0045] When compressing and reconstructing rendered images using CNN-VAE, it's crucial not only to ensure the reconstruction quality but also to ensure that the compressed vehicle scene features can express the similarity between vehicle scene data. Therefore, a data comparison loss function is introduced during the reconstruction process, ensuring that the extracted vehicle scene features measure the similarity between vehicle scene data. This loss function is typically a triplet loss.

[0046] Step 304 : further compressing and reconstructing the first vehicle driving scene feature vector sequence using a recurrent neural network to obtain a second vehicle driving scene feature vector sequence.

[0047] In this embodiment, the execution entity can further compress and reconstruct the first vehicle driving scene feature vector sequence using an RNN (Recurrent Neural Network) to obtain a second vehicle driving scene feature vector sequence. For each sequence of first vehicle driving scene feature vectors, a corresponding sequence of second vehicle driving scene feature vectors is obtained. The second vehicle driving scene feature vectors can be directly used as vehicle driving scene features for scene classification.

[0048] Recurrent neural networks (RNNs) are a type of recurrent neural network that takes sequence data as input, performs recursion in the direction of the sequence's evolution, and has all nodes (recurrent units) connected in a chain-like fashion. Bidirectional RNNs (Bi-RNNs) and LSTMs (Long Short-Term Memory Networks) are common RNNs. RNNs have memory, parameter sharing, and Turing completeness, making them particularly advantageous for learning nonlinear features from sequences.

[0049] During the RNN's further compression and reconstruction of the first vehicle driving scene feature vector, a prediction branch was added in addition to the reconstruction branch to ensure the representativeness of the resulting vehicle driving scene features. Furthermore, a local aggregation loss function was introduced during RNN training to ensure the closeness of the vehicle driving scene features.

[0050] In this way, through two rounds of compression and reconstruction, each vehicle driving scene data can be represented by a feature vector of the same size. The vehicle driving scene features are compressed twice in time series, which takes into account the compactness of vehicle driving scene features and reduces the consumption of computing resources during the classification process.

[0051] Step 305 , classifying the vehicle driving scene features using a K-means clustering algorithm, and determining the amount of vehicle driving scene data included in each scene category based on the silhouette score, to obtain the scene category to which the vehicle driving scene data belongs.

[0052] In this embodiment, the above-mentioned execution entity can use the k-means clustering algorithm to classify the vehicle driving scene features, and determine the amount of vehicle driving scene data included in each scene category based on the silhouette score to obtain the scene category to which the vehicle driving scene data belongs.

[0053] The k-means clustering algorithm is an iterative cluster analysis algorithm. It divides the data into K groups, randomly selects K objects as initial cluster centers, calculates the distance between each object and each seed cluster center, and assigns each object to the cluster center closest to it. The cluster centers and the objects assigned to them represent a cluster. With each assigned sample, the cluster center is recalculated based on the existing objects in the cluster. This process repeats until a termination criterion is met. These termination criteria can be: no (or a minimum number of) objects are reassigned to different clusters, no (or a minimum number of) cluster centers change, and the sum of squared errors reaches a local minimum.

[0054] The most important parameter in the entire k-means clustering algorithm is K. K is determined based on the silhouette score S. The silhouette score S is shown in the following formula:

[0055] ;

[0056] Among them, a represents the average distance between the sample point and all other points in the same cluster, that is, the similarity between the sample point and other points in the same cluster; b represents the average distance between the sample point and all points in the next nearest cluster, that is, the similarity between the sample point and other points in the next nearest cluster.

[0057] K-means aims to minimize the variance within each cluster and maximize the variance outside the cluster. The silhouette score (S) is a key metric for describing this variance. As the formula shows, S ranges from -1 to 1. The closer S is to 1, the better the clustering effect, while the closer it is to -1, the worse the clustering effect.

[0058] The k-means clustering algorithm is used to classify vehicle driving scene features, with fast convergence speed. Furthermore, the k-means clustering algorithm only requires changing a single parameter, K, making it simple to use and highly interpretable.

[0059] from Figure 3 It can be seen that Figure 2Compared with the corresponding embodiment, the process 300 of the scene classification method in this embodiment highlights the feature extraction step and the scene classification step. Therefore, the scheme described in this embodiment is compressed and reconstructed twice, and each vehicle driving scene data can be represented by a feature vector of the same size. The vehicle driving scene features are compressed twice in time series, and the compactness of the vehicle driving scene features is taken into account during the compression process, which reduces the consumption of computing resources in the classification process, thereby improving the classification efficiency, and is applicable to the processing of large-scale vehicle driving scene data. The vehicle driving scene features are classified using the k-means clustering algorithm, and the convergence speed is fast. In addition, when adjusting the parameters of the k-means clustering algorithm, only one parameter K needs to be changed, and the principle is simple and the interpretability is good.

[0060] For ease of understanding, Figure 4 The following diagram shows an application scenario of the scene classification method according to an embodiment of the present application. Figure 4As shown in the figure, while driving, an autonomous vehicle collects on-road data such as map information, obstacle trajectories, and vehicle trajectories. This data sequence is then generated based on the on-road data. The data frames in this data sequence correspond to on-road data collected at the same moment. Next, data rendering is performed on the data sequence to produce uniformly rendered images. Frame feature extraction is performed on the uniformly rendered images using a CNN-VAE, resulting in a data sequence. The data frames in this data sequence are frame feature vectors. The CNN-VAE reconstructs the uniformly rendered images, introducing a reconstruction loss function during the reconstruction process to produce frame feature vectors. Finally, sequence feature compression is performed on the data sequence using an RNN to produce sequence feature vectors. The RNN includes a large number of GRUs (Gated Recurrent Units). Some of these GRUs process the frame feature vector to produce a sequence feature vector. Reconstruction branches composed of several GRUs then perform reconstruction to produce reconstructed frame features. Simultaneously, prediction branches composed of several GRUs perform prediction to produce predicted frame features. Finally, a clustering algorithm is used to cluster the sequence feature vectors to produce clustering results. On-road data clustered into the same cluster belongs to the same scene category.

[0061] Further references Figure 5 As an implementation of the methods shown in the above figures, the present application provides an embodiment of a scene classification device, which is similar to Figure 2 Corresponding to the method embodiment shown, the device can be specifically applied to various electronic devices.

[0062] like Figure 5As shown, the scene classification device 500 of this embodiment may include: an acquisition module 501, an extraction module 502, and a classification module 503. The acquisition module 501 is configured to acquire vehicle driving scene data; the extraction module 502 is configured to extract features from the vehicle driving scene data using self-supervision technology in deep learning to obtain vehicle driving scene features; and the classification module 503 is configured to classify the vehicle driving scene features to obtain the scene category to which the vehicle driving scene data belongs.

[0063] In this embodiment, the specific processing of the acquisition module 501, the extraction module 502 and the classification module 503 and the technical effects thereof can be referred to in the respective embodiments. Figure 2 The relevant descriptions of steps 201-203 in the corresponding embodiment are not repeated here.

[0064] In some optional implementations of this embodiment, the vehicle driving scene data includes vehicle trajectory data and map data; and the extraction module includes: a rendering submodule, configured to render the vehicle trajectory data and map data simultaneously to obtain a rendered image sequence; a first reconstruction submodule, configured to compress and reconstruct the rendered image sequence through a convolutional neural network-variational autoencoder, introducing a triplet loss function in the reconstruction process to obtain a first vehicle driving scene feature vector sequence.

[0065] In some optional implementations of this embodiment, the extraction module also includes: a second reconstruction submodule, configured to use a recurrent neural network to further compress and reconstruct the first vehicle driving scene feature vector sequence to obtain a second vehicle driving scene feature vector sequence.

[0066] In some optional implementations of this embodiment, a prediction branch is added to the recurrent neural network, and a local clustering loss function is introduced during the training process.

[0067] In some optional implementations of this embodiment, the classification module is further configured to: classify vehicle driving scene features using a K-means clustering algorithm, and determine the amount of vehicle driving scene data included in each scene category based on the silhouette score.

[0068] According to an embodiment of the present application, the present application also provides an electronic device and a readable storage medium.

[0069] like Figure 6, is a block diagram of an electronic device according to the scene classification method of an embodiment of the present application. The electronic device is intended to represent various forms of digital computers, such as laptop computers, desktop computers, workstations, personal digital assistants, servers, blade servers, mainframe computers, and other suitable computers. The electronic device can also represent various forms of mobile devices, such as personal digital processing, cellular phones, smart phones, wearable devices and other similar computing devices. The components shown herein, their connections and relationships, and their functions are merely examples and are not intended to limit the implementation of the present application described and / or required herein.

[0070] like Figure 6 As shown, the electronic device includes: one or more processors 601, a memory 602, and interfaces for connecting various components, including high-speed interfaces and low-speed interfaces. The various components are connected to each other using different buses and can be installed on a common mainboard or installed in other ways as needed. The processor can process instructions executed in the electronic device, including instructions stored in or on the memory to display graphical information of a GUI on an external input / output device (such as a display device coupled to the interface). In other embodiments, if necessary, multiple processors and / or multiple buses can be used together with multiple memories and multiple memories. Similarly, multiple electronic devices can be connected, and each device provides some necessary operations (for example, as a server array, a group of blade servers, or a multi-processor system). Figure 6 A processor 601 is taken as an example.

[0071] Memory 602 is the non-transitory computer-readable storage medium provided in this application. The memory stores instructions executable by at least one processor to cause the at least one processor to perform the scene classification method provided in this application. The non-transitory computer-readable storage medium of this application stores computer instructions for causing a computer to perform the scene classification method provided in this application.

[0072] The memory 602 is a non-transient computer-readable storage medium that can be used to store non-transient software programs, non-transient computer executable programs and modules, such as the program instructions / modules corresponding to the scene classification method in the embodiment of the present application (for example, the attached Figure 5 The processor 601 executes the non-transient software programs, instructions, and modules stored in the memory 602 to execute various functional applications and data processing of the server, that is, to implement the scene classification method in the above method embodiment.

[0073] The memory 602 may include a program storage area and a data storage area, wherein the program storage area may store an operating system and applications required for at least one function; the data storage area may store data created by the use of the electronic device according to the scene classification method, etc. In addition, the memory 602 may include a high-speed random access memory, and may also include a non-transient memory, such as at least one disk storage device, a flash memory device, or other non-transient solid-state storage device. In some embodiments, the memory 602 may optionally include a memory remotely located relative to the processor 601, and these remote memories may be connected to the electronic device of the scene classification method via a network. Examples of the above-mentioned network include, but are not limited to, the Internet, an intranet, a local area network, a mobile communication network, and combinations thereof.

[0074] The electronic device of the scene classification method may further include: an input device 603 and an output device 604. The processor 601, the memory 602, the input device 603 and the output device 604 may be connected via a bus or other means. Figure 6 The bus connection is taken as an example.

[0075] The input device 603 can receive input digital or character information and generate key signal input related to user settings and function control of the electronic device for the scene classification method. Input devices such as a touch screen, keypad, mouse, trackpad, touchpad, pointer, one or more mouse buttons, trackball, joystick, etc. can be used. The output device 604 can include a display device, auxiliary lighting devices (e.g., LEDs), and tactile feedback devices (e.g., vibration motors). The display device may include, but is not limited to, a liquid crystal display (LCD), a light-emitting diode (LED) display, and a plasma display. In some embodiments, the display device may be a touch screen.

[0076] Various implementations of the systems and techniques described herein can be realized in digital electronic circuitry, integrated circuitry, dedicated ASICs (application specific integrated circuits), computer hardware, firmware, software, and / or combinations thereof. These various implementations can include implementation in one or more computer programs that are executable and / or interpreted on a programmable system comprising at least one programmable processor, which can be a special purpose or general purpose programmable processor that can receive data and instructions from a storage system, at least one input device, and at least one output device, and transmit data and instructions to the storage system, the at least one input device, and the at least one output device.

[0077] These computer programs (also referred to as programs, software, software applications, or code) include machine instructions for programmable processors and can be implemented using high-level procedural and / or object-oriented programming languages, and / or assembly / machine languages. As used herein, the terms "machine-readable medium" and "computer-readable medium" refer to any computer program product, apparatus, and / or device (e.g., a magnetic disk, an optical disk, a memory, a programmable logic device (PLD)) for providing machine instructions and / or data to a programmable processor, including a machine-readable medium that receives machine instructions as a machine-readable signal. The term "machine-readable signal" refers to any signal for providing machine instructions and / or data to a programmable processor.

[0078] To provide interaction with a user, the systems and techniques described herein can be implemented on a computer having: a display device (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor) for displaying information to the user; and a keyboard and pointing device (e.g., a mouse or trackball) through which the user can provide input to the computer. Other types of devices can also be used to provide interaction with the user; for example, the feedback provided to the user can be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user can be received in any form (including acoustic input, voice input, or tactile input).

[0079] The systems and techniques described herein can be implemented in a computing system that includes back-end components (e.g., as a data server), or a computing system that includes middleware components (e.g., an application server), or a computing system that includes front-end components (e.g., a user computer with a graphical user interface or a web browser through which a user can interact with implementations of the systems and techniques described herein), or a computing system that includes any combination of such back-end components, middleware components, or front-end components. The components of the system can be interconnected by any form or medium of digital data communication (e.g., a communication network). Examples of communication networks include a local area network (LAN), a wide area network (WAN), and the Internet.

[0080] Computer systems may include clients and servers. A client and server are generally remote from each other and typically interact through a communication network. The client and server relationship arises through computer programs running on the respective computers and having a client-server relationship to each other.

[0081] According to the technical solution of this application, vehicle driving scene data is first acquired; then, self-supervised deep learning techniques are used to extract features from the vehicle driving scene data to obtain vehicle driving scene features; finally, the vehicle driving scene features are classified to determine the scene category to which the vehicle driving scene data belongs. During the scene classification process, self-supervised deep learning techniques are used to extract features, replacing manual feature definition, ensuring the objectivity of the features and thereby improving the accuracy of scene classification.

[0082] It should be understood that the various forms of the processes shown above can be used to reorder, add, or delete steps. For example, the steps described in this application can be performed in parallel, sequentially, or in a different order, as long as the desired results of the technical solutions disclosed in this application can be achieved. This is not a limitation herein.

[0083] The above specific embodiments do not constitute a limitation on the scope of protection of this application. Those skilled in the art will appreciate that various modifications, combinations, sub-combinations, and substitutions may be made based on design requirements and other factors. Any modifications, equivalent substitutions, and improvements made within the spirit and principles of this application shall be included within the scope of protection of this application.

Claims

1. A scene classification method, comprising: Obtain vehicle driving scene data; Using the self-supervision technology in deep learning to extract features from the vehicle driving scene data, to obtain vehicle driving scene features; Classifying the vehicle driving scene features to obtain the scene category to which the vehicle driving scene data belongs; Wherein, the vehicle driving scene data includes vehicle trajectory data and map data; and The method of extracting features from the vehicle driving scene data using the self-supervision technology in deep learning to obtain vehicle driving scene features includes: Rendering the vehicle trajectory data and the map data simultaneously to obtain a rendered image sequence; compressing and reconstructing the rendered image sequence through a convolutional neural network-variational autoencoder, introducing a triplet loss function in the reconstruction process, and obtaining a first vehicle driving scene feature vector sequence; The first vehicle driving scene feature vector sequence is further compressed and reconstructed using a recurrent neural network to obtain a second vehicle driving scene feature vector sequence.

2. The method according to claim 1, wherein The recurrent neural network adds a prediction branch and introduces a local clustering loss function during the training process.

3. The method according to claim 1 or 2, wherein: The classifying the vehicle driving scene features includes: The vehicle driving scene features are classified using a K-means clustering algorithm, wherein K is determined based on the silhouette score.

4. A scene classification device, comprising: an acquisition module, configured to acquire vehicle driving scene data; an extraction module configured to extract features from the vehicle driving scene data using a self-supervised technique in deep learning to obtain vehicle driving scene features; a classification module configured to classify the vehicle driving scene features to obtain the scene category to which the vehicle driving scene data belongs; Wherein, the vehicle driving scene data includes vehicle trajectory data and map data; and The extraction module includes: a rendering submodule, configured to simultaneously render the vehicle trajectory data and the map data to obtain a rendered image sequence; a first reconstruction submodule configured to compress and reconstruct the rendered image sequence through a convolutional neural network-variational autoencoder, introducing a triplet loss function in the reconstruction process, and obtain a first vehicle driving scene feature vector sequence; The second reconstruction submodule is configured to further compress and reconstruct the first vehicle driving scene feature vector sequence using a recurrent neural network to obtain a second vehicle driving scene feature vector sequence.

5. The device according to claim 4, wherein The recurrent neural network adds a prediction branch and introduces a local clustering loss function during the training process.

6. The device according to claim 4 or 5, wherein: The classification module is further configured to: The vehicle driving scene features are classified using a K-means clustering algorithm, wherein K is determined based on the silhouette score.

7. An electronic device comprising: at least one processor; as well as a memory communicatively connected to the at least one processor; wherein, The memory stores instructions that can be executed by the at least one processor, and the instructions are executed by the at least one processor to enable the at least one processor to perform the method according to any one of claims 1 to 3.

8. A non-transitory computer-readable storage medium storing computer instructions, wherein the computer instructions are used to cause the computer to execute the method according to any one of claims 1 to 3.

9. A computer program product comprising a computer program, which, when executed by a processor, implements the method according to any one of claims 1 to 3.

Images