Scene data determination method and device, storage medium and electronic device

Abstract

The invention discloses a scene data determination method and device, a storage medium and an electronic device, and relates to the field of automatic driving, and the scene data determination method comprises the steps: determining an expected scene vector corresponding to a target driving task under the condition that the target vehicle has the to-be-executed target driving task, determining scene vectors respectively corresponding to the plurality of driving tasks; wherein the expected scene vector is obtained by vectorizing expected scene data corresponding to a target driving task, and the target driving task is any one of the plurality of driving tasks; the plurality of scene vectors are obtained by vectorizing scene data pre-collected by a plurality of driving tasks; determining at least one target scene vector matched with the expected scene vector from the plurality of scene vectors; and determining target scene data of the target driving task according to the at least one target scene vector.

Description

Technical Field

[0001] The present application relates to the field of intelligent driving, and specifically, to a method and device for determining scene data, a storage medium, and an electronic device. Background Art

[0002] With the development of autonomous driving technology, driving data, as one of the core elements of autonomous driving (algorithms, computing power, and data), plays an important role in all aspects of autonomous driving.

[0003] Among driving data, unstructured data (such as image scene data, point cloud scene data, video clips and trajectory scene data, etc.) constitute an important part of driving data. They carry rich environmental information and driving situation details, and are crucial for training and verifying the algorithms of autonomous driving systems.

[0004] Currently, when querying unstructured data, searches are usually performed based on file attributes such as file size and name. Since the file attributes are too broad, data irrelevant to the driving task will always be found, affecting the search accuracy.

[0005] Regarding the problem in the related art where retrieval is based on file attributes such as file size and name, since the file attributes are too broad, data irrelevant to the driving task will always be found, affecting the search accuracy. No effective solution has been proposed so far. Therefore, it is necessary to improve the related art to overcome the above-mentioned defects in the related art. Summary of the Invention

[0006] The embodiments of the present application provide a method and device for determining scene data, a storage medium, and an electronic device to at least solve the problem in the related art that retrieval is based on file attributes such as file size and name. Since the file attributes are too broad, data irrelevant to the driving task will always be found, affecting the search accuracy.

[0007] According to one aspect of an embodiment of the present application, a method for determining scene data is provided, comprising: when a target driving task to be performed on a target vehicle exists, determining an expected scene vector corresponding to the target driving task, and determining scene vectors corresponding to a plurality of driving tasks, respectively; wherein the expected scene vector is obtained by vectorizing expected scene data corresponding to the target driving task, and the target driving task is any one of the plurality of driving tasks; the plurality of scene vectors are obtained by vectorizing scene data pre-collected for the plurality of driving tasks; determining at least one target scene vector from the plurality of scene vectors that matches the expected scene vector; and determining target scene data for the target driving task based on the at least one target scene vector.

[0008] In an exemplary embodiment, determining scene vectors corresponding to multiple driving tasks includes: obtaining scene data pre-collected for the multiple driving tasks; determining a coding strategy that matches the scene data based on the data type of the scene data; wherein the coding strategy is used to vectorize the scene data; and for each driving task in the multiple driving tasks, vectorizing the scene data of the driving task based on the coding strategy that matches the scene data of the driving task to obtain the scene vector for the driving task.

[0009] In an exemplary embodiment, the encoding strategy based on matching the scene data of the targeted driving task vectorizes the scene data of the targeted driving task to obtain the scene vector of the targeted driving task, including: when the scene data of the targeted driving task includes image scene data, performing image feature extraction on the image scene data, determining the image features extracted from the image scene data as the image vector corresponding to the image scene data, and performing text description on the image scene data to generate text data of the image scene data; performing text feature extraction on the text data, determining the text features extracted from the text data as the text vector corresponding to the text data; and fusing the image vector and the text vector to obtain the image scene vector of the image scene data.

[0010] In an exemplary embodiment, the encoding strategy based on the scene data matching the driving task is used to vectorize the scene data of the driving task to obtain the scene vector of the driving task, including: when the scene data includes point cloud scene data, performing image conversion on the point cloud scene data to map the point cloud scene data to a pre-divided plane grid to construct point cloud image data; performing image feature extraction on the point cloud image data, determining the image features extracted from the point cloud image data as a point cloud image vector corresponding to the point cloud image data, and performing text feature extraction on the point cloud image data, determining the text features extracted from the point cloud image data as a point cloud text vector corresponding to the point cloud image data; and fusing the point cloud image vector and the point cloud text vector to determine the point cloud scene vector of the point cloud scene data.

[0011] In an exemplary embodiment, the encoding strategy based on the matching of the scene data of the driving task is used to vectorize the scene data of the driving task to obtain the scene vector of the driving task, including: when the scene data includes trajectory scene data, determining the global feature vector and the local feature vector of the trajectory scene data; and fusing the global feature vector and the local feature vector to obtain the trajectory scene vector of the trajectory scene data.

[0012] In an exemplary embodiment, determining the global feature vector and the local feature vector of the trajectory scene data includes: performing trajectory feature extraction on the trajectory scene data, determining the trajectory features extracted from the trajectory scene data as the global feature vector corresponding to the trajectory scene data, and splitting the trajectory scene data to obtain multiple trajectory split data; performing trajectory feature extraction on the multiple trajectory split data, and determining the trajectory features extracted from the multiple trajectory split data as the local feature vector of the trajectory scene data.

[0013] In an exemplary embodiment, after determining the scene vectors corresponding to multiple driving tasks respectively, the method further includes: obtaining a pre-constructed vector database; wherein the vector database includes at least a vector field and a scalar field; writing the scene vectors corresponding to the multiple driving tasks respectively into the vector field in the vector database, and writing the data identifier associated with the scene vector into the scalar field in the vector database, wherein the data identifier is used to distinguish the scene data corresponding to the scene vector.

[0014] In an exemplary embodiment, determining the target scene data of the target driving task based on the at least one target scene vector includes: determining a target data identifier associated with the at least one target scene vector; based on the target data identifier, matching with a data identifier in a pre-built task database to determine a data identifier that matches the target data identifier; wherein the task database includes multiple data identifiers and scene data corresponding to the multiple data identifiers; the data identifier is used to distinguish the scene data corresponding to the scene vector; and the scene data corresponding to the data identifier is determined as the target scene data of the target driving task.

[0015] According to another aspect of an embodiment of the present application, a device for determining scene data is provided, including:

[0016] The first determination module is used to determine, when a target driving task to be performed on a target vehicle exists, an expected scene vector corresponding to the target driving task, and to determine scene vectors corresponding to each of multiple driving tasks; wherein the expected scene vector is obtained by vectorizing expected scene data corresponding to the target driving task, and the target driving task is any one of the multiple driving tasks; and the multiple scene vectors are obtained by vectorizing scene data pre-collected for the multiple driving tasks; the second determination module is used to determine at least one target scene vector that matches the expected scene vector from the multiple scene vectors; and the third determination module is used to determine target scene data for the target driving task based on the at least one target scene vector.

[0017] According to another aspect of the embodiments of the present application, a computer-readable storage medium is further provided, wherein the computer-readable storage medium includes a stored program, wherein the program is configured to execute the above-mentioned method for determining scene data when running.

[0018] According to another aspect of an embodiment of the present application, an electronic device is provided, including a memory and a processor, wherein a computer program is stored in the memory, wherein the processor is configured to execute the above-mentioned method for determining scene data through the computer program.

[0019] According to another aspect of the embodiments of the present application, a computer program product is provided, including a computer program, and the method for determining the scene data described above when the computer program is executed by a processor.

[0020] Through the present application, when a target vehicle has a target driving task to be executed, an expected scene vector corresponding to the target driving task is determined, and scene vectors corresponding to multiple driving tasks are determined, wherein the scene vector is obtained by vectorizing scene data pre-collected for the multiple driving tasks; and at least one target scene vector matching the expected scene vector is determined from the multiple scene vectors. Since the target scene vector is a vector that matches the expected scene vector among the multiple scene vectors, and the expected scene vector is obtained by vectorizing the expected scene data corresponding to the target driving task, the target scene data determined by at least one target scene vector must be related to the target driving task, thereby effectively improving the data search accuracy. BRIEF DESCRIPTION OF THE DRAWINGS

[0021] The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments consistent with the present application and, together with the description, serve to explain the principles of the present application.

[0022] In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the following briefly introduces the drawings required for use in the embodiments or the description of the prior art. Obviously, for ordinary technicians in this field, other drawings can be obtained based on these drawings without any creative work.

[0023] Figure 1 This is a hardware structure block diagram of a vehicle terminal for a method for determining scene data according to an embodiment of the present application;

[0024] Figure 2 is a flowchart of a method for determining scene data according to an embodiment of the present application;

[0025] Figure 3 is another flow chart of a method for determining scene data according to an embodiment of the present application;

[0026] Figure 4 is a flowchart of processing image scene data according to an embodiment of the present application;

[0027] Figure 5 is a flowchart of processing trajectory scene data according to an embodiment of the present application;

[0028] Figure 6 is an example diagram of the architecture of a neural network model according to an embodiment of the present application;

[0029] Figure 7 is a flowchart of a data query process of a method for determining scene data according to an embodiment of the present application;

[0030] Figure 8 is a schematic diagram of experimental results of a method for determining scene data according to an embodiment of the present application;

[0031] Figure 9 This is a structural block diagram of a device for determining scene data according to an embodiment of the present application. DETAILED DESCRIPTION

[0032] In order to enable those skilled in the art to better understand the present invention, the following will clearly and completely describe the technical solutions in the embodiments of the present invention in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments in the present invention, all other embodiments obtained by ordinary technicians in this field without making creative efforts should fall within the scope of protection of this application.

[0033] It should be noted that the terms "first", "second", etc. in the specification and claims of the present application and the above-mentioned drawings are used to distinguish similar objects and are not necessarily used to describe a specific order or sequential order. It should be understood that the data used in this way can be interchangeable where appropriate, so that the embodiments of the present application described herein can be implemented in a sequence other than those illustrated or described herein. In addition, the terms "including" and "having" and any of their variations are intended to cover non-exclusive inclusions, for example, a process, method, system, product or device comprising a series of steps or units is not necessarily limited to those steps or units clearly listed, but may include other steps or units that are not clearly listed or inherent to these processes, methods, products or devices.

[0034] The method embodiments provided in the embodiments of the present application can be executed in a vehicle-mounted terminal or a similar computing device. Taking running on a vehicle-mounted terminal as an example, Figure 1 This is a hardware structure block diagram of a vehicle terminal for a method of determining scene data according to an embodiment of the present application. Figure 1 As shown, the vehicle terminal may include one or more ( Figure 1 Only one is shown) a processor 102 (the processor 102 may include but is not limited to a microprocessor (MP) or a programmable logic device (FPGA)) and a memory 104 for storing data. The vehicle-mounted terminal may also include a transmission device 106 and an input / output device 108 for communication functions. It will be understood by those skilled in the art that Figure 1 The structure shown is only for illustration and does not limit the structure of the vehicle-mounted terminal. Figure 1 More or fewer components than shown, or with Figure 1 Different configurations shown.

[0035] The memory 104 can be used to store computer programs, such as software programs and modules of application software, such as the computer program corresponding to the method for detecting aerodynamic imbalance in the embodiment of the present application. The processor 102 executes the computer program stored in the memory 104 to execute various functional applications and data processing, thereby implementing the above-mentioned method. The memory 104 may include high-speed random access memory and may also include non-volatile memory, such as one or more magnetic storage devices, flash memory, or other non-volatile solid-state memory. In some examples, the memory 104 may further include memory remotely located relative to the processor 102, and these remote memories can be connected to the vehicle terminal via a network. Examples of the above-mentioned network include, but are not limited to, the Internet, corporate intranets, local area networks, mobile communication networks, and combinations thereof.

[0036] The transmission device 106 is used to receive or send data via a network. A specific example of the aforementioned network may include a wireless network provided by the vehicle terminal's communications provider. In one embodiment, the transmission device 106 includes a network interface controller (NIC), which can be connected to other network devices via a base station to enable communication with the Internet. In one embodiment, the transmission device 106 may be a radio frequency (RF) module, which is used to communicate with the Internet wirelessly.

[0037] In this embodiment, a method for determining scene data is provided, which is applied to a vehicle-mounted terminal. Figure 2 is a flow chart of a method for determining scene data according to an embodiment of the present application. Figure 2 As shown, the process includes the following steps S202-S206:

[0038] Step S202: When a target driving task to be performed exists for the target vehicle, determining an expected scene vector corresponding to the target driving task, and determining scene vectors corresponding to each of the plurality of driving tasks; wherein the expected scene vector is obtained by vectorizing expected scene data corresponding to the target driving task, the target driving task being any one of the plurality of driving tasks; and the plurality of scene vectors are obtained by vectorizing scene data pre-collected for the plurality of driving tasks;

[0039] It should be noted that autonomous driving tasks refer to a series of operations or objectives involved in the autonomous driving field. These tasks include, but are not limited to, model training and validation, abnormal event analysis, and autonomous driving system debugging. Specifically, during model training and validation, it is necessary to retrieve specific types of annotated data, such as images or point cloud data containing pedestrians, vehicles, and traffic signs, for model training and validation. For example, to improve the model's recognition capabilities at complex intersections, a large amount of data containing complex intersection scenes can be retrieved and used for training. Abnormal event analysis refers to the process of analyzing abnormal events. For example, when an autonomous driving system encounters an abnormal situation during testing or actual operation, such as an incorrectly recognized obstacle or an anomalous driving decision, relevant datasets, such as data frames containing the abnormal event, can be retrieved to analyze the cause of the problem. During autonomous driving system debugging, data related to specific functions or modules can be retrieved to verify their performance and the accuracy of the recognition algorithm. For example, to debug a vehicle steering prediction model, point cloud data containing vehicle steering movements can be retrieved.

[0040] Scenario data can include a physical description of the surrounding environment, such as the road layout, traffic signs, and the position, speed, and direction of static and dynamic obstacles such as pedestrians and vehicles. Expected scenario data is the scenario data expected to be required to complete the target driving task or achieve a specific goal.

[0041] It is understood that scene vectors can be used to characterize the features of real or simulated scene data collected during a driving task. Scene vectors are derived by extracting features and vectorizing unstructured data (such as images, point clouds, videos, and trajectory data) using a deep neural network model. Expected scene vectors can be used to characterize the data features of the expected scene data corresponding to the target driving task.

[0042] In some embodiments, when the target driving task is model training, it may be necessary to find images containing certain specific objects or features to enhance the training dataset. For example, if the model performs poorly at recognizing bicycles, the desired scene data may be images containing bicycles. Based on this desired scene data, more bicycle-related images can be retrieved from the scene data for model retraining and optimization.

[0043] Step S204: determining at least one target scene vector matching the expected scene vector from the multiple scene vectors.

[0044] It should be noted that in the process of determining the target scene vector that matches the expected scene vector, this can be achieved by calculating the similarity or distance between the scene vector and the expected scene vector, which can specifically include Euclidean distance, cosine similarity, etc. The target scene vector can provide the case that is most similar to the target driving task or situation.

[0045] Step S206: Determine target scene data for the target driving task based on the at least one target scene vector.

[0046] In an exemplary embodiment, determining the target scene data of the target driving task based on the at least one target scene vector includes: determining a target data identifier associated with the at least one target scene vector; based on the target data identifier, matching with a data identifier in a pre-built task database to determine a data identifier that matches the target data identifier; wherein the task database includes multiple data identifiers and scene data corresponding to the multiple data identifiers; the data identifier is used to distinguish the scene data corresponding to the scene vector; and the scene data corresponding to the data identifier is determined as the target scene data of the target driving task.

[0047] It is understood that each target scene vector can carry one or more data identifiers. A data identifier is a unique identifier generated when the scene vector is stored to associate it with specific scene data. The data identifier can be a file name, dataset ID, timestamp, etc., and is used to identify and distinguish different scene data. For example, a target scene vector may be associated with the data identifier "intersection left turn_20230315_123456", indicating that it is a scene of turning left at an intersection at a specific time.

[0048] Specifically, the target data identifier associated with the target scene vector can be matched with the data identifier in the pre-built task database to determine the data identifier that matches the target data identifier. The task database is different from the vector database. It mainly stores scene data (such as images, videos, point clouds, etc.) and data identifiers related to these scene data. The data identifiers in the task database (such as "turn left at the intersection_20230315_123456") are directly associated with the stored scene data and are used to quickly locate and retrieve specific scene data.

[0049] It is understandable that the target scene data can contain relevant information under the target driving task, such as road conditions, obstacles, location and behavior of traffic participants, vehicle status, etc., which is the basis for the understanding and decision-making of the autonomous driving system.

[0050] In some embodiments, when matching the target data identifier with the data identifiers in the pre-built task database, exact matching (i.e., the identifiers are exactly the same) or fuzzy matching (allowing for differences in some parts of the identifiers) can be used. Fuzzy matching is useful when processing scenarios with similar but not identical features, such as "left turn" operations performed at different times or weather conditions.

[0051] In some embodiments, the acquired target scenario data can be used to execute autonomous driving tasks. For example, by analyzing data from a "left turn at an intersection" scenario, the optimal left turn timing, expected traffic flow, and the direction of the vehicle ahead can be determined, thereby planning the safest and most efficient path and driving strategy.

[0052] In the above embodiment, based on the target data identification, the target scene data of the target driving task is accurately located, providing a solid data foundation for realizing intelligent and safe driving decisions.

[0053] Through the above steps, when a target driving task to be performed on a target vehicle is present, an expected scene vector corresponding to the target driving task is determined, as well as scene vectors corresponding to a plurality of driving tasks, respectively. The scene vectors are obtained by vectorizing scene data pre-collected for the plurality of driving tasks. At least one target scene vector matching the expected scene vector is determined from the plurality of scene vectors. Since the target scene vector is a vector matching the expected scene vector among the plurality of scene vectors, and the expected scene vector is obtained by vectorizing the expected scene data corresponding to the target driving task, the target scene data determined by the at least one target scene vector must be related to the target driving task, thereby effectively improving the data search accuracy.

[0054] In an exemplary embodiment, determining scene vectors corresponding to multiple driving tasks includes: obtaining scene data pre-collected for the multiple driving tasks; determining a coding strategy that matches the scene data based on the data type of the scene data; wherein the coding strategy is used to vectorize the scene data; and for each driving task in the multiple driving tasks, vectorizing the scene data of the driving task based on the coding strategy that matches the scene data of the driving task to obtain the scene vector for the driving task.

[0055] Among them, scene data of multiple driving tasks can be collected, and the data types of these scene data can include images, point clouds, trajectories, text, etc. The encoding strategy refers to the strategy of converting scene data into numerical vectors. Since different data types (such as images, point clouds, text, etc.) have different characteristics, it is necessary to select a suitable encoding strategy for each type of data. For example, for image scene data, a convolutional neural network (CNN (Convolutional Neural Network) can be used for feature extraction and vectorization; for point cloud data, a point cloud to image conversion (such as BEV (Bird's Eye View Transformation) conversion) may be adopted, and then an image encoding strategy is used; for text data (such as driving logs), a text embedding model (such as BERT (Bidirectional Encoder Representations from Transformers)) can be used.

[0056] It's important to note that for each driving task, a matching encoding strategy is applied to the scene data type for vectorization. This step is crucial for converting unstructured data into a structured vector representation, enabling it to be understood and processed by machine learning models and vector databases. For example, for image scene data, features are extracted using a CNN model to produce a fixed-dimensional vector. For point cloud scene data from a turning task, BEV conversion is first performed, followed by a vector generated using an image encoding strategy.

[0057] In some embodiments, after encoding and vectorizing the scene data for each driving task, corresponding scene vectors are generated. These scene vectors not only contain the key features of the scene data but can also include scene-related metadata (such as timestamp, location, weather conditions, etc.) to further enhance the vector's descriptive power and retrieval accuracy.

[0058] In some embodiments, scene vectors can be stored in a vector database. Upon storage, the vector database can store the scene vectors and related metadata according to a pre-set schema for subsequent retrieval and analysis. Simultaneously, the vector database constructs corresponding index structures (e.g., IVF_FLAT, HNSW) to support efficient and accurate similarity retrieval, which is crucial for autonomous driving systems to make decisions during mission execution.

[0059] In the above embodiment, the scene data of each driving task is converted into a vector representation that is easy to retrieve and analyze, and stored in a vector database, providing a solid data foundation and high-performance data retrieval capabilities for the real-time decision-making and intelligent driving of the autonomous driving system, thereby not only improving the efficiency of data processing, but also enhancing the understanding and adaptability of the autonomous driving field to complex driving scenarios.

[0060] In an exemplary embodiment, the vectorization processing of the scene data of the driving task is performed based on the encoding strategy matching the scene data of the driving task to obtain the scene vector of the driving task. This can be achieved by the following steps S21-S23:

[0061] Step S21: When the scene data of the driving task includes image scene data, performing image feature extraction on the image scene data, determining the image features extracted from the image scene data as image vectors corresponding to the image scene data, and performing text description on the image scene data to generate text data of the image scene data;

[0062] It is understandable that when the scene data includes image scene data, feature extraction can be performed on the image scene data. This can be done specifically through a convolutional neural network (CNN), which can identify and extract key features in images, such as edges, textures, shapes, colors, etc. For example, for obstacle recognition in driving tasks, CNN can learn the typical visual features of obstacles. The extracted features are encoded into a fixed-length vector, namely the image vector, which summarizes the main information of the image and facilitates subsequent vector retrieval.

[0063] In some embodiments, to enhance the descriptive power of image vectors, a textual description of the image data can also be generated. This can be manually added labels or annotations, or automatically generated image descriptions, such as those obtained through image-to-text conversion models (e.g., Image Captioning models).

[0064] Step S22: performing text feature extraction on the text data, and determining the text features extracted from the text data as text vectors corresponding to the text data;

[0065] Specifically, feature extraction is performed on text data, and a text embedding model such as BERT is used to convert the text into a text vector. The text vector contains semantic information about the image scene, such as "straight road" or "pedestrians ahead".

[0066] Step S23: Fusing the image vector and the text vector to obtain an image scene vector of the image scene data.

[0067] Fusion of image and text vectors can produce a more comprehensive and semantically rich representation, namely the image scene vector. This fusion can be achieved in a variety of ways, such as by performing element-wise addition or multiplication of the two vectors, or by training a joint embedding model to learn how to optimally fuse the two types of information.

[0068] In some embodiments, the image vector and the text vector may be combined through a weighted average or concatenation operation to form a comprehensive vector, namely, an image scene vector.

[0069] In the above examples, image scene data is converted into a structured, searchable vector form, significantly improving the efficiency and accuracy of scene understanding and retrieval in autonomous driving. This method of constructing scene vectors leverages the complementary nature of image and text data, providing a richer and more detailed representation of environmental information for autonomous driving.

[0070] In an exemplary embodiment, the encoding strategy based on the scene data matching the driving task is used to vectorize the scene data of the driving task to obtain the scene vector of the driving task, including: when the scene data includes point cloud scene data, performing image conversion on the point cloud scene data to map the point cloud scene data to a pre-divided plane grid to construct point cloud image data; performing image feature extraction on the point cloud image data, determining the image features extracted from the point cloud image data as a point cloud image vector corresponding to the point cloud image data, and performing text feature extraction on the point cloud image data, determining the text features extracted from the point cloud image data as a point cloud text vector corresponding to the point cloud image data; and fusing the point cloud image vector and the point cloud text vector to determine the point cloud scene vector of the point cloud scene data.

[0071] It should be noted that when the scene data includes point cloud scene data, the point cloud scene data can be converted into an image format. For example, this can be achieved by constructing a Bird's Eye View (BEV) image. The point cloud scene data is projected onto a pre-divided plane grid to form a two-dimensional image. This process is called BEV conversion. The BEV image provides a top-down perspective to observe the environment, which can clearly show the layout of roads, vehicles, pedestrians, etc. in a plane coordinate system. Through this conversion, the three-dimensional spatial information of the point cloud scene data is simplified into a two-dimensional image, which facilitates subsequent feature extraction and processing.

[0072] Furthermore, point cloud image data can be used for image feature extraction using deep learning models, such as convolutional neural networks (CNNs). CNNs are capable of identifying various features in images, such as shape, texture, color, as well as object boundaries and positions. The image features extracted from point cloud image data are encoded into a fixed-length vector, the point cloud image vector, which summarizes the main visual information of the point cloud image.

[0073] In addition to image features, point cloud scene data can also be accompanied by text descriptions, such as annotations or explanations of specific objects in the scene. Natural language processing techniques, such as BERT, are used for feature extraction. The features extracted from the text data are encoded into a text vector, the point cloud text vector, which contains semantic information about the scene.

[0074] Finally, to form a comprehensive scene vector representation, the point cloud image vector and the point cloud text vector can be fused. This fusion can be accomplished by vector concatenation or by training a joint embedding model. The fused vector, the point cloud scene vector, contains both visual and semantic information about the scene, making scene retrieval based on similarity search possible.

[0075] In some embodiments, the point cloud scene data can be pre-processed, such as removing abnormal points, performing coordinate transformation and normalization, etc., for subsequent projection. The pre-processed point cloud scene data is projected into a two-dimensional plane grid to form a BEV image, where the value of each grid cell can be the average value of the number, height or other features of points in the area. CNN is used to extract features from the BEV image to generate a point cloud image vector. In the case where there is relevant text description of the point cloud scene data, NLP (Natural Language Processing) is used to convert the text description into a text vector. The point cloud image vector and the point cloud text vector are combined into a high-dimensional vector representing the point cloud scene through vector splicing or a more complex fusion mechanism.

[0076] In the above embodiment, the point cloud scene data is effectively encoded into a vector, which not only contains the visual features of the scene but also integrates semantic information, greatly improving the efficiency and accuracy of scene understanding and retrieval in the field of autonomous driving.

[0077] In an exemplary embodiment, the encoding strategy based on the matching of the scene data of the driving task is used to vectorize the scene data of the driving task to obtain the scene vector of the driving task, including: when the scene data includes trajectory scene data, determining the global feature vector and the local feature vector of the trajectory scene data; and fusing the global feature vector and the local feature vector to obtain the trajectory scene vector of the trajectory scene data.

[0078] It should be noted that trajectory scene data can include the trajectory points of a vehicle or object. These points are arranged in time and space, forming a specific driving path or motion pattern. To convert this information into a vector, global and local features of the trajectory can be extracted.

[0079] Global eigenvectors can be calculated using statistical and geometric methods and summarize the overall shape of the trajectory. Local eigenvectors: Local features focus on details within the trajectory, such as curvature during turns, acceleration and deceleration, and changes in relative position to other objects. These features are typically extracted by performing a sliding window analysis on the trajectory points. The points within the window are used to calculate local dynamic properties, such as changes in curvature and velocity.

[0080] Specifically, after obtaining the global and local feature vectors, they are fused into a single vector representation, the trajectory scene vector. This fusion can be done through simple vector concatenation, where two vectors are joined together to form a new, longer vector. Alternatively, feature fusion can be performed through a learning model, such as using a neural network model that takes global and local features as input and outputs a new vector that combines information from both.

[0081] In some embodiments, when extracting global features, statistical information of the trajectory scene data, such as total length, maximum and minimum coordinate values, average speed, etc., can be calculated. This information constitutes part of the global feature vector. When extracting local features, a sliding window method can be used to analyze different segments of the trajectory, calculate the dynamic features of each segment, such as curvature, velocity change rate, etc., and then summarize these local feature vectors. Finally, the global feature vector and the local feature vector can be spliced together, and the global feature vector and the local feature vector can be used as input to output a comprehensive trajectory scene vector.

[0082] In some embodiments, after obtaining a trajectory scene vector, it can be stored in a vector database, such as Milvus, along with relevant scalar information, such as the trajectory ID and timestamp, for easy association during retrieval. During retrieval, a desired scene vector can be provided, and the vector database will find the closest scene vector to it, thereby locating the corresponding trajectory scene data, enabling efficient retrieval of similar trajectories.

[0083] In the above embodiment, by vectorizing and fusing the global and local features of the trajectory data, the resulting trajectory scene vector not only retains the basic information of the trajectory, but also contains rich dynamic and behavioral characteristics, making it possible to apply it to trajectory analysis, anomaly detection, behavior prediction, and other aspects in autonomous driving scenarios.

[0084] In an exemplary embodiment, determining the global feature vector and the local feature vector of the trajectory scene data includes: performing trajectory feature extraction on the trajectory scene data, determining the trajectory features extracted from the trajectory scene data as the global feature vector corresponding to the trajectory scene data, and splitting the trajectory scene data to obtain multiple trajectory split data; performing trajectory feature extraction on the multiple trajectory split data, and determining the trajectory features extracted from the multiple trajectory split data as the local feature vector of the trajectory scene data.

[0085] It should be noted that the global feature vector is a vector representation of the global feature. The global feature can characterize the macro properties of the trajectory. The global features can include the coordinates of the starting and ending points of the trajectory, the trajectory length, the trajectory duration, the coverage area of the trajectory, the direction change of the trajectory, the time series properties of the trajectory, etc.

[0086] When segmenting trajectory scene data, the entire trajectory scene dataset can be split into multiple smaller segments. These segments are called trajectory segmentation data. Specifically, the trajectory scene data can be segmented into multiple smaller trajectory segments to obtain trajectory segmentation data. When segmenting trajectory scene data, a predefined segmentation strategy can be used. The segmentation strategy can be based on a fixed-length time window, a fixed number of trajectory points, or certain predefined events (such as turns and lane changes). Feature extraction is performed again for each trajectory segmentation data.

[0087] A local feature vector is a vector representation of local features. Local features focus on the specific details of a trajectory within a short period of time or a small range. Local features include but are not limited to the speed change of the trajectory and the relationship between the trajectory and the static environment: for example, the distance from the roadside and the proximity to the intersection.

[0088] The extraction of local feature vectors usually requires more fine-grained data processing, such as support vector machines (SVM), random forests (RF), or neural networks (NN), to analyze the features of each segment and encode them into vectors.

[0089] In the above examples, the extraction of global and local feature vectors aims to construct a comprehensive trajectory scene vector that not only captures the macroscopic characteristics of the trajectory but also captures specific dynamic details. This is particularly important for autonomous driving data management, as it allows the system to retrieve and analyze trajectory data based not only on location information but also on more complex factors such as driving style and road condition adaptability, thereby improving the accuracy of subsequent data queries.

[0090] In an exemplary embodiment, after determining the scene vectors corresponding to the multiple driving tasks, the method further includes: obtaining a pre-built vector database; wherein the vector database includes at least a vector field and a scalar field; writing the scene vectors corresponding to the multiple driving tasks into the vector field in the vector database; and writing a data identifier associated with the scene vector into the scalar field in the vector database, wherein the data identifier is used to distinguish the scene data corresponding to the scene vector. It should be noted that if the i-th message does not exist among the N sorted messages, that is, all messages have been fused, the loop is exited, and the reference message generated by the last iteration is determined to be the final fused message.

[0091] It is understandable that after completing the vectorization processing of the scene data, the scene vectors corresponding to multiple driving tasks are obtained. These vectors can then be stored in a specially designed vector database for efficient retrieval and analysis.

[0092] It should be noted that scene vectors can be stored by accessing an existing vector database. For example, a vector database like Milvus can efficiently handle the storage of large vector sets and approximate nearest neighbor (ANN) retrieval to meet the needs of autonomous driving data platforms.

[0093] A vector database can include at least two types of fields: Vector fields: used to store converted scene vectors, which are the core component of the database. Scalar fields: used to store scalar information associated with scene vectors, namely metadata, such as data identifiers, timestamps, locations, and driving mission IDs. This information is crucial for understanding context and locating results during retrieval.

[0094] Specifically, the scene vectors for the multiple driving tasks identified are written into the vector fields of the vector database, which is the primary data storage operation. Simultaneously, data identifiers closely associated with each scene vector, such as the scene's unique ID and data version, are written into the database's scalar fields to quickly locate specific scene data during retrieval.

[0095] In the above embodiment, not only the storage and retrieval process of autonomous driving data is optimized, but also a more intuitive and efficient data query method is provided for data analysts, which helps to deeply understand the scene details in the driving task.

[0096] Obviously, the embodiments described above are only part of the embodiments of the present invention, rather than all the embodiments. In order to better understand the above method, the above process is described below in conjunction with the embodiments, but it is not intended to limit the technical solutions of the embodiments of the present invention. Specifically:

[0097] In this application, a vector database is introduced to perform vectorized feature extraction on unstructured data. After embedding (vectorization processing), the scalars and vectors are synchronously stored in the vector database Milvus (an open source vector database management system) according to the file type and the business attributes related to the associated data set. These operations are all asynchronously processed and will not affect the original process. In terms of data retrieval, in addition to retrieving from a relational database, a RAG (Retrieval-Augmented Generation) method is introduced to recall the data from the vector database, and then the business information of the relational database is merged based on the information returned by the vector database to realize new user retrieval methods such as image query, image query, point cloud query of data set or data frame, POI (Point of Interest) query of data set, etc.

[0098] refer to Figure 3 The following is the overall flow chart of this application, which specifically includes the following steps:

[0099] Data preprocessing S301: Dataset uploading operation, that is, converting the format of the collected scene data. Specifically, in the annotation platform dataset management function, select the uploaded scene data and upload it. The data package uploaded to the object storage / cloud storage is first downloaded and decompressed to a temporary directory. Then, according to the data folder in the dataset directory, the parameter json file (which contains the file mapping relationship, trajectory, internal and external parameters, distortion parameters, etc. for each frame) is downloaded. First, the conversion is performed according to the directory and structure defined by the platform: the image file name corresponding to the json relationship is renamed and copied to the folder directory of the corresponding camera; the point cloud file in the original point cloud directory is copied to a preset folder; the json configuration file is converted according to the defined format; if the dataset has an answer (true value label), the answer is converted to the platform format and assembled into a memory object for storage in the next step; the true value label on the point cloud is converted to 2D projection transformation and then processed; then the point cloud is assembled according to the file and parameter relationship of each frame.

[0100] Furthermore, for scene data of different data types, upload is carried out. In S302, it is determined whether the data type of the scene data is a picture (i.e. corresponding to image prenatal examination data) / point cloud (i.e. corresponding to point cloud scene data). If the data type is a picture or a point cloud, S305 is used to upload the processed data in the platform format to the storage file table A. If the data type is neither a picture nor a point cloud, S303 is entered. In S303, it is determined whether the data type is not a track. If the data type is a track (i.e. corresponding to track scene data), S306 is used to upload the processed data in the platform format to the storage file table B. If the data type is not a track, S304 is entered: other processes are executed. During the upload process, thumbnails in three formats, large, medium and small, can be generated based on the picture and uploaded to the object storage; 2D thumbnails can be generated for point cloud files and uploaded to the object storage; all files are stored in the file table according to their format and object storage path and other information.

[0101] Furthermore, for scene data whose data type is pictures or point clouds, enter S308 for processing; for scene data whose data type is trajectories, enter S309 for processing; in S308, feature extraction and vectorization are performed on pictures / point clouds, and in S309, feature extraction and vectorization are performed on trajectories to meet the needs of semantic queries.

[0102] For the image processing process, you can refer to Figure 4 The flowchart shown, Figure 4 The following steps may be included: S401: Acquire image scene data; S402: Perform text description on the image scene data to obtain text data; S403: Perform feature extraction on the image scene data, which may be accomplished through a convolutional neural network (CNN). The extracted features will be encoded into a vector of a fixed length, namely the image vector in S405. S404: Perform feature extraction on the text data, and use a text embedding model such as BERT to convert the text into a text vector in S406. Fusion of the image vector and the text vector can produce a more comprehensive and semantic representation, namely the image scene vector. Finally, through S407: the image scene vector can be stored in the vector database.

[0103] For the point cloud processing process, the point cloud scene data can be converted into a point cloud to bev image, that is, converted into point cloud image data, and then vectorized according to the processing method of the image scene data to obtain the point cloud image vector and point cloud text vector.

[0104] For trajectory processing, the trajectory scene data can be preprocessed (to remove some noise), then split into multiple trajectory segmentation data. Global feature extraction is performed based on the trajectory scene data to obtain a global feature vector, and local feature extraction is performed based on the trajectory segmentation data to obtain local feature vectors. Finally, the local feature vectors and the global feature vectors can be combined to obtain a high-dimensional feature vector that represents the trajectory trend of the trajectory scene data.

[0105] Furthermore, the obtained high-dimensional feature vector can be subjected to dimensionality reduction processing, which is achieved by using b-NN (binary neural network). b-NN can map the 118-dimensional vector into a 32-dimensional t-vector (trajectory feature vector representing the trajectory data characteristics). The structure of b-NN is as follows Figure 6 As shown in , l1, l2…lm, g1, g2…gn represent high-dimensional feature vectors. The b-NN has 7 fully connected hidden layers, each with 128 nodes. To obtain an embedding vector with better classification performance, a bottleneck layer is located between the output layer and the last hidden layer and contains 16 nodes. The last layer of the b-NN is the classification layer, which contains four nodes, representing passenger planes, drones, birds, and other targets respectively. During training, all supporting data are used for training, and the learning rate is 0.01. When generating a t-vector, the last output layer is removed, and the local and global features corresponding to each trajectory are used as input, and then a t-vector is output. That is, each t-vector corresponds to a trajectory sample.

[0106] After the images, point clouds and trajectories are vectorized and their corresponding vectors are obtained, they can be stored in the vector database, that is, corresponding to Figure 3 In S307: update the file table vector-associated business field A, and S310: update the file table vector-associated business field B. Specifically, the vector database Milvus can be used to store vectors. Milvus is powerful, has good performance, and has an active open source community. All vectors can be streamed into the database and connected to Milvus using the source->sink method. This way, the throughput of asynchronous + streaming operations will be very high. In addition, the horizontal scalability of Milvus can also support the problems caused by the increase in business volume. The schema design of the vector database adopts the characteristics of scalar and vector association. The advantage is that it breaks through the defects of some vector databases that only have default primary keys and vector storage, associates some business attributes, and directly locates some related data of the platform business.

[0107] When building the Milvus vector database, you first create the database schema, define an auto-incrementing primary key (ID), define a dataset ID with scalar business attributes, and define vector fields. You can also add other scalar fields to expand query relevance. Next, you create an index. You can choose IVF_FLAT (inverted file index), a quantized index suitable for scenarios that require an ideal balance between accuracy and speed (high-speed queries and high recall). If memory resources are sufficient, you can choose HNSW (graph-based index), suitable for scenarios that require high query efficiency (high-speed queries, the highest possible recall, and large memory resources). Furthermore, you can create collections (relational database tables). Each data type will be created in a collection. Finally, you insert the vector into the Milvus database. All of these operations are asynchronous multi-threaded during the dataset upload process and do not affect the overall workflow.

[0108] When all data are stored in the vector database, data retrieval can be carried out, that is, data query S311: based on the target driving task to be executed, query the target driving data. The flowchart of data query is as follows: Figure 7 As shown, the method specifically includes the following steps: S701, determining the data to be queried, that is, determining the expected scene data related to the target driving task; S702, determining the data type of the expected scene data; S703: based on the data type, vectorizing the expected scene data; S704: obtaining the vector to be queried based on S703; S705: determining at least one target scene vector matching the vector to be queried by searching in a vector database; S706: determining a target data identifier associated with the target scene vector; S707: matching the target data identifier with a data identifier in a pre-constructed task database based on the target data identifier; S708: determining the scene data corresponding to the data identifier in the business database as the target scene data of the target driving task.

[0109] In order to verify the data query efficiency of this application, the data query process was tested and the test data was recorded. Figure 8 As shown in the figure, throughput (queries per second) and search precision (recalls) are inversely proportional to the number of probes. For a dataset of 100,000 clusters, searching only the 100 most recent clusters for each query results in 96% recall, while searching 1,000 clusters (1% of the dataset) results in 99.8% accuracy. A balance between throughput and recall is crucial, and extensive testing can yield a good empirical value.

[0110] The method for determining scene data of the present application can apply vectorization and the storage and retrieval of vector databases to the data set storage and retrieval of the autonomous driving data platform, and realize the vectorization processing of multi-modal files. An index retrieval method is designed for the characteristics of various modal files, and the transition from vector similarity query to business attribute retrieval is realized, which also provides good scalability in terms of performance. In this way, a large amount of unstructured data can be quickly represented by vectors, and fast retrieval queries, image query, image query of data set / data frame, trajectory query of data set / data frame, point cloud query of data set / data frame, and query performance of small and medium data sets in seconds can be achieved.

[0111] Through the description of the above implementation methods, those skilled in the art can clearly understand that the method according to the above embodiment can be implemented by means of software plus the necessary general hardware platform, and of course it can also be implemented by hardware, but in many cases the former is a better implementation method. Based on this understanding, the technical solution of the present application, or the part that contributes to the prior art, can be embodied in the form of a software product, which is stored in a storage medium (such as ROM / RAM, magnetic disk, optical disk), and includes a number of instructions for enabling a terminal device (which can be a mobile phone, computer, server, or network device, etc.) to execute the methods of each embodiment of the present application.

[0112] In this embodiment, a device for determining scene data is also provided, which is used to implement the above-mentioned embodiments and preferred embodiments. Details that have already been described will not be repeated here. As used below, the term "module" may refer to a combination of software and / or hardware that implements a predetermined function. Although the devices described in the following embodiments are preferably implemented in software, implementation using hardware, or a combination of software and hardware, is also possible and contemplated.

[0113] Figure 9 : is a structural block diagram of a device for determining scene data according to an embodiment of the present application, the device comprising:

[0114] Determination module 902 is configured to, when a target driving task to be performed on a target vehicle exists, determine an expected scene vector corresponding to the target driving task, and determine scene vectors corresponding to each of the plurality of driving tasks; wherein the expected scene vector is obtained by vectorizing expected scene data corresponding to the target driving task, the target driving task being any one of the plurality of driving tasks; and the plurality of scene vectors are obtained by vectorizing scene data pre-collected for the plurality of driving tasks;

[0115] A second determining module 904 is configured to determine at least one target scene vector matching the expected scene vector from the multiple scene vectors;

[0116] The third determining module 906 is configured to determine target scene data for the target driving task according to the at least one target scene vector.

[0117] The above-mentioned device, when a target driving task to be performed on a target vehicle exists, determines an expected scene vector corresponding to the target driving task, and determines scene vectors corresponding to multiple driving tasks, respectively, wherein the scene vectors are obtained by vectorizing scene data pre-collected for the multiple driving tasks; and determines at least one target scene vector that matches the expected scene vector from the multiple scene vectors. Since the target scene vector is a vector that matches the expected scene vector among the multiple scene vectors, and the expected scene vector is obtained by vectorizing the expected scene data corresponding to the target driving task, the target scene data determined by at least one target scene vector must be related to the target driving task, thereby effectively improving the data search accuracy.

[0118] In an exemplary embodiment, the first determination module 902 is further used to obtain scene data pre-collected for the multiple driving tasks; based on the data type of the scene data, determine a coding strategy that matches the scene data; wherein the coding strategy is used to vectorize the scene data; for each driving task in the multiple driving tasks, based on the coding strategy that matches the scene data of the driving task, vectorize the scene data of the driving task to obtain a scene vector for the driving task.

[0119] In an exemplary embodiment, the first determination module 902 is further used to, when the scene data of the driving task includes image scene data, perform image feature extraction on the image scene data, determine the image features extracted from the image scene data as image vectors corresponding to the image scene data, and perform text description on the image scene data to generate text data of the image scene data; perform text feature extraction on the text data, determine the text features extracted from the text data as text vectors corresponding to the text data; and fuse the image vector and the text vector to obtain the image scene vector of the image scene data.

[0120] In an exemplary embodiment, the first determination module 902 is further used to, when the scene data targeted includes point cloud scene data, perform image conversion on the point cloud scene data to map the point cloud scene data to a pre-divided plane grid to construct point cloud image data; perform image feature extraction on the point cloud image data, determine the image features extracted from the point cloud image data as a point cloud image vector corresponding to the point cloud image data, and perform text feature extraction on the point cloud image data, determine the text features extracted from the point cloud image data as a point cloud text vector corresponding to the point cloud image data; and fuse the point cloud image vector and the point cloud text vector to determine the point cloud scene vector of the point cloud scene data.

[0121] In an exemplary embodiment, the first determination module 902 is further configured to, when the targeted scene data includes trajectory scene data, determine a global feature vector and a local feature vector of the trajectory scene data; and fuse the global feature vector and the local feature vector to obtain a trajectory scene vector of the trajectory scene data.

[0122] In an exemplary embodiment, the first determination module 902 is further configured to perform trajectory feature extraction on the trajectory scene data, determine the trajectory features extracted from the trajectory scene data as global feature vectors corresponding to the trajectory scene data, and split the trajectory scene data to obtain a plurality of trajectory split data; perform trajectory feature extraction on the plurality of trajectory split data, and determine the trajectory features extracted from the plurality of trajectory split data as local feature vectors of the trajectory scene data.

[0123] In an exemplary embodiment, the device also includes a storage module; the storage module is used to obtain a pre-built vector database; wherein the vector database includes at least a vector field and a scalar field; the scene vectors corresponding to the multiple driving tasks are written into the vector field in the vector database, and the data identifier associated with the scene vector is written into the scalar field in the vector database, wherein the data identifier is used to distinguish the scene data corresponding to the scene vector.

[0124] In an exemplary embodiment, the third determination module 906 is further used to determine a target data identifier associated with the at least one target scene vector; based on the target data identifier, the target data identifier is matched with a data identifier in a pre-built task database to determine a data identifier that matches the target data identifier; wherein the task database includes multiple data identifiers and scene data corresponding to the multiple data identifiers; the data identifier is used to distinguish the scene data corresponding to the scene vector; and the scene data corresponding to the data identifier is determined as the target scene data of the target driving task.

[0125] An embodiment of the present application further provides a computer-readable storage medium, in which a computer program is stored. The computer program is configured to execute the steps of any of the above method embodiments when run.

[0126] Optionally, in this embodiment, the storage medium may be configured to store a computer program for performing the following steps:

[0127] S1. When a target driving task to be performed is present for a target vehicle, determining an expected scene vector corresponding to the target driving task, and determining scene vectors corresponding to each of a plurality of driving tasks; wherein the expected scene vector is obtained by vectorizing expected scene data corresponding to the target driving task, the target driving task being any one of the plurality of driving tasks; and the plurality of scene vectors are obtained by vectorizing scene data pre-collected for the plurality of driving tasks;

[0128] S2, determining at least one target scene vector matching the expected scene vector from the multiple scene vectors;

[0129] S3: Determine target scene data of the target driving task according to the at least one target scene vector.

[0130] In an exemplary embodiment, the computer-readable storage medium may include, but is not limited to, various media that can store computer programs, such as a USB flash drive, a read-only memory (ROM), a random access memory (RAM), a mobile hard disk, a magnetic disk, or an optical disk.

[0131] For specific examples in this embodiment, reference may be made to the examples described in the above embodiments and exemplary implementation modes, and this embodiment will not be described in detail here.

[0132] An embodiment of the present application further provides a computer program product, including a computer program, and the computer program performs the steps of any of the above method embodiments when executed by a processor.

[0133] An embodiment of the present application further provides an electronic device, comprising a memory and a processor, wherein the memory stores a computer program, and the processor is configured to run the computer program to execute the steps in any one of the above method embodiments.

[0134] Optionally, in this embodiment, the processor may be configured to execute the following steps through a computer program:

[0135] S1. When a target driving task to be performed is present for a target vehicle, determining an expected scene vector corresponding to the target driving task, and determining scene vectors corresponding to each of a plurality of driving tasks; wherein the expected scene vector is obtained by vectorizing expected scene data corresponding to the target driving task, the target driving task being any one of the plurality of driving tasks; and the plurality of scene vectors are obtained by vectorizing scene data pre-collected for the plurality of driving tasks;

[0136] S2, determining at least one target scene vector matching the expected scene vector from the multiple scene vectors;

[0137] S3: Determine target scene data of the target driving task according to the at least one target scene vector.

[0138] In an exemplary embodiment, the electronic device may further include a transmission device and an input / output device, wherein the transmission device is connected to the processor, and the input / output device is connected to the processor.

[0139] For specific examples in this embodiment, reference may be made to the examples described in the above embodiments and exemplary implementation modes, and this embodiment will not be described in detail here.

[0140] Obviously, those skilled in the art should understand that the modules or steps of the present application described above can be implemented using a general-purpose computing device, they can be concentrated on a single computing device, or distributed across a network composed of multiple computing devices, they can be implemented using program code executable by the computing device, and thus, they can be stored in a storage device and executed by the computing device, and in some cases, the steps shown or described can be performed in a different order than herein, or they can be fabricated into separate integrated circuit modules, or multiple modules or steps can be fabricated into a single integrated circuit module for implementation. Thus, the present application is not limited to any specific combination of hardware and software.

[0141] The above is only a preferred embodiment of the present application. It should be pointed out that for ordinary technicians in this technical field, several improvements and modifications can be made without departing from the principles of the present application. These improvements and modifications should also be regarded as the scope of protection of the present application.

Claims

1. A method for determining scene data, characterized in that: include: When a target driving task to be performed is present for a target vehicle, determining an expected scene vector corresponding to the target driving task, and determining scene vectors corresponding to each of the plurality of driving tasks; wherein the expected scene vector is obtained by vectorizing expected scene data corresponding to the target driving task, the target driving task being any one of the plurality of driving tasks; and the plurality of scene vectors are obtained by vectorizing scene data pre-collected for the plurality of driving tasks; determining at least one target scene vector from the plurality of scene vectors that matches the desired scene vector; Target scene data of the target driving task is determined according to the at least one target scene vector.

2. The method according to claim 1, characterized in that Determine the scene vectors corresponding to multiple driving tasks, including: Acquiring pre-collected scene data for the plurality of driving tasks; Determining, based on a data type of the scene data, a coding strategy that matches the scene data; wherein the coding strategy is used to perform vectorized processing on the scene data; For each driving task among the multiple driving tasks, based on a coding strategy that matches the scene data of the driving task, vectorization processing is performed on the scene data of the driving task to obtain a scene vector of the driving task.

3. The method according to claim 2, characterized in that The step of performing vectorization processing on the scene data of the targeted driving task based on the encoding strategy matching the scene data of the targeted driving task to obtain the scene vector of the targeted driving task includes: In a case where the scene data of the targeted driving task includes image scene data, performing image feature extraction on the image scene data, determining the image features extracted from the image scene data as image vectors corresponding to the image scene data, and performing text description on the image scene data to generate text data of the image scene data; Performing text feature extraction on the text data, and determining the text features extracted from the text data as text vectors corresponding to the text data; The image vector and the text vector are fused to obtain an image scene vector of the image scene data.

4. The method according to claim 2, characterized in that The step of performing vectorization processing on the scene data of the targeted driving task based on the encoding strategy matching the scene data of the targeted driving task to obtain the scene vector of the targeted driving task includes: In a case where the targeted scene data includes point cloud scene data, performing image conversion on the point cloud scene data to map the point cloud scene data into a pre-divided plane grid to construct point cloud image data; performing image feature extraction on the point cloud image data, determining the image features extracted from the point cloud image data as a point cloud image vector corresponding to the point cloud image data, and performing text feature extraction on the point cloud image data, determining the text features extracted from the point cloud image data as a point cloud text vector corresponding to the point cloud image data; The point cloud image vector and the point cloud text vector are fused to determine a point cloud scene vector of the point cloud scene data.

5. The method according to claim 2, characterized in that The step of performing vectorization processing on the scene data of the targeted driving task based on the encoding strategy matching the scene data of the targeted driving task to obtain the scene vector of the targeted driving task includes: In a case where the targeted scene data includes trajectory scene data, determining a global feature vector and a local feature vector of the trajectory scene data; The global feature vector and the local feature vector are fused to obtain a trajectory scene vector of the trajectory scene data.

6. The method according to claim 5, characterized in that The determining of the global feature vector and the local feature vector of the trajectory scene data includes: performing trajectory feature extraction on the trajectory scene data, determining the trajectory features extracted from the trajectory scene data as global feature vectors corresponding to the trajectory scene data, and splitting the trajectory scene data to obtain a plurality of trajectory split data; Trajectory features are extracted from the plurality of trajectory segmentation data, and the trajectory features extracted from the plurality of trajectory segmentation data are determined as local feature vectors of the trajectory scene data.

7. The method according to claim 1, characterized in that After determining the scene vectors corresponding to the plurality of driving tasks, the method further includes: Obtaining a pre-built vector database; wherein the vector database includes at least a vector field and a scalar field; The scene vectors corresponding to the multiple driving tasks are written into the vector field in the vector database, and the data identifier associated with the scene vector is written into the scalar field in the vector database, wherein the data identifier is used to distinguish the scene data corresponding to the scene vector.

8. The method according to claim 1, characterized in that The determining, based on the at least one target scene vector, target scene data for the target driving task includes: determining a target data identifier associated with the at least one target scene vector; Based on the target data identifier, matching is performed with data identifiers in a pre-built task database to determine a data identifier that matches the target data identifier; wherein the task database includes multiple data identifiers and scene data corresponding to the multiple data identifiers; the data identifier is used to distinguish the scene data corresponding to the scene vector; The scene data corresponding to the data identifier is determined as the target scene data of the target driving task.

9. A device for determining scene data, characterized in that: include: A first determination module is configured to, when a target driving task to be performed on a target vehicle exists, determine an expected scene vector corresponding to the target driving task, and determine scene vectors corresponding to each of a plurality of driving tasks; wherein the expected scene vector is obtained by vectorizing expected scene data corresponding to the target driving task, the target driving task being any one of the plurality of driving tasks; and the plurality of scene vectors are obtained by vectorizing scene data pre-collected for the plurality of driving tasks; The second determining module is further configured to determine at least one target scene vector matching the expected scene vector from the multiple scene vectors; The third determination module is further configured to determine target scene data for the target driving task based on the at least one target scene vector.

10. A computer-readable storage medium, characterized in that: The computer-readable storage medium includes a stored program, wherein the program executes the method according to any one of claims 1 to 8 when executed.

11. An electronic device comprising a memory and a processor, characterized in that: A computer program is stored in the memory, and the processor is configured to execute the method according to any one of claims 1 to 8 through the computer program.

Images