Scene library building method and device, equipment and medium

By collecting and storing vehicle driving information and environmental information in the intelligent driving system, establishing scene description files, and automatically managing intelligent driving data, the problems of low data management efficiency and accuracy in the existing technology are solved, and efficient data processing and use.

CN119028136APending Publication Date: 2024-11-26SINO TRUK JINAN POWER CO LTD
View PDF 0 Cites 0 Cited by

Patent Information

Application Number
CN202411222708.1
Authority / Receiving Office
CN · China
Patent Type
Applications(China)
Current Assignee / Owner
Filing Date
2024-09-02
Publication Date
2024-11-26

AI Technical Summary

Technical Problem

The degree of automated processing of existing intelligent driving data is low, which leads to difficulty in data screening and insufficient recording, and requires manual management, resulting in low data management efficiency and accuracy.

Method used

The vehicle's driving information and environment information are collected through the server, stored as CAN data files and video files, and scene description files are established, and relevant CAN data files and video files are obtained from the scene description files according to the triggering instructions to realize automated management.

Benefits of technology

It improves the management efficiency and accuracy of intelligent driving data, realizes automated processing, reduces manual intervention, and improves data utilization efficiency.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN119028136A_ABST
    Figure CN119028136A_ABST
Patent Text Reader

Abstract

The invention provides a scene library building method and device, equipment and a medium. The method comprises the steps that a server obtains driving information and environment information of a vehicle through an acquisition terminal, the driving information is stored as a CAN data file, and the environment information is stored as a video file; establishing a scene description file according to the driving information and the environment information; if a trigger instruction of a client side is received, the server obtains the CAN data file and the video file according to the scene description file and sends the CAN data file and the video file to the client side, and the trigger instruction comprises scene keywords. According to the method, automatic processing of the intelligent driving data can be realized, and the management efficiency and accuracy of the intelligent driving data are improved.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present application relates to the field of intelligent driving technology, and in particular to a method, device, equipment and medium for building a scene library. Background Art

[0002] The intelligent driving system uses technologies such as sensors, artificial intelligence and machine learning to assist drivers in driving, so as to improve driving safety, traffic efficiency and user experience. The intelligent driving function will generate a large amount of data during operation, and optimizing the processing of this data is to optimize the intelligent driving system.

[0003] In the prior art, the methods and tools for collecting intelligent driving data are relatively complete. However, the degree of automation in using the collected intelligent driving data is low. Engineers are required to place the collected intelligent driving data on different hard disks and manage them manually in a decentralized manner, which leads to difficulties in data screening and insufficient data recording.

[0004] Therefore, this application proposes a scene library construction method to solve the above problems. Summary of the invention

[0005] The embodiments of the present application provide a scenario library construction method, device, equipment and medium to achieve automated processing of intelligent driving data and improve the efficiency and accuracy of intelligent driving data management.

[0006] In a first aspect, the present application provides a method for building a scene library, comprising:

[0007] The server obtains the driving information and environmental information of the vehicle through the acquisition terminal, and stores the driving information as a CAN data file and stores the environmental information as a video file;

[0008] Establishing a scene description file according to the driving information and the environmental information;

[0009] If a trigger instruction from the client is received, the server obtains the CAN data file and the video file according to the scene description file and sends them to the client, wherein the trigger instruction includes a scene keyword.

[0010] In a possible implementation, storing the driving information as a CAN data file and storing the environment information as a video file includes:

[0011] storing different types of driving information at the scene triggering moment as CAN data files, wherein the types of driving information include vehicle status, intelligent driving function working status and driver control signals;

[0012] The different types of environmental information at the scene triggering moment are stored as video files, wherein the types of environmental information include weather information, geographical location information, obstacle information and road information.

[0013] In a possible implementation manner, the establishing a scene description file according to the driving information and the environment information includes:

[0014] Extracting scene keywords according to the driving information and the environmental information, wherein the scene keywords include weather information, obstacle information and vehicle status;

[0015] Establishing a first key description of the CAN data file related to the scene keyword and a storage location of the CAN data file in the scene description file;

[0016] A second key description of the video file associated with the scene keyword and a storage location of the video file are established in the scene description file.

[0017] In a possible implementation, acquiring the CAN data file and the video file according to the scene description file and sending them to the client includes:

[0018] In the scene description file, obtaining a second key description related to the scene keyword in the trigger instruction, and obtaining a target video file related to the second key description;

[0019] Acquire a collection time period of data in the target video file, and acquire the CAN data file in which the description time period in the first key description is the same as the collection time period, to obtain a target CAN data file;

[0020] The target video file and the target CAN data file are sent to the client from the storage location of the target video file and the storage location of the target CAN data file.

[0021] In a possible implementation, the server obtains the driving information and environmental information of the vehicle through the acquisition terminal, including:

[0022] The server obtains the driving information and the environmental information of the vehicle through the active mode and / or passive mode of the acquisition terminal; wherein the passive mode is a working mode for collecting the driving information and the environmental information by triggering a scene according to a preset condition; and the active mode is a working mode for collecting the driving information and the environmental information by triggering a scene according to a target detection and tracking model.

[0023] In a possible implementation, the server is connected to the acquisition terminal via MQTT network communication, and the server is connected to the client via the MQTT network communication;

[0024] If a first topic is defined according to the MQTT identifier of the server and the MQTT identifier of the receiving end, and the collection terminal is set to subscribe to the first topic, the server is the sending end, and the collection terminal is the receiving end;

[0025] If a second topic is defined according to the MQTT identifier of the collection terminal and the MQTT identifier of the receiving end, and the server is set to subscribe to the second topic, the server is the receiving end, and the collection terminal is the sending end.

[0026] In a possible implementation, if a trigger instruction and a refill instruction from the client are received, the method further includes:

[0027] The server obtains the CAN data file and the video file according to the scenario description file, and generates an intelligent driving simulation algorithm corresponding to the data volume of the CAN data file and the video file;

[0028] The intelligent driving simulation algorithm is used to perform multi-threaded synchronous simulation on the data of the CAN data file and the video file to confirm whether there is a thread anomaly.

[0029] In a second aspect, the present application provides a scene library building device, comprising:

[0030] The first processing module is used for the server to obtain the driving information and environmental information of the vehicle through the acquisition terminal, and store the driving information as a CAN data file and store the environmental information as a video file;

[0031] A second processing module, used for establishing a scene description file according to the driving information and the environmental information;

[0032] The transmission module is used for, if receiving a trigger instruction from the client, the server obtains the CAN data file and the video file according to the scene description file and sends them to the client, wherein the trigger instruction includes a scene keyword.

[0033] In a third aspect, the present application provides a scene library building device, including: at least one processor and a memory;

[0034] The memory stores computer-executable instructions;

[0035] The at least one processor executes the computer-executable instructions stored in the memory, so that the at least one processor executes the scene library building method as described above.

[0036] In a fourth aspect, the present application provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the steps of the scene library building method as described above.

[0037] The embodiments of the present application provide a method, device, equipment and medium for building a scene library. The server obtains driving information and environmental information of a vehicle through a collection terminal, and stores the driving information as a CAN data file, and stores the environmental information as a video file; a scene description file is established according to the driving information and the environmental information; if a trigger instruction from a client is received, the server obtains the CAN data file and the video file according to the scene description file and sends them to the client, wherein the trigger instruction includes a scene keyword.

[0038] In the above method, the intelligent driving data includes driving information and environmental information, the collection terminal is responsible for collecting driving information and environmental information, the server is responsible for storing driving information and environmental information, and the client is responsible for downloading and displaying driving information and environmental information from the server; the server can respectively establish CAN data files and video files, as well as scene description files according to the driving information and environmental information obtained from the collection terminal, wherein the scene description file has a general description related to the driving information and environmental information, and the CAN data file and the video file have specific data content of the driving information and environmental information, and the specific data content can be found according to the general description; if the client has the need to find relevant data content according to a certain scene keyword, it can issue a trigger instruction, so that the server can find the corresponding CAN data file and video file according to the scene description file, and feed it back to the client; it realizes the management of intelligent driving data in an automated manner, thereby improving the management efficiency and accuracy of intelligent driving data. BRIEF DESCRIPTION OF THE DRAWINGS

[0039] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative labor.

[0040] Figure 1 A schematic diagram of a scene library built in an embodiment of the present application;

[0041] Figure 2 A schematic diagram of a method for building a scene library provided in an embodiment of the present application Figure 1 ;

[0042] Figure 3A schematic diagram of a method for building a scene library provided in an embodiment of the present application Figure 2 ;

[0043] Figure 4 A schematic diagram of a method for building a scene library provided in an embodiment of the present application Figure 3 ;

[0044] Figure 5 A schematic diagram of a method for building a scene library provided in an embodiment of the present application Figure 4 ;

[0045] Figure 6 A diagram of a scene library construction device provided in an embodiment of the present invention;

[0046] Figure 7 A hardware schematic diagram of a scene library building device provided in an embodiment of the present invention. DETAILED DESCRIPTION

[0047] In order to make the purpose, technical solution and advantages of the embodiments of the present invention clearer, the technical solution in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0048] Intelligent driving is increasingly being used in automobiles. Compared with traditional driving systems, it can better assist drivers in driving, and has advantages such as improving driving safety, traffic efficiency, and user experience. The application of intelligent driving also generates a large amount of data during driving. The management and use of this data is less automated, so the data cannot be used efficiently.

[0049] For example, in the prior art, based on the existing methods and tools for intelligent driving data collection, the ability to collect complete intelligent driving data has gradually been developed; however, in terms of using the collected data, engineers usually store the collected data in independent hard disks, and the scattered data is highly difficult to manage and maintain manually; this will lead to difficulties in scene data screening and insufficient data records, among which the deficiencies in data records include unnecessary duplicate records and omissions of key data.

[0050] In order to improve the automated processing of data, this application proposes a scenario library construction method to solve the above problems.

[0051] The following is a detailed description of the implementation process of a scene library building method proposed in this application in conjunction with the accompanying drawings and specific embodiments.

[0052] Figure 1 A schematic diagram of a scene library provided in an embodiment of the present application. Figure 1 As shown, the system includes: a collection terminal 101, a server 102, a client 103 and a database 104; wherein the collection terminal 101 is used to be set on a vehicle, and the location and number of the collection terminal 101 can be determined according to actual needs; the data collected by the collection terminal 101 can be transmitted to the server 102, and the data received by the server 102 can be displayed on the client 103; the server 102 can be a data-driven server, and the data received by the server 102 can be stored in the database 104, and the database 104 can be set inside the server 102; the client 103 includes a desktop display;

[0053] The collection terminal 101, the server 102 and the client 103 can be connected through a message queue telemetry transport (MQTT) network; MQTT adopts a publish / subscribe mode for message transmission. In this mode, the publisher of the message publishes the message to a specific topic, and the subscriber receives the message by subscribing to the relevant topic; this mode publishes messages one-to-many, decouples the sender and receiver of the message, and facilitates flexible management of communication;

[0054] Intelligent driving data includes vehicle driving information and environmental information; the driving information mainly refers to the vehicle’s own driving information; the environmental information mainly refers to the driving information around the vehicle, such as road information;

[0055] The collection terminal 101 is responsible for collecting intelligent driving data; the server 102 is responsible for storing the intelligent driving data through the database 104, and storing the driving information as a CAN data file and the environmental information as a video file in a form supported by the database 104, and establishing a scene description file based on the driving information and the environmental information; the scene description file is a general description of the CAN data file and the video file, and the CAN data file and the video file store specific intelligent driving data;

[0056] When the client 103 has a query requirement, it can send a trigger instruction to the server 102. After receiving the trigger instruction from the client, the server 102 can query the corresponding CAN data file and video file in the scene description file according to the scene keywords in the trigger instruction, and provide the client with download and display; among them, the scene keywords can be keywords related to driving information and environmental information.

[0057] Figure 2 A schematic diagram of a method for building a scene library provided in an embodiment of the present application Figure 1 .like Figure 2 As shown, the method includes:

[0058] S201. The server obtains driving information and environmental information of the vehicle through a collection terminal, and stores the driving information as a CAN data file, and stores the environmental information as a video file.

[0059] The server includes a front-end and a back-end. The front-end can be developed using the C++ programming language and can interact with users such as engineers, such as through web pages or user interfaces. The back-end can be developed using the Java programming language and is used for business logic processing, database management, and server configuration.

[0060] Intelligent driving data is also the road scene data generated during the intelligent driving process, including driving information and environmental information; the driving information mainly includes the vehicle's own state information during driving, such as the vehicle's speed, engine torque, etc.; the environmental information mainly includes the state information of the surrounding road environment during driving, such as road information and weather information, etc.

[0061] The acquisition terminal can be set at the corresponding part of the vehicle according to needs, and is used to collect the vehicle's driving information and environmental information; the server is responsible for receiving the driving information and environmental information, and storing the driving information as a CAN data file and the environmental information as a video file through the database.

[0062] The acquisition terminal can collect data in different working modes:

[0063] For example, the server obtains the driving information and the environmental information of the vehicle through the active mode and / or passive mode of the acquisition terminal; wherein the passive mode is a working mode for collecting the driving information and the environmental information by triggering a scene according to a preset condition; and the active mode is a working mode for collecting the driving information and the environmental information by triggering a scene according to a target detection and tracking model.

[0064] The acquisition terminal is developed based on the embedded Linux system and uses the high-performance RK3588 processor. It has a hardware configuration of 6 CAN channels, 4 GMSL video channels, and 1 vehicle Ethernet. It also has Wifi6 and high-precision positioning functions.

[0065] The RK3588 processor adopts the advanced 8nm process technology and 8-core architecture, including quad-core 64-bit Cortex-A76, quad-core Cortex-A55, and independent NEON coprocessor. The computing power of its NPU (embedded neural network processor) core is as high as 6TOPS; the terminal has a hardware configuration of 6 CAN channels, 4 GMSL (Gigabit Multimedia Serial Link) video channels, and 1 in-vehicle Ethernet, and also has Wifi6 and high-precision positioning functions.

[0066] The acquisition terminal is set up with active mode and passive mode for information collection. The passive mode is to trigger the scene according to the preset conditions. The preset conditions include the preset vehicle CAN bus data trigger conditions, such as intelligent driving function trigger, driver-specific operations, etc.

[0067] The active mode is triggered by the scene according to the target detection and tracking model. The implementation of this target detection and tracking model depends on the capabilities of the NPU core of the RK3588 processor. The target detection and tracking model can track the target in real time and output the target type, speed, distance and other data. It can also identify road conditions, such as road congestion, tunnels, viaducts, etc. The target detection and tracking model uses the YOLOv8 model and the DeepSORT model. The YOLOv8 model is mainly responsible for target detection, and the DeepSORT model is mainly responsible for target tracking.

[0068] YOLOv8 is one of the versions of the YOLO series of models. The YOLO series of models is a target detection algorithm based on a convolutional neural network. It can simultaneously predict the location and category of multiple targets in a single forward propagation. The main features of YOLOv8 include strong real-time performance, high detection accuracy, and suitability for embedded devices.

[0069] DeepSORT is a target tracking algorithm based on SORT (Simple Online and Realtime Tracking), which combines the feature extraction capability of deep learning. DeepSORT uses a deep learning model to extract the appearance features of the target, and combines Kalman filtering and the Hungarian algorithm for target association and tracking. The main features of DeepSORT include the ability to handle complex scenes such as occlusion and target reappearance, and high tracking accuracy.

[0070] When used in combination with YOLOv8 and DeepSORT, YOLOv8 obtains the target location and category in each frame of the image, and DeepSORT uses the information obtained by YOLOv8 to track the target, generate a unique ID for each target and associate consecutive frames, achieving accurate multi-target tracking.

[0071] S202: Create a scene description file according to the driving information and the environment information.

[0072] The server can also establish a scene description file based on the information in the driving information and the environmental information. The scene description file is used to summarize the basic information in the driving information and the environmental information, including the collection time of the driving information and the corresponding name, while the CAN data file and the video file include the specific data content of the driving information and the environmental information. For example, the scene is vehicle braking, the time and weather when the vehicle brakes, and the path of the CAN data file and the video file of this braking are stored in the scene description file, the engine torque of the vehicle during braking and other data are stored in the CAN data file, and the road video of the vehicle during braking is stored in the video file.

[0073] S203: If a trigger instruction from the client is received, the server obtains the CAN data file and the video file according to the scene description file and sends them to the client, wherein the trigger instruction includes a scene keyword.

[0074] The intelligent driving data stored in the server can be queried by the client so that the client user can quickly find the desired information; when the client queries the data, it can issue a trigger instruction. After the server receives the trigger instruction, it can query the scene description file for relevant CAN data files and video files that match the scene keywords in the trigger instruction, such as bad weather.

[0075] In the embodiment of the present application, the intelligent driving data includes driving information and environmental information. The collection terminal is responsible for collecting driving information and environmental information, the server is responsible for storing driving information and environmental information, and the client is responsible for downloading and displaying driving information and environmental information from the server; the server can respectively establish CAN data files and video files, as well as scene description files based on the driving information and environmental information obtained from the collection terminal, wherein the scene description file has a general description related to the driving information and environmental information, and the CAN data file and video file have specific data content of the driving information and environmental information, and the specific data content can be found according to the general description; if the client has the need to find relevant data content based on a certain scene keyword, it can issue a trigger instruction, so that the server can find the corresponding CAN data file and video file according to the scene description file and feedback to the client; it realizes the management of intelligent driving data in an automated manner, thereby improving the management efficiency and accuracy of intelligent driving data

[0076] Figure 3 A schematic diagram of a method for building a scene library provided in an embodiment of the present application Figure 2 .like Figure 3 As shown, the method includes:

[0077] S301. Store different types of driving information at the scene triggering moment as a CAN data file, wherein the types of driving information include vehicle status, intelligent driving function working status and driver control signal.

[0078] The acquisition device triggers scene acquisition in active mode or passive mode. The scene can be emergency braking, tunnel, etc. The moment when the scene observable in active mode or passive mode occurs is the scene triggering moment. At the scene triggering moment, all kinds of driving information in the scene can be stored as CAN data files by the server.

[0079] S302. Store the different types of environmental information at the scene triggering moment as a video file, wherein the types of environmental information include weather information, geographical location information, obstacle information and road information.

[0080] At the moment of scene triggering, video information can also be collected, and the weather corresponding to the video at the current moment can be collected through the corresponding equipment. The weather information can be marked in the video file; environmental information related to the vehicle's surrounding environment can be stored as a video file by the server;

[0081] For example, the database of the server may be a lightweight embedded (Sqlite3) database operated by Structured Query Language (SQL), and the Sqlite3 database may store various data files collected during the intelligent driving process;

[0082] The files stored in the database are json files, which include data such as the vehicle status at the time of scene triggering, the working status of the intelligent driving function, and the driver's control signal / driver operation. They may also include a list of scene files and weather information, geographic location information, obstacle information, and road information at the time of scene triggering.

[0083] The json files stored in the database mainly include CAN data files, video files and scene description files.

[0084] The scene description file is associated with the CAN data file and the video file:

[0085] For example, scene keywords are extracted according to the driving information and the environmental information, wherein the scene keywords include weather information, obstacle information, and vehicle status;

[0086] Establishing a first key description of the CAN data file related to the scene keyword and a storage location of the CAN data file in the scene description file;

[0087] A second key description of the video file associated with the scene keyword and a storage location of the video file are established in the scene description file.

[0088] The scene keywords are related to the driving information and environmental information that constitute the scene, for example, vehicle braking, vehicle encountering a tunnel, and vehicle encountering weather conditions; the scene keywords are first extracted based on the driving information and environmental information, so that after the scene keywords are triggered, the corresponding driving information and environmental information of the vehicle when the scene keywords are triggered can be collected; the scene description file contains the first key description of the CAN data file, the first key description is such as the vehicle speed of the CAN data file collected at a certain moment; the scene description file contains the second key description of the video file, the second key description is such as the weather conditions of the video file collected at a certain moment; the scene description file contains the storage location of the CAN data file and the video file, which is convenient for subsequent positioning according to the scene keywords.

[0089] The server can index CAN data files and video files through scene description files:

[0090] For example, in the scene description file, a second key description related to the scene keyword in the trigger instruction is obtained, and a target video file related to the second key description is obtained;

[0091] Acquire a collection time period of data in the target video file, and acquire the CAN data file in which the description time period in the first key description is the same as the collection time period, to obtain a target CAN data file;

[0092] The target video file and the target CAN data file are sent to the client from the storage location of the target video file and the storage location of the target CAN data file.

[0093] If the scene keyword has a second key description related to the video file, then the relevant target video file can be found according to the second key description; for example, if the scene keyword has cloudy, then the target video file related to cloudy can be found, and according to the storage location of the target video file related to cloudy, it can be downloaded and displayed to the client; the first key description also has a description time period describing the data acquisition time, and the second key description also has a collection time period describing the data acquisition time, and the target CAN data file with the same collection time as the target video file can also be downloaded at the same time and displayed to the client;

[0094] The client can be developed based on the C++ cross-platform graphical user interface application development framework (QT), and can run across both Windows and Ubuntu platforms. Ubuntu is a Linux operating system that mainly focuses on desktop applications. The client is developed in multiple threads, so that tasks such as video rendering and data analysis can run simultaneously.

[0095] In the embodiment of the present application, based on the database of the server, the driving information and environmental information related to the scene are accurately classified and stored in the form of different files, so that the client can search according to the scene keywords.

[0096] Figure 4 A schematic diagram of a method for building a scene library provided in an embodiment of the present application Figure 3 .like Figure 4 As shown, the method includes: the server is connected to the acquisition terminal through MQTT network communication, and the server is connected to the client through the MQTT network communication;

[0097] S401. If a first topic is defined according to the MQTT identifier of the server and the MQTT identifier of the receiving end, and the collection terminal is set to subscribe to the first topic, the server is the sending end, and the collection terminal is the receiving end.

[0098] The configured server can interact with the collection terminal and client through the MQTT network; the collection terminal and client can also interact through the MQTT network;

[0099] MQTT is a lightweight message transmission protocol designed for communication in low-bandwidth, high-latency or unstable network environments. The MQTT network protocol improves the reliability of communication between acquisition terminals, servers, and clients.

[0100] The definition of the topic needs to be combined with the direction of message sending and receiving and the ID of the MQTT node. One device can correspond to one MQTT node, and each acquisition terminal, server, and client corresponds to a different ID. The MQTT identifier of the receiving direction: the receiving end is SUB, and the sending end is PUB.

[0101] If the collection terminal is used as a receiving end to receive messages sent by the server, the first topic can be set according to the MQTT identifier of the server and the MQTT identifier of the receiving end, and the collection terminal can be subscribed to the first topic.

[0102] S402: If a second topic is defined according to the MQTT identifier of the collection terminal and the MQTT identifier of the receiving end, and the server is set to subscribe to the second topic, the server is the receiving end, and the collection terminal is the sending end.

[0103] On the contrary, if the server is to be used as a receiving end to receive messages sent by the collection terminal, a second topic can be set according to the MQTT identifier of the server and the MQTT identifier of the receiving end, and the server can be subscribed to the second topic.

[0104] The MQTT network communication methods between the server and the client, the server and the collection terminal, and the client and the collection terminal are the same:

[0105] Take the example of the collection terminal receiving client messages. The ID of the collection terminal is D0004444, the ID of the server is B0001111, and the ID of the client is C0002222. The priority order is defined as D>C>B. The topic of the message sent by C0002222 to D0004444 is "DSUB_D0004444". When the collection terminal of D0004444 subscribes to the topic "DSUB_D0004444", the MQTT nodes in the same network can send messages through the topic "DSUB_D0004444", and the collection terminals subscribed to the topic "DSUB_D0004444" can receive the messages.

[0106] When the client needs to operate multiple collection terminals to transmit data, it can send the corresponding subscribed topics to different collection terminals one by one. These topics all start with "DSUB".

[0107] In the embodiment of the present application, communication is established based on the MQTT network to achieve accurate and reliable communication on the network, and data feedback and execution of user operation commands are achieved with the help of the network.

[0108] Figure 5 A schematic diagram of a method for building a scene library provided in an embodiment of the present application Figure 4 .like Figure 5 As shown, the method includes:

[0109] S501. If the trigger instruction and the re-injection instruction of the client are received, the server obtains the CAN data file and the video file according to the scenario description file, and generates an intelligent driving simulation algorithm corresponding to the data volume of the CAN data file and the video file.

[0110] For multiple scene keywords selected by the client, they can be submitted to the server for feedback analysis at one time; after the server receives the trigger instruction and the feedback instruction at the same time, it downloads the CAN data file and video file from the database, and generates an intelligent driving simulation algorithm corresponding to the data volume of the CAN data file and the video file through the thread pool technology. The algorithm logic corresponding to the intelligent driving simulation algorithm can be the one currently used by the vehicle.

[0111] S502: Perform multi-threaded synchronous simulation on the data of the CAN data file and the video file through the intelligent driving simulation algorithm to confirm whether there is a thread abnormality.

[0112] The data of the CAN data file and the video file are loaded into the thread pool, and the data of the CAN data file and the video file are simulated synchronously by multiple threads through the intelligent driving simulation algorithm. An exception is thrown for the thread that cannot be executed, and a result report of the algorithm simulation re-injection is given after the execution of other threads is completed; if a thread abnormality is detected, a new intelligent driving simulation algorithm without abnormality can be generated to replace the algorithm currently being used by the vehicle, so as to facilitate the vehicle algorithm upgrade.

[0113] In the embodiment of the present application, the scene data simulation re-injection function based on multi-threading can efficiently and synchronously re-inject the scene data selected by the client; after the server completes the simulation re-injection based on multi-threading, it automatically generates a re-injection report, and the results of the simulation re-injection can be quickly obtained through the report, especially whether the algorithm design meets the design index requirements.

[0114] Figure 6 A diagram of a scene library construction device provided by an embodiment of the present invention, such as Figure 6 As shown, the device includes: a first processing module 601, a second processing module 602, and a transmission module 603;

[0115] The first processing module 601 is used for the server to obtain the driving information and environmental information of the vehicle through the acquisition terminal, and store the driving information as a CAN data file and store the environmental information as a video file.

[0116] The first processing module 601 is further used to store different types of driving information at the scene triggering moment as a CAN data file, wherein the types of driving information include vehicle status, intelligent driving function working status and driver control signal;

[0117] The different types of environmental information at the scene triggering moment are stored as video files, wherein the types of environmental information include weather information, geographical location information, obstacle information and road information.

[0118] The second processing module 602 is used to create a scene description file according to the driving information and the environmental information.

[0119] The second processing module 602 is further used to extract scene keywords according to the driving information and the environmental information, wherein the scene keywords include weather information, obstacle information and vehicle status;

[0120] Establishing a first key description of the CAN data file related to the scene keyword and a storage location of the CAN data file in the scene description file;

[0121] A second key description of the video file associated with the scene keyword and a storage location of the video file are established in the scene description file.

[0122] The transmission module 603 is used for, if receiving a trigger instruction from the client, the server obtains the CAN data file and the video file according to the scene description file and sends them to the client, wherein the trigger instruction includes a scene keyword.

[0123] The transmission module 603 is further used to obtain a second key description related to the scene keyword in the trigger instruction in the scene description file, and obtain a target video file related to the second key description;

[0124] Acquire a collection time period of data in the target video file, and acquire the CAN data file in which the description time period in the first key description is the same as the collection time period, to obtain a target CAN data file;

[0125] The target video file and the target CAN data file are sent to the client from the storage location of the target video file and the storage location of the target CAN data file.

[0126] The present application also provides a scene library building device, including: at least one processor and a memory;

[0127] The memory stores computer-executable instructions;

[0128] The at least one processor executes the computer-executable instructions stored in the memory, so that the at least one processor executes the scene library building method.

[0129] Figure 7 The hardware diagram of the scene library building device provided by the embodiment of the present invention is as follows. Figure 7 As shown, the scene library building device 70 provided in this embodiment includes: at least one processor 701 and a memory 702. The device 70 also includes a communication component 703. The processor 701, the memory 702 and the communication component 703 are connected via a bus 704.

[0130] In the specific implementation process, at least one processor 701 executes the computer execution instructions stored in the memory 702, so that at least one processor 701 executes the above scene library building method.

[0131] The specific implementation process of the processor 701 can be found in the above method embodiment, and its implementation principle and technical effect are similar, so this embodiment will not be repeated here.

[0132] In the above Figure 7 In the illustrated embodiment, it should be understood that the processor may be a central processing unit (CPU), or other general-purpose processors, digital signal processors (DSP), application-specific integrated circuits (ASIC), etc. A general-purpose processor may be a microprocessor or any conventional processor, etc. The steps of the method disclosed in the invention may be directly implemented as being executed by a hardware processor, or may be executed by a combination of hardware and software modules in the processor.

[0133] The memory may include a high-speed memory (Random Access Memory, RAM), and may also include a non-volatile memory (NVM), such as at least one disk storage.

[0134] The bus can be an Industry Standard Architecture (ISA) bus, a Peripheral Component Interconnect (PCI) bus, or an Extended Industry Standard Architecture (EISA) bus, etc. The bus can be divided into an address bus, a data bus, a control bus, etc. For ease of representation, the bus in the drawings of this application is not limited to only one bus or one type of bus.

[0135] The present application also provides a computer-readable storage medium, in which computer-executable instructions are stored. When a processor executes the computer-executable instructions, the method described above is implemented.

[0136] The computer-readable storage medium mentioned above can be implemented by any type of volatile or non-volatile storage device or a combination thereof, such as static random access memory (SRAM), electrically erasable programmable read-only memory (EEPROM), erasable programmable read-only memory (EPROM), programmable read-only memory (PROM), read-only memory (ROM), magnetic memory, flash memory, magnetic disk or optical disk. The readable storage medium can be any available medium that can be accessed by a general or special purpose computer.

[0137] An exemplary readable storage medium is coupled to a processor so that the processor can read information from the readable storage medium and write information to the readable storage medium. Of course, the readable storage medium can also be a component of the processor. The processor and the readable storage medium can be located in an application specific integrated circuit (Application Specific Integrated Circuits, referred to as: ASIC). Of course, the processor and the readable storage medium can also exist in the device as discrete components.

[0138] The division of the units is only a logical function division, and there may be other divisions in actual implementation, such as multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. In addition, the mutual coupling or direct coupling or communication connection shown or discussed may be an indirect coupling or communication connection through some interface, device or unit, which may be electrical, mechanical or other forms.

[0139] The units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed on multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0140] Those skilled in the art can understand that all or part of the steps of implementing the above-mentioned method embodiments can be completed by hardware related to program instructions. The aforementioned program can be stored in a computer-readable storage medium. When the program is executed, the steps of the above-mentioned method embodiments are executed; and the aforementioned storage medium includes: ROM, RAM, disk or optical disk and other media that can store program codes.

[0141] Finally, it should be noted that those skilled in the art will readily conceive of other embodiments of the present invention after considering the specification and practicing the invention disclosed herein. The present invention is intended to cover any variation, use or adaptation of the present invention, which follows the general principles of the present invention and includes common knowledge or conventional techniques in the art not disclosed by the present invention, is not limited to the precise structure described above and shown in the drawings, and may be modified and changed in various ways without departing from the scope thereof.

Claims

1. A method for building a scene library, characterized in that: include: The server obtains the driving information and environmental information of the vehicle through the acquisition terminal, and stores the driving information as a CAN data file and stores the environmental information as a video file; Establishing a scene description file according to the driving information and the environmental information; If a trigger instruction from the client is received, the server obtains the CAN data file and the video file according to the scene description file and sends them to the client, wherein the trigger instruction includes a scene keyword.

2. The method according to claim 1, characterized in that The storing of the driving information as a CAN data file and the storing of the environmental information as a video file includes: storing different types of driving information at the scene triggering moment as CAN data files, wherein the types of driving information include vehicle status, intelligent driving function working status and driver control signals; The different types of environmental information at the scene triggering moment are stored as video files, wherein the types of environmental information include weather information, geographical location information, obstacle information and road information.

3. The method according to claim 2, characterized in that The step of establishing a scene description file according to the driving information and the environment information includes: Extracting scene keywords according to the driving information and the environmental information, wherein the scene keywords include weather information, obstacle information and vehicle status; Establishing a first key description of the CAN data file related to the scene keyword and a storage location of the CAN data file in the scene description file; A second key description of the video file associated with the scene keyword and a storage location of the video file are established in the scene description file.

4. The method according to claim 3, characterized in that The acquiring the CAN data file and the video file according to the scene description file and sending them to the client includes: In the scene description file, obtaining a second key description related to the scene keyword in the trigger instruction, and obtaining a target video file related to the second key description; Acquire a collection time period of data in the target video file, and acquire the CAN data file in which the description time period in the first key description is the same as the collection time period, to obtain a target CAN data file; The target video file and the target CAN data file are sent to the client from the storage location of the target video file and the storage location of the target CAN data file.

5. The method according to claim 2, characterized in that: The server obtains the vehicle's driving information and environmental information through the acquisition terminal, including: The server obtains the driving information and the environmental information of the vehicle through the active mode and / or passive mode of the acquisition terminal; wherein the passive mode is a working mode for collecting the driving information and the environmental information by triggering a scene according to a preset condition; and the active mode is a working mode for collecting the driving information and the environmental information by triggering a scene according to a target detection and tracking model.

6. The method according to claim 1, characterized in that The server is connected to the acquisition terminal via MQTT network communication, and the server is connected to the client via MQTT network communication; If a first topic is defined according to the MQTT identifier of the server and the MQTT identifier of the receiving end, and the collection terminal is set to subscribe to the first topic, the server is the sending end, and the collection terminal is the receiving end; If a second topic is defined according to the MQTT identifier of the collection terminal and the MQTT identifier of the receiving end, and the server is set to subscribe to the second topic, the server is the receiving end and the collection terminal is the sending end.

7. The method according to claim 1, characterized in that If the trigger instruction and the refill instruction of the client are received, the method further includes: The server obtains the CAN data file and the video file according to the scenario description file, and generates an intelligent driving simulation algorithm corresponding to the data volume of the CAN data file and the video file; The intelligent driving simulation algorithm is used to perform multi-threaded synchronous simulation on the data of the CAN data file and the video file to confirm whether there is a thread anomaly.

8. A scene library building device, characterized in that: include: The first processing module is used for the server to obtain the driving information and environmental information of the vehicle through the acquisition terminal, and store the driving information as a CAN data file and store the environmental information as a video file; A second processing module, used for establishing a scene description file according to the driving information and the environmental information; The transmission module is used for, if receiving a trigger instruction from the client, the server obtains the CAN data file and the video file according to the scene description file and sends them to the client, wherein the trigger instruction includes a scene keyword.

9. A scene library building device, characterized in that: include: at least one processor and memory; The memory stores computer-executable instructions; The at least one processor executes the computer-executable instructions stored in the memory, so that the at least one processor executes the scene library building method according to any one of claims 1 to 7.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the steps of the scene library building method as described in any one of claims 1 to 7 are implemented.