Method and system for verifying intelligent driving domain controller perception algorithm based on simulated camera

Abstract

The present invention relates to the field of intelligent driving camera simulation technology, and discloses a method and system for verifying the perception algorithm of an intelligent driving domain controller based on a simulated camera. The method includes: collecting view data and laser radar point cloud data of a real driving scene; preprocessing the view data; performing camera simulation in a module simulation module, outputting intermediate data and data packages to the intelligent driving domain controller, and outputting the laser radar point cloud data to the intelligent driving domain controller through an Ethernet board to obtain the true value information of the target object in the real camera coordinate system; the perception algorithm unit outputs the perception result according to the original format data, and compares it with the true value information of the target object in the real camera coordinate system to verify the perception algorithm performance index. It can improve the test efficiency, improve the authenticity of the simulation scene and the credibility of the test results, and then accurately identify the problems existing in the perception algorithm and reduce the test risk.

Description

Technical Field

[0001] The present invention relates to the field of intelligent driving camera simulation technology, and in particular to a method and system for verifying an intelligent driving domain controller perception algorithm based on a simulated camera. Background Art

[0002] At present, the testing and verification of perception algorithms for autonomous vehicles has always been a difficult problem in the industry. In the process of traditional automobile development, actual vehicle verification is often chosen, and the verification is based on test mileage. However, for autonomous vehicles, actual vehicle testing is inefficient and has poor reproducibility. In addition, the perception algorithms of autonomous vehicles are relatively complex, and it is difficult to fully evaluate the perception algorithm capabilities from the mileage dimension alone. Therefore, the industry generally believes that the testing and verification of autonomous vehicles should be based on scenario-based simulation verification. Autonomous vehicles are generally equipped with a variety of sensors, including cameras, millimeter-wave radars, lidars, ultrasonic radars, etc., and with the improvement of the level of autonomous driving, the number of sensors is also gradually increasing, resulting in a large amount of real driving scene data. How to use this data to verify the perception algorithm poses a challenge.

[0003] The in-the-loop simulation verification method generally outputs simulation scenarios through simulation software, and the intelligent driving domain controller performs perception and recognition after receiving the simulation scenarios. However, the industry currently questions the authenticity of the simulation scenarios output by the simulation software, believing that there is a certain gap in authenticity between the simulation scene images and the real-world driving scenes; the actual vehicle test has high authenticity and the test results are highly credible, but the test efficiency is low, and it is difficult to cope with the development and iteration speed of autonomous driving perception algorithms; and the test process of the actual vehicle test is difficult to reproduce 100%, that is, the test process is difficult to trace, and the actual vehicle test is in principle impossible to reproduce perfectly, which cannot meet the requirements of the positioning perception algorithm problem; the actual vehicle test has a high test risk when facing dangerous scenarios.

[0004] Therefore, there is an urgent need for methods and systems to verify the perception algorithms of intelligent driving domain controllers based on simulated cameras, which can improve test efficiency, the realism of simulation scenarios and the credibility of test results, and thus accurately identify problems with perception algorithms and reduce test risks. Summary of the invention

[0005] In order to solve the above technical problems, the present invention provides a method and system for verifying the perception algorithm of an intelligent driving domain controller based on a simulated camera, which can improve test efficiency, improve the realism of the simulation scene and the credibility of the test results, and then accurately identify problems existing in the perception algorithm and reduce test risks.

[0006] The present invention provides a method and system for verifying a perception algorithm of an intelligent driving domain controller based on a simulated camera, comprising the following steps:

[0007] The view data and the laser radar point cloud data of the real driving scene are collected by a real camera and a laser radar respectively, and the view data and the laser radar point cloud data are synchronously processed; wherein the view data includes video data and / or picture data;

[0008] Preprocessing the view data of the real driving scene that has been processed by data synchronization to obtain raw format data, and sending the raw format data to the video injection board; wherein the raw format data includes the raw view data that has not been processed by the real camera after being collected by the real camera, and the raw format data is parallel data;

[0009] The camera simulation is performed in the module simulation module of the video injection board, and the module simulation module includes a camera simulation module and a communication simulation module. The camera simulation module receives the original format data and performs data restoration processing to obtain the intermediate data in the original format, and sends the intermediate data and the data package obtained according to the camera simulation to the communication simulation module. The communication simulation module sends the intermediate data and the data package to the string adding unit of the video injection board; wherein the intermediate data includes the original view data and communication information data collected by the real camera but not processed by the real camera;

[0010] The intermediate data and data packets are converted into serial data through the serialization unit of the video injection board, and the converted intermediate data and data packets are output to the intelligent driving domain controller. At the same time, the laser radar point cloud data that has undergone data synchronization processing is output to the intelligent driving domain controller through the Ethernet board to obtain the true value information of the target object in the real camera coordinate system;

[0011] The converted intermediate data and data packets are restored to parallel data through the deserialization unit in the intelligent driving domain controller and input into the perception algorithm unit in the intelligent driving domain controller;

[0012] The perception algorithm unit outputs the perception results based on the original format data and compares them with the true value information of the target object in the real camera coordinate system to verify the performance indicators of the perception algorithm.

[0013] Further, the camera simulation is performed in the module simulation module of the video injection board, specifically including:

[0014] The module parameters of the camera simulation module are configured according to the parameter information of the real camera, and the module parameters are packaged into a data package; wherein the module parameters include resolution, pixel space, frame rate, communication protocol, data field, synchronization information, and the data package is parallel data;

[0015] The communication parameters of the communication simulation module are configured according to the transmission protocol of the real camera; wherein the communication parameters include the binary address and value of the register.

[0016] Furthermore, configuring the communication parameters of the communication simulation module according to the transmission protocol of the real camera includes:

[0017] Parse and process according to the transmission protocol of the real camera to obtain the binary addresses and values ​​of multiple registers that need to be configured;

[0018] The binary address and value of the register corresponding to the read operation of the intelligent driving domain controller need to be configured completely according to the binary address and value of the register of the real camera;

[0019] The binary address of the register corresponding to the write operation performed by the intelligent driving domain controller needs to be configured completely according to the binary address of the register of the real camera, and the value of the binary address is configured to 0.

[0020] Furthermore, data synchronization processing of the view data and the lidar point cloud data includes:

[0021] The time of the data collected by the real camera based on the master clock and the time of the data collected by the laser radar based on the master clock are obtained respectively; wherein the master clock is the time series of the Ethernet board;

[0022] The time of the data collected by the real camera is used as the time axis, and the time of the data collected by the lidar is mapped to the time axis, so as to achieve time synchronization and alignment of the view data and the lidar point cloud data.

[0023] Furthermore, when the intermediate data and the data group packet are converted into serial data through the serialization unit of the video injection board, and the converted intermediate data and the data group packet are output to the intelligent driving domain controller, and at the same time, the laser radar point cloud data is output to the intelligent driving domain controller through the Ethernet board, it also includes:

[0024] The time series of the Ethernet board is used as the master clock to synchronize the video injection board, ensuring the synchronization of the intermediate data and lidar point cloud data output to the intelligent driving domain controller.

[0025] Furthermore, the view data of the real driving scene is preprocessed to obtain the original format data including:

[0026] When the view data includes only video data, converting the video data into raw format data;

[0027] When the view data includes picture data, the picture data is converted into video stream data, and then the video stream data is converted into original format data.

[0028] Furthermore, the laser radar point cloud data that has been processed by data synchronization is output to the intelligent driving domain controller through the Ethernet board, and the true value information of the target object in the real camera coordinate system is obtained, including:

[0029] The intelligent driving domain controller outputs the true value information of the target objects in the real scene based on the lidar point cloud data, and converts the true value information of the target objects into the true value information of the target objects in the real camera coordinate system through coordinate mapping; among them, the target objects include vehicles, road lines, and traffic signs, and the true value information includes position, type, and motion status.

[0030] The present invention also provides a system for verifying the perception algorithm of an intelligent driving domain controller based on a simulated camera, which is used to execute any of the above methods for verifying the perception algorithm of an intelligent driving domain controller based on a simulated camera, and the system includes the following modules:

[0031] A data acquisition module, the data acquisition module includes a real camera and a laser radar, which are used to respectively collect view data and laser radar point cloud data of a real driving scene, and perform data synchronization processing on the view data and the laser radar point cloud data; wherein the view data includes video data and / or image data;

[0032] A preprocessing module is connected to the real camera and is used to preprocess the view data of the real driving scene that has been processed by data synchronization to obtain raw format data, and send the raw format data to the video injection board; wherein the raw format data includes the raw view data that has not been processed by the real camera after being collected by the real camera, and the raw format data is parallel data;

[0033] A video injection board, the video injection board is connected to the preprocessing module, the video injection board includes a module simulation module and a string adding unit, the module simulation module is used to perform camera simulation, the module simulation module includes a camera simulation module and a communication simulation module, the camera simulation module receives raw format data and performs data restoration processing to obtain raw format intermediate data, and sends the intermediate data and the data package obtained according to the camera simulation to the communication simulation module, the communication simulation module sends the intermediate data and the data package to the string adding unit of the video injection board; wherein the intermediate data includes raw view data and communication information data that are collected by a real camera but not processed by the real camera;

[0034] The serial adding unit is connected to the module simulation module, and is used to convert the intermediate data and data group packets into serial data, and output the converted intermediate data and data group packets to the intelligent driving domain controller;

[0035] Ethernet board, which is connected to the LiDAR and is used to output the LiDAR point cloud data to the intelligent driving domain controller to obtain the true value information of the target object in the real camera coordinate system;

[0036] An intelligent driving domain controller, which is connected to the video injection board and the Ethernet board. The intelligent driving domain controller includes a deserialization unit and a perception algorithm unit. The deserialization unit is used to restore the converted intermediate data and data packets into parallel data and input them into the perception algorithm unit in the intelligent driving domain controller;

[0037] The perception algorithm unit is connected to the deserialization unit and is used to output the perception results according to the original format data and compare them with the true value information of the target object in the real camera coordinate system to verify the performance indicators of the perception algorithm.

[0038] Furthermore, the preprocessing module also includes a video decoding module, which is used to convert video data or video stream data into original format data.

[0039] The embodiments of the present invention have the following technical effects:

[0040] By using the real collected video data or image data as input data, the reality is higher and the test results are more credible than the simulation scenes output by the simulation software; by simulating the simulation camera, the test efficiency is greatly improved compared to directly using the real camera for testing; the simulated camera is configured according to the real camera to make the simulated camera exactly the same as the real camera, and the real collected video data or image data is sent to the intelligent driving domain controller through the simulated camera, making the entire test process completely traceable and reproducible, and able to accurately identify problems with the perception algorithm; by testing with a simulated camera, there is no test risk even in dangerous scenarios, and the risk factor is 0. BRIEF DESCRIPTION OF THE DRAWINGS

[0041] In order to more clearly illustrate the specific implementation methods of the present invention or the technical solutions in the prior art, the drawings required for use in the specific implementation methods or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are some implementation methods of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0042] Figure 1 It is a flow chart of a method for verifying a perception algorithm of an intelligent driving domain controller based on a simulated camera provided by an embodiment of the present invention;

[0043] Figure 2 It is a structural diagram of a system for verifying the perception algorithm of an intelligent driving domain controller based on a simulated camera provided by an embodiment of the present invention. DETAILED DESCRIPTION

[0044] In order to make the purpose, technical solution and advantages of the present invention clearer, the technical solution of the present invention will be described clearly and completely below. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work belong to the scope of protection of the present invention.

[0045] Figure 1 This is a flowchart of a method for verifying the perception algorithm of an intelligent driving domain controller based on a simulated camera provided by an embodiment of the present invention, see Figure 1 , specifically including:

[0046] S1. Collect the view data and lidar point cloud data of the real driving scene through the real camera and lidar respectively, and perform data synchronization processing on the view data and lidar point cloud data.

[0047] Specifically, data collection of real driving scenes can be completed by real vehicles, real cameras, laser radars, collection software, data recording systems, and host computers. Among them, the view data is collected by the real camera, and the laser radar point cloud data is collected by the laser radar. The collected view data is saved in H264 format, and the collected radar point cloud data is saved in PCD format. The view data and radar point cloud data are stored in the data recording system and can be directly displayed in the collection software in the host computer.

[0048] Furthermore, the view data includes video data and / or image data. The view data of the real driving scene is used as the input data of the intelligent driving domain controller perception algorithm, and the lidar point cloud data is used as the true value data to compare with the results output by the intelligent driving domain controller perception algorithm to evaluate the performance of the perception algorithm output results.

[0049] Furthermore, since the sampling frequencies of the two types of data may be different, it is necessary to perform data synchronization processing on the view data and the LiDAR point cloud data, that is, time synchronization and alignment processing. The data synchronization processing on the view data and the LiDAR point cloud data specifically includes:

[0050] S11. Obtain the time of data collected by the real camera based on the master clock and the time of data collected by the laser radar based on the master clock respectively; wherein the master clock is the time series of the Ethernet board.

[0051] Specifically, the time series of the Ethernet board is used as the master clock, and the master clock time is set as the highest priority time for collecting data. This time is guaranteed by the vehicle-mounted GPS system. At the same time, the time node when the sensor collects data is obtained through the counter. This process is called data acquisition time. In this way, the time when the real camera and lidar collect data based on the master clock is obtained.

[0052] S12. Using the time of the data collected by the real camera as the time axis, map the time of the data collected by the lidar to the time axis, thereby achieving time synchronization and alignment of the view data and the lidar point cloud data.

[0053] Specifically, there will be a time deviation between the data collected by the real camera and the laser radar based on the master clock time, namely the first data deviation, the second data deviation, ..., the nth data deviation. Since the view data collected by the real camera is mainly used as the test data in the future, and the laser radar point cloud data is used as the verification data, the time of the data collected by the real camera is used as the time axis, and the time of the data collected by the laser radar point cloud data is mapped to the time axis of the real camera. The average value of the n data deviations is selected as the data collection deviation of the two sensors. The average value is used to consider the hardware stability factors of the real camera and the laser radar, which can minimize the error within the data collection cycle.

[0054] S2. Preprocess the view data of the real driving scene that has undergone data synchronization processing to obtain original format data, and send the original format data to the video injection board.

[0055] Specifically, the raw format data includes the raw view data that has not been processed by the real camera after being collected. At this time, the raw format data is parallel data and is not suitable for long-distance transmission. The raw view data captured by the real camera will be rendered into a human-recognizable picture by the ISP (image processing module), and this step requires the implementation of the inverse process, that is, the collected human-recognizable picture or video frame is inversely transformed into the raw view data.

[0056] Preprocess the view data of the real driving scene to obtain the original format data including:

[0057] S2.1. When the view data only includes video data, convert the video data into raw format data; wherein, converting the video data into raw format data can be implemented by a video decoding module, and the specific form of the raw format should be a data format supported by the domain controller, such as RAW DATA format.

[0058] Exemplarily, the video decoding module is composed of a reversible neural network mapping model, which uses a one-dimensional convolutional layer to optimize the learning function parameters to obtain the optimal transformation function weight matrix; at the same time, in order to prevent data compression distortion during data conversion, a differentiable compression algorithm model is designed, and a compression function is designed based on the Fourier series to achieve data reconstruction, thereby realizing the conversion of actual data to original data with minimal loss.

[0059] S2.2. When the view data includes picture data, the picture data is converted into video stream data, and then the video stream data is converted into original format data.

[0060] Furthermore, converting image data into video stream data generally requires image reading, encoding, decoding and writing functions. First, the images are arranged in a certain order and encoded, that is, the image data is compressed into a video code stream. The video resolution, frame rate and encoding format are set according to the actual acquisition data format to reduce the size of the video file for easy storage and transmission. The image is then interpolated and compensated to make the encoded video stream smoother, thereby obtaining the video stream data required for simulation.

[0061] S3. Perform camera simulation in the module simulation module of the video injection board.

[0062] Specifically, the camera simulation is performed in the module simulation module of the video injection board, that is, a programmable FPGA development board is used to simulate the CMOS module of the real camera. The module simulation module includes a camera simulation module and a communication simulation module. The camera simulation module receives the original format data and performs data restoration processing to obtain the intermediate data in the original format, and sends the intermediate data and the data package obtained according to the camera simulation to the communication simulation module. The communication simulation module sends the intermediate data and the data package to the string adding unit of the video injection board. Among them, the intermediate data includes the original view data and communication information data that have not been processed by the real camera after being collected by the real camera.

[0063] Specifically, the camera simulation in the module simulation module of the video injection board specifically includes:

[0064] S3.1. Configure the module parameters of the camera simulation module according to the parameter information of the real camera, and package the module parameters into a data package; wherein the module parameters include resolution, pixel space, frame rate, communication protocol, data field, synchronization information, etc., and the module parameters of the camera simulation module are completely consistent with the parameter information of the real camera; at this time, the data package is parallel data and is not suitable for long-distance transmission.

[0065] Among them, the camera simulation module specifically includes a pixel array, a row driver, a column driver, a timing control logic, an AD converter, a data bus output interface, a control interface, a single clock, etc., and a CMOS driver is configured at the same time. At this time, each pixel of the RAW DATA data on the FPGA development board can only sense one of R, G, and B, so the data stored in each pixel is monochrome light data.

[0066] S3.2. Configure the communication parameters of the communication simulation module according to the transmission protocol of the real camera; wherein the communication parameters include the binary address and value of the register.

[0067] Specifically, a programmable FPGA development board is embedded in the video injection board, and a reference clock is set to adapt the system data source. In addition to view data, the data transmission between the real camera and the intelligent driving domain controller also involves some communication data, so the communication data also needs to be reproduced.

[0068] Furthermore, configuring the communication parameters of the communication simulation module according to the transmission protocol of the real camera includes:

[0069] Parse and process according to the transmission protocol of the real camera to obtain the binary addresses and values ​​of multiple registers that need to be configured.

[0070] The binary address and value of the register corresponding to the read operation performed by the intelligent driving domain controller need to be configured completely according to the binary address and value of the register of the real camera.

[0071] The binary address of the register corresponding to the write operation performed by the intelligent driving domain controller needs to be configured completely according to the binary address of the register of the real camera, and the value of the binary address is configured to 0.

[0072] For example, the communication data transmission method is I2C protocol. Specifically, it includes:

[0073] Connect a video injection board in series between the real camera and the intelligent driving domain controller. The data flow is real camera-video injection board-intelligent driving domain controller. Configure the video injection board and record the view data and I2C data at the same time. After the intelligent driving domain controller is powered on, the intelligent driving domain controller will send a trigger signal to the real camera and modify the register configuration of the intelligent driving domain camera. These behaviors will be saved in the I2C data in the form of I2C protocol. These are all data generated in real situations.

[0074] In order to realize the communication behavior between the above-mentioned real camera and the intelligent driving domain controller in the simulation, in the CMOS module simulation, the I2C data obtained in the above-mentioned real situation is parsed and processed to obtain the binary addresses and values ​​of multiple registers that need to be configured, and a storage area is opened in the storage area of ​​the video injection board to simulate the I2C data.

[0075] The intelligent driving domain controller has two behaviors on the camera: reading and writing. For the write operation, it is only necessary to ensure that the binary address of the register corresponding to the write operation exists, and the value can be set to 0, because the intelligent driving domain controller will overwrite the original register value during the write operation. For the read operation, it is necessary to reproduce 100% of the data in the real situation, and the accuracy and format cannot be wrong, and there must be no missing data. This is because once the intelligent driving domain controller fails to read the correct data, it cannot guarantee normal communication with the camera, which will cause the entire data path to be unable to proceed.

[0076] S4. Convert the intermediate data and data packets into serial data through the serial unit of the video injection board, and output the converted intermediate data and data packets to the intelligent driving domain controller. At the same time, output the laser radar point cloud data that has undergone data synchronization processing to the intelligent driving domain controller through the Ethernet board to obtain the true value information of the target object in the real camera coordinate system.

[0077] Specifically, the long-distance transmission of data can be achieved by converting the intermediate data and data packets into serial data through the string adding unit, and the string adding unit outputs the data bit by bit. The specific implementation method is: the FPGA chip in the video injection board is used as a CMOS module to simulate a real camera, and outputs the intermediate data in the original format to the string adding unit. The specific content transmitted to the string adding unit includes line synchronization information, frame synchronization information, image data information, pixel clock, etc.

[0078] Exemplarily, the string adding unit may be configured as follows:

[0079] Configure the main clock synchronization data stream. In this solution, the clock frequency is configured to 1.2GHz, the resolution of the transmitted data is 3840×2160, and the frame rate is 30fps. Since the intelligent driving domain controller under test receives the RAW DATA data format, the data format of the serial unit is set to RAW DATA mode. The serial unit generally supports multiple data channels, up to 4 channels. Since the amount of 4k video data transmitted is large, 4 data channels are set for data transmission. The serial unit generally has a low power mode and a high performance mode. To ensure the losslessness of data transmission, it is set to high performance mode.

[0080] Furthermore, the intermediate data and lidar point cloud data will be input into the intelligent driving domain controller through the video injection board and Ethernet board respectively. Since the intelligent driving domain controller needs to obtain the intermediate data through triggering, when the data is injected back into the intelligent driving domain controller, the time series of the Ethernet board is used as the master clock to synchronize the video injection board to ensure the synchronization of the intermediate data and lidar point cloud data output to the intelligent driving domain controller.

[0081] Furthermore, the laser radar point cloud data that has been processed by data synchronization is output to the intelligent driving domain controller through the Ethernet board, and the true value information of the target object in the real camera coordinate system is obtained, including:

[0082] The intelligent driving domain controller outputs the true value information of the target objects in the real scene based on the lidar point cloud data, and converts the true value information of the target objects into the true value information of the target objects in the real camera coordinate system through coordinate mapping; among them, the target objects may include vehicles, road lines, traffic signs, etc., and the true value information may include position, type, motion status, etc.

[0083] Furthermore, the truth value information of the target objects in the real scene can be output based on the lidar point cloud data through the truth value system computing platform in the intelligent driving domain controller.

[0084] S5. The converted intermediate data and data packets are restored to parallel data through the deserialization unit in the intelligent driving domain controller, and input into the perception algorithm unit in the intelligent driving domain controller.

[0085] Specifically, the deserializer receives the serial data and clock signal source output by the serializer, samples the input serial data according to the rising or falling edge of the clock signal, and stores the data in the internal register of the deserializer. The sampling and storage operations are controlled by the clock signal, and the frequency of the clock signal and the serial data must remain consistent.

[0086] S6. The perception algorithm unit outputs the perception result according to the original format data, and compares it with the true value information of the target object in the real camera coordinate system to verify the performance index of the perception algorithm.

[0087] Exemplarily, the perception algorithm performance indicators may include accuracy, precision, etc. A preset accuracy may be set, and when the accuracy between the perception result output by the perception algorithm unit according to the original format data and the true value information of the target object in the real camera coordinate system is greater than or equal to the preset accuracy, it indicates that the perception algorithm has excellent performance.

[0088] In the embodiment of the present invention, by using the real collected video data or picture data as input data, the reality is higher and the credibility of the test result is higher than that of the simulation scene output by the simulation software; by simulating the simulation camera, the test efficiency is greatly improved compared with directly using the real camera for testing; the simulated simulation camera is configured according to the real camera so that the simulated simulation camera is completely consistent with the real camera, and the real collected video data or picture data is sent to the intelligent driving domain controller through the simulated simulation camera, so that the whole test process is completely traceable and reproducible, and the problems of the perception algorithm can be accurately identified; by testing by simulating the simulation camera, there is no test risk even in the face of dangerous scenes, and the risk factor is 0.

[0089] Figure 2 is a schematic diagram of the structure of a system for verifying the perception algorithm of an intelligent driving domain controller based on a simulated camera provided by an embodiment of the present invention. The system is used to execute the method for verifying the perception algorithm of an intelligent driving domain controller based on a simulated camera described in the above embodiment, such as Figure 2 As shown, the system includes the following modules:

[0090] A data acquisition module, the data acquisition module includes a real camera and a laser radar, which are used to respectively collect view data and laser radar point cloud data of a real driving scene, and perform data synchronization processing on the view data and the laser radar point cloud data; wherein the view data includes video data and / or image data;

[0091] A preprocessing module is connected to the real camera and is used to preprocess the view data of the real driving scene that has been processed by data synchronization to obtain raw format data, and send the raw format data to the video injection board; wherein the raw format data includes the raw view data that has not been processed by the real camera after being collected by the real camera, and the raw format data is parallel data;

[0092] A video injection board, the video injection board is connected to the preprocessing module, the video injection board includes a module simulation module and a string adding unit, the module simulation module is used to perform camera simulation, the module simulation module includes a camera simulation module and a communication simulation module, the camera simulation module receives raw format data and performs data restoration processing to obtain raw format intermediate data, and sends the intermediate data and the data package obtained according to the camera simulation to the communication simulation module, the communication simulation module sends the intermediate data and the data package to the string adding unit of the video injection board; wherein the intermediate data includes raw view data and communication information data that are collected by a real camera but not processed by the real camera;

[0093] The serial adding unit is connected to the module simulation module, and is used to convert the intermediate data and data group packets into serial data, and output the converted intermediate data and data group packets to the intelligent driving domain controller;

[0094] Ethernet board, which is connected to the LiDAR and is used to output the LiDAR point cloud data to the intelligent driving domain controller to obtain the true value information of the target object in the real camera coordinate system;

[0095] An intelligent driving domain controller, which is connected to the video injection board and the Ethernet board. The intelligent driving domain controller includes a deserialization unit and a perception algorithm unit. The deserialization unit is used to restore the converted intermediate data and data packets into parallel data and input them into the perception algorithm unit in the intelligent driving domain controller;

[0096] The perception algorithm unit is connected to the deserialization unit and is used to output the perception results according to the original format data and compare them with the true value information of the target object in the real camera coordinate system to verify the performance indicators of the perception algorithm.

[0097] Furthermore, the preprocessing module also includes a video decoding module, which is used to convert video data or video stream data into original format data.

[0098] In the embodiment of the present invention, the view data of the real scene is collected by a real camera, which is more realistic and the test result is more credible than the simulated scene output by the simulation software; the real camera is simulated by the video decoding module, the module simulation module, and the string adding unit, and the test efficiency is greatly improved compared with directly using the real camera for testing; the real camera is simulated based on the module simulation module to send the real collected view data to the intelligent driving domain controller, and the entire test process is completely traceable and reproducible, which can accurately identify problems with the perception algorithm; by testing in a simulated camera manner, there is no test risk even in the face of dangerous scenes, and the risk factor is 0.

[0099] It should be noted that the terms used in the present invention are only for describing specific embodiments, rather than limiting the scope of the present application. As shown in the present specification, unless the context clearly indicates an exception, the words "one", "a", "a kind of" and / or "the" do not specifically refer to the singular, but may also include the plural. The terms "include", "comprise" or any other variant thereof are intended to cover non-exclusive inclusion, so that the process, method or device including a series of elements includes not only those elements, but also includes other elements not explicitly listed, or also includes elements inherent to such process, method or device. In the absence of more restrictions, the elements defined by the sentence "include one..." do not exclude the presence of other identical elements in the process, method or device including the elements.

[0100] It should also be noted that the terms "center", "up", "down", "left", "right", "vertical", "horizontal", "inside", "outside", etc., indicating the orientation or positional relationship, are based on the orientation or positional relationship shown in the drawings, and are only for the convenience of describing the present invention and simplifying the description, rather than indicating or implying that the device or element referred to must have a specific orientation, be constructed and operated in a specific orientation, and therefore cannot be understood as a limitation on the present invention. Unless otherwise clearly specified and limited, the terms "installed", "connected", "connected", etc. should be understood in a broad sense, for example, it can be a fixed connection, a detachable connection, or an integral connection; it can be a mechanical connection or an electrical connection; it can be a direct connection, or it can be an indirect connection through an intermediate medium, or it can be a connection between the two elements. For those of ordinary skill in the art, the specific meanings of the above terms in the present invention can be understood according to specific circumstances.

[0101] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit it. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or replace some or all of the technical features therein by equivalents. However, these modifications or replacements do not deviate the essence of the corresponding technical solutions from the technical solutions of the embodiments of the present invention.

Claims

1. A method for verifying the perception algorithm of an intelligent driving domain controller based on a simulated camera, characterized in that: The steps include: The view data and the laser radar point cloud data of the real driving scene are collected by a real camera and a laser radar respectively, and the view data and the laser radar point cloud data are synchronously processed; wherein the view data includes video data and / or picture data; Preprocessing the view data of the real driving scene that has undergone data synchronization processing to obtain original format data, and sending the original format data to the video injection board; wherein the preprocessing is inverse processing, which is used to inversely process the view data acquired by the real camera to restore it to the original view data that has not been processed by the real camera, that is, the original format data includes the original view data that has not been processed by the real camera after being acquired by the real camera, and the original format data is parallel data; A camera simulation is performed in the module simulation module of the video injection board, the module simulation module includes a camera simulation module and a communication simulation module, the camera simulation module receives the original format data and performs data restoration processing to obtain intermediate data in the original format, and sends the intermediate data and the data group package obtained according to the camera simulation to the communication simulation module, and the communication simulation module sends the intermediate data and the data group package to the string adding unit of the video injection board; wherein the intermediate data includes the original view data and communication information data after being collected by the real camera and not processed by the real camera; The camera simulation in the module simulation module of the video injection board specifically includes: Configuring module parameters of the camera simulation module according to the parameter information of the real camera, and packaging the module parameters into a data package; wherein the module parameters include resolution, pixel space, frame rate, communication protocol, data field, synchronization information, and the data package is parallel data; According to the transmission protocol of the real camera, the communication parameters of the communication simulation module are configured; wherein the communication parameters include the binary address and value of the register; The intermediate data and the data group packet are converted into serial data through the serialization unit of the video injection board, and the converted intermediate data and the data group packet are output to the intelligent driving domain controller. At the same time, the laser radar point cloud data that has been processed by data synchronization is output to the intelligent driving domain controller through the Ethernet board to obtain the true value information of the target object in the real camera coordinate system; The laser radar point cloud data after data synchronization processing is output to the intelligent driving domain controller through the Ethernet board, and the true value information of the target object in the real camera coordinate system is obtained, including: The intelligent driving domain controller outputs the true value information of the target object in the real scene according to the laser radar point cloud data, and converts the true value information of the target object into the true value information of the target object in the real camera coordinate system through coordinate mapping; wherein the target object includes a vehicle, a road line, and a traffic sign, and the true value information includes a position, a type, and a motion state; The converted intermediate data and the data group packet are restored into parallel data through a deserialization unit in the intelligent driving domain controller, and input into a perception algorithm unit in the intelligent driving domain controller; The perception algorithm unit outputs a perception result according to the original format data, and compares it with the true value information of the target object in the real camera coordinate system to verify the perception algorithm performance index.

2. The method for verifying the perception algorithm of the intelligent driving domain controller based on a simulated camera according to claim 1 is characterized in that: The communication parameters of the communication simulation module configured according to the transmission protocol of the real camera include: Parse and process according to the transmission protocol of the real camera to obtain binary addresses and values ​​of multiple registers that need to be configured; The binary address and value of the register corresponding to the read operation performed by the intelligent driving domain controller need to be configured completely according to the binary address and value of the register of the real camera; The binary address of the register corresponding to the write operation performed by the intelligent driving domain controller needs to be configured completely according to the binary address of the register of the real camera, and the value of the binary address is configured to 0.

3. The method for verifying the perception algorithm of the intelligent driving domain controller based on a simulated camera according to claim 1 is characterized in that: The data synchronization processing of the view data and the laser radar point cloud data includes: Respectively obtain the time of the data collected by the real camera based on the master clock and the time of the data collected by the laser radar based on the master clock; wherein the master clock is the time series of the Ethernet board; The time of the data collected by the real camera is used as the time axis, and the time of the data collected by the laser radar is mapped to the time axis, so as to achieve time synchronization and alignment of the view data and the laser radar point cloud data.

4. The method for verifying the perception algorithm of the intelligent driving domain controller based on a simulated camera according to claim 3 is characterized in that: When the intermediate data and the data group packet are converted into serial data by the serialization unit of the video injection board, and the converted intermediate data and the data group packet are output to the intelligent driving domain controller, and at the same time, the laser radar point cloud data is output to the intelligent driving domain controller through the Ethernet board, it also includes: The time series of the Ethernet board is used as the master clock to synchronize the video injection board, ensuring the synchronization of the intermediate data and the lidar point cloud data output to the intelligent driving domain controller.

5. The method for verifying the perception algorithm of the intelligent driving domain controller based on a simulated camera according to claim 1 is characterized in that: The preprocessing of the view data of the real driving scene to obtain the original format data includes: When the view data only includes video data, converting the video data into the original format data; When the view data includes picture data, the picture data is converted into video stream data, and then the video stream data is converted into the original format data.

6. A system for verifying the perception algorithm of an intelligent driving domain controller based on a simulated camera, used to execute the method for verifying the perception algorithm of an intelligent driving domain controller based on a simulated camera as described in any one of claims 1 to 5 above, characterized in that: The system includes the following modules: A data acquisition module, the data acquisition module includes a real camera and a laser radar, which are respectively used to collect view data and laser radar point cloud data of a real driving scene, and perform data synchronization processing on the view data and the laser radar point cloud data; wherein the view data includes video data and / or picture data; A preprocessing module is connected to the real camera, and is used to preprocess the view data of the real driving scene that has been processed by data synchronization to obtain raw format data, and send the raw format data to the video injection board; wherein the preprocessing is inverse processing, and is used to inversely process the view data acquired by the real camera to restore it to the original view data that has not been processed by the real camera, that is, the original format data includes the original view data that has not been processed by the real camera after being acquired by the real camera, and the original format data is parallel data; A video injection board, the video injection board is connected to the preprocessing module, the video injection board includes a module simulation module and a string adding unit, the module simulation module is used to perform camera simulation, the module simulation module includes a camera simulation module and a communication simulation module, the camera simulation module receives the original format data and performs data restoration processing to obtain the intermediate data in the original format, and sends the intermediate data and the data group package obtained according to the camera simulation to the communication simulation module, the communication simulation module sends the intermediate data and the data group package to the string adding unit of the video injection board; wherein the intermediate data includes the original view data and communication information data that are collected by a real camera but not processed by the real camera; The serial adding unit is connected to the module simulation module, and is used to convert the intermediate data and the data group packet into serial data, and output the converted intermediate data and the data group packet to the intelligent driving domain controller; An Ethernet board, which is connected to the laser radar and is used to output the laser radar point cloud data to the intelligent driving domain controller to obtain the true value information of the target object in the real camera coordinate system; An intelligent driving domain controller, the intelligent driving domain controller is connected to the video injection board and the Ethernet board, the intelligent driving domain controller includes a deserialization unit and a perception algorithm unit, the deserialization unit is used to restore the converted intermediate data and the data group packet into parallel data, and input it into the perception algorithm unit in the intelligent driving domain controller; The perception algorithm unit is connected to the deserialization unit, and is used to output the perception result according to the original format data, and compare it with the true value information of the target object in the real camera coordinate system to verify the perception algorithm performance index.

7. The system for verifying the perception algorithm of the intelligent driving domain controller based on a simulated camera according to claim 6 is characterized in that: The preprocessing module further includes a video decoding module, and the video decoding module is used to convert video data or video stream data into original format data.

Images