CI / CD-based 4D radar function test method and system

Abstract

The invention discloses a CI / CD-based 4D radar function test method and system, and the method comprises the steps: S1, collecting data through a 4D radar and a laser radar in different scenes based on a test vehicle carrying a 4D radar and laser radar system, uploading the data to an edge server, and triggering a downstream assembly line after a CI / CD assembly line tool detects new data, further uploading the data to a control terminal; s2, after receiving the uploaded data file, the terminal controller processes the collected data based on a CI / CD assembly line; and S3, aiming at a result processed in the step S2, automatically generating a test report through a CI / CD assembly line tool, and sending the test report to related personnel. By means of a CI / CD assembly line tool, the test automation degree is improved, manual intervention is reduced, and the test period is shortened. Detection results and abnormal points are visually presented through a visual interface of the cloud platform, playback and manual labeling are supported, a closed loop of problem tracking is formed, and traceability and consistency of the test results are guaranteed.

Description

Technical Field

[0001] The present invention belongs to the technical field of automobile radars, and in particular relates to a 4D radar function testing method and system based on CI / CD. Background Art

[0002] Domestic and foreign automotive radars (including 4D imaging radars) usually undergo functional testing and verification before R&D and mass production. The traditional process mainly relies on laboratory echo simulation platforms (hardware-in-the-loop, HIL) or field tests to verify the radar's performance in target detection, speed measurement, distance measurement, and angle resolution.

[0003] The current common practice is to complete it through manual scripts, testers manually triggering test tools or single integration tests. There is a lack of automated pipeline management and it cannot well support the continuous integration and iteration of software versions.

[0004] Therefore, some internationally renowned radar suppliers (such as Bosch, Continental, etc.) and domestic radar solution providers have begun to apply software-in-the-loop (SIL) and hardware-in-the-loop (HIL) testing methods, using simulation tools to establish target scenarios and simulate various reflection conditions and noise environments to verify the key algorithms of 4D radars. However, these tests are more phased and manually triggered, making it difficult to achieve continuous automated testing and rapid regression of radar software version iterations.

[0005] In addition, some vehicle manufacturers or research institutions have tried to build automated testing platforms, combining scripts or simple pipeline building tools to automate verification to a certain extent. However, in most cases, these solutions focus on the software layer or specific modules, and cannot perform complete system-level testing of radars. There is a lack of efficient verification of 4D radars in real perception scenarios, and the test dimensions and coverage are insufficient.

[0006] In general, the functional testing and verification of automotive radars in R&D and before mass production has the following deficiencies and defects: 1) The degree of test automation is low, making it difficult to cope with frequent iterations: Existing technologies often rely on manual or semi-automated scripts for test triggering and result collection, and lack efficient CI / CD pipeline support. With the rapid iteration of 4D radar algorithms and the continuous enrichment of environmental perception scenarios, the testing process cannot keep pace with code iterations, and testing often lags behind or does not cover new features.

[0007] 2) Lack of adequate coverage of real scenarios: Whether it is laboratory echo simulation or field data collection, there are data gaps in certain environments or angles. Existing test methods are difficult to complete efficient coverage of multiple typical or extreme scenarios within a limited time, resulting in problems not being exposed in time, affecting the reliability and safety of the final system.

[0008] 3) High cost of environment construction and data management: 4D radar involves large signal processing and complex hardware interfaces, and needs to cooperate with multi-sensor fusion and vehicle bus verification. Existing technologies usually require a lot of manual intervention and hardware configuration when building a test environment. The collection, version management, playback and comparison of test data also consume a lot of resources, and lack systematic automated management methods.

[0009] 4) Insufficient traceability and consistency of test results: In the traditional test mode, due to the lack of integrated pipeline management, the correspondence between test scenarios and test versions is often unclear. Once a failure or bug occurs during the test, R&D personnel are often required to manually trace back the test conditions, which makes it difficult to quickly and accurately locate the problem, and easily leads to repeated work and long debugging cycles. Summary of the invention

[0010] The purpose of the present invention is to provide a 4D radar functional testing method and system based on CI / CD to solve the problems of insufficient automation, limited scene coverage, high cost of environment construction and data management, poor result traceability, and insufficient software and hardware collaborative iteration capabilities in the existing 4D radar testing and verification process.

[0011] In order to solve the above technical problems, this application provides the following technical solutions: The 4D radar functional testing method based on CI / CD includes the following steps: Step S1: Based on the test vehicle equipped with 4D radar and lidar systems, data is collected through 4D radar and lidar in different scenarios and stored in ROS2 Bag format; after the collection is completed, the collected data is uploaded to the edge server through an automated script, and the CI / CD pipeline tool triggers the downstream pipeline after detecting new data, and further uploads the data to the control terminal.

[0012] Step S2: After receiving the uploaded data file, the terminal controller processes the collected data based on the CI / CD pipeline.

[0013] Step S3: Based on the results of step S2, a test report is automatically generated through the CI / CD pipeline tool and sent to relevant personnel.

[0014] Further optimization, in step S1, the data collected by the 4D radar is the perception data; the data collected by the lidar is the true value and can be regarded as a high-precision environment and target reference.

[0015] With further optimization, the test scenarios include but are not limited to highways, tunnels, under bridges, and closed test sites; the collected data include but are not limited to target distance, speed, azimuth, pitch angle and signal-to-noise ratio.

[0016] The present invention can also utilize scalable simulation scenarios and field data playback mechanisms to quickly iterate and reuse a variety of typical and extreme test scenarios, thereby addressing the defect that the prior art is difficult to fully cover the real road environment within limited time and resources.

[0017] Further optimization, in step S2, the CI / CD pipeline processes the collected data, specifically including the following steps: Step S2.1: The control terminal receives the uploaded data file, unpacks and preliminarily parses it, and converts it into a unified structured format.

[0018] Step S2.2: Extract the key fields of the perception data collected by the 4D radar and the true value data collected by the lidar, and align the timestamps and spatial coordinates of the two to ensure the consistency of the data collected by the two sensors.

[0019] Step S2.3: According to the alignment processing data in step S2.2, the perception data collected by the 4D radar is compared with the corresponding true value data collected by the lidar, and the performance indicators such as the detection rate, false detection rate, and target position error of the 4D radar are calculated.

[0020] Step S2.4: Compare the calculation result of each performance indicator with the corresponding threshold value set. When the calculation result is less than the corresponding threshold value, identify and extract the corresponding data segment in the perception data stream, and analyze the problems that arise. Identify various potential problems in the perception data collected by the 4D radar, such as frame loss, missed detection, false detection, etc., and automatically generate problem logs to mark suspicious data segments.

[0021] Step S2.5: Based on the analysis results of steps S2.3 and S2.4, display the results and provide feedback on the issues.

[0022] The present invention incorporates the test data collection, version management, comparison and playback process into a unified CI / CD workflow by setting up a standardized test environment configuration and data management platform, reducing manual configuration and maintenance costs. With the help of pipeline management and versioning tools, it ensures that each test can accurately correspond to a specific software branch, algorithm version and hardware firmware version, quickly locate the cause of failure and reproduce the problem, and overcome the drawbacks of repeated debugging and difficulty in tracking in the traditional mode.

[0023] Further optimization, in step S2.4, during the data processing process, when the automated script detects that a certain performance indicator is abnormal, the following steps are performed: 1) Identify the problem: based on the set threshold, automatically identify the test data segment with abnormal false detection rate, and classify the problem type through the rule engine; such as data reception interruption, target missed detection or false detection, etc.

[0024] 2) Record problems: Generate a problem log to record the specific time, scenario, and corresponding parameters of the problem; associate the problem marker with the corresponding data segment to facilitate subsequent analysis and playback.

[0025] Further optimization, step S2.5 specifically includes: visually displaying the test results through a visual interface, reviewing the data segment where the problem is located, and the tester marking the problems found on the interface, forming a closed-loop feedback to promote the R&D team to make corrections and optimizations.

[0026] In the present invention, a visualization module is set in the terminal controller, such as the "Data Display and Playback" area shown in the figure. Testers can view the target distribution and trajectory overlap of the 4D radar and lidar detection in the target detection map or the top view; or understand the trend of error changes over time or scenes through curve graphs / statistical graphs. For automatically marked perception problem fragments, they can be viewed and played back in detail. Testers can mark, annotate or manually correct these problems in the cloud platform interface, forming a problem tracking and closed-loop management process, and promoting the R&D team to correct and optimize the performance parameters of the 4D radar.

[0027] Further optimization, in step S3, the test report includes statistical results such as detection rate, false detection rate, main error scenarios, and visual charts such as target trajectory comparison chart and error distribution chart; the test report is sent to relevant personnel via email, instant messaging tools or internal systems.

[0028] Further optimization, the terminal controller is deployed in the cloud, which has the advantages of rapid deployment, strong flexibility, high availability, and data security.

[0029] 4D radar functional test system based on CI / CD, including: The data acquisition module is used to collect 4D radar perception data and lidar ground truth data through the test vehicle equipped with 4D radar and lidar systems, and store them in ROS2 Bag format; The data transmission module is used to upload the collected data to the edge server, and the CI / CD pipeline tool detects new data and triggers the downstream pipeline to further upload the data to the terminal controller; The terminal controller includes a data import and processing unit, a perception problem extraction unit, and a data display and playback unit; the data import and processing unit is used to receive uploaded data files, unpack, preliminarily parse, convert formats, extract key fields, and perform coordinate conversion; and perform frame-level matching of data collected by 4D radar and lidar according to timestamps; the perception problem extraction unit is used to automatically identify a variety of potential perception problems of 4D radar; the data display and playback unit displays the target distribution and trajectory overlap detected by 4D radar and lidar through a visual interface, supports focused viewing and playback of problem segments, and testers can mark, annotate or manually correct problems on the interface; The test report generation module is used to automatically generate test reports through CI / CD pipeline tools. The report contains test statistics, visual charts and problem lists, and sends the report to relevant personnel via email, instant messaging tools or internal systems.

[0030] Compared with the prior art, the present invention has the following beneficial effects: 1. Use CI / CD pipeline tools to improve test automation, reduce manual intervention, and shorten the test cycle.

[0031] 2. Enhance scene coverage and authenticity, using simulation scenes and field data playback mechanisms to cover more real road environments. Compare the high-precision true values ​​provided by the lidar with the 4D radar monitoring results in multiple dimensions, calculate the key performance indicators of the 4D radar, and obtain accurate test results.

[0032] 3. Through the visual interface of the cloud platform, the test results and abnormal points are intuitively presented, and playback and manual annotation are supported to form a closed loop of problem tracking, ensure the traceability and consistency of the test results, quickly locate the cause of the problem, and improve the efficiency of problem location. At the same time, it simplifies the environment construction and data management, and reduces the cost of manual configuration and maintenance.

[0033] 4. When running in a CI / CD environment, it can be combined with a code / firmware version management system. When a new version submission or new data upload is detected, the test can be automatically triggered and a comparison report can be generated to ensure fast and stable iteration. BRIEF DESCRIPTION OF THE DRAWINGS

[0034] Figure 1 This is a flow chart of the 4D radar functional testing method based on CI / CD; Figure 2 It is a schematic diagram of the 4D radar functional test system based on CI / CD. DETAILED DESCRIPTION

[0035] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention. Embodiment 1:

[0036] like Figure 1 As shown in the figure, the 4D radar function testing method based on CI / CD includes the following steps: Step S1: Based on the test vehicle equipped with 4D radar and LiDAR system, data is collected by 4D radar and LiDAR in different scenarios and stored in ROS2 Bag format; after the collection is completed, when the tester clicks the Finish button, the automated script uploads the collected data to the edge server, and Jenkins detects that new data is available, and automatically triggers the downstream pipeline to further upload the data to the control terminal. In this embodiment, the terminal controller is deployed in the cloud, such as Alibaba Cloud.

[0037] In this embodiment, Jenkins is configured to monitor a specific directory in the cloud storage and immediately trigger the CI / CD pipeline once a newly uploaded ROS2 Bag file is detected. The pipeline script defines the subsequent processing steps, including data import, processing, analysis, and report generation.

[0038] In this embodiment, the 4D radar and the laser radar are installed at the same position of the test vehicle. The data collected by the 4D radar is the perception data; the data collected by the laser radar is the true value and can be regarded as a high-precision environment and target reference. The test scenarios include but are not limited to highways, tunnels, under bridges, and closed test sites. The collected data include but are not limited to target distance, speed, azimuth, pitch angle, and signal-to-noise ratio. For example, a vehicle is traveling at a speed of 60 km / h in a tunnel, and there is a stationary target in front (such as an obstacle on the tunnel wall).

[0039] Step S2: After receiving the uploaded data file, the cloud platform processes the collected data based on the CI / CD pipeline. The specific steps include: Step S2.1: The cloud platform receives the ROS2 Bag file uploaded by Jenkins, and the message import module unpacks and preliminarily parses it and converts it into a unified structured format for subsequent processing.

[0040] Step S2.2: Extract key fields of the perception data collected by the 4D radar and the true value data collected by the lidar, such as target position, speed, detection confidence, and target category and posture information marked by the lidar; at the same time, perform coordinate conversion according to the vehicle installation parameters, posture information, etc. Align the timestamps and spatial coordinates of the two to ensure the consistency of the data collected by the two sensors, so as to facilitate subsequent analysis and comparison of the two.

[0041] Step S2.3: According to the alignment processing data in step S2.2, the perception data collected by the 4D radar is compared with the corresponding true value data collected by the lidar, and the performance indicators such as the detection rate, false detection rate, and target position error of the 4D radar are calculated, and the analysis results are stored in the database for subsequent display and reporting.

[0042] Step S2.4: Compare the calculation result of each performance indicator with the corresponding threshold value set. When the calculation result is less than the corresponding threshold value, identify and extract the corresponding data segment in the perception data stream, and analyze the problems that arise. Identify various potential problems in the perception data collected by the 4D radar, such as frame loss, missed detection, false detection, etc., and automatically generate problem logs to mark suspicious data segments.

[0043] In this embodiment, during the data processing, when the detection rate is lower than the set 90%, the test data segment with abnormal false detection rate is automatically identified, and the problem type is classified through the rule engine; such as data reception interruption, target missed detection or false detection, etc. Then the problem is recorded: a problem log is generated to record the specific time, scene and corresponding parameters of the problem; the problem mark is associated with the corresponding data segment for subsequent analysis and playback.

[0044] Step S2.5: Based on the analysis results of step S2.3 and step S2.4, display the results and provide feedback on the problem. Specifically, it includes: 1) Displaying the overall test statistics on the dashboard, such as the total detection rate, false detection rate distribution diagram, etc., providing the target trajectory diagram, and intuitively displaying the detection comparison between 4D radar and laser radar.

[0045] 2) Testers click on the link in the problem log to replay the specific data segment and view the actual performance of the perception system. It supports multi-dimensional data views, such as 2D top view and 3D space view, to facilitate in-depth analysis.

[0046] 3) Testers mark the problems they find on the interface, forming a closed-loop feedback loop to encourage the R&D team to make corrections and optimizations.

[0047] Step S3: Based on the results of step S2, a test report is automatically generated through the CI / CD pipeline tool and sent to relevant personnel.

[0048] In step S3, after completing data processing and problem extraction, Jenkins automatically generates a detailed test report and sends it to relevant team members via email. Specifically, the report generation tool is called at the last stage of the pipeline to summarize the test results, performance indicators and problems found. The report includes charts, statistics and problem details, formatted as PDF or HTML documents. Jenkins automatically distributes the test report to testers and R&D teams via email, instant messaging tools (such as Slack, DingTalk) or internal systems, and provides a download link and online viewing portal for the report to ensure timely communication of information. Embodiment 2:

[0049] like Figure 2 As shown in the figure, the 4D radar functional test system based on CI / CD includes: The data acquisition module is used to collect 4D radar perception data and lidar true value data through a test vehicle equipped with a 4D radar and lidar system, and store them in the ROS2 Bag format.

[0050] The data transmission module is used to upload the collected data to the edge server, and the CI / CD pipeline tool detects new data and triggers the downstream pipeline to further upload the data to the cloud platform.

[0051] The cloud platform includes a data import and processing unit, a perception problem extraction unit, and a data display and playback unit; the data import and processing unit is used to receive uploaded data files, unpack, preliminarily parse, convert formats, extract key fields, and perform coordinate conversion; as well as perform frame-level matching of data collected by 4D radar and lidar according to timestamps; the perception problem extraction unit is used to automatically identify a variety of potential perception problems of 4D radar; the data display and playback unit displays the target distribution and trajectory overlap detected by 4D radar and lidar through a visual interface, supports focused viewing and playback of problem segments, and testers can mark, annotate or manually correct problems on the interface.

[0052] The test report generation module is used to automatically generate test reports through CI / CD pipeline tools. The report contains test statistics, visual charts and problem lists, and sends the report to relevant personnel via email, instant messaging tools or internal systems.

[0053] Although the preferred embodiments of the present application have been described, those skilled in the art may make additional changes and modifications to these embodiments once they have learned the basic creative concept. Therefore, the appended claims are intended to be interpreted as including the preferred embodiments and all changes and modifications falling within the scope of the present application.

Claims

1. A 4D radar functional testing method based on CI / CD, characterized in that: The steps include: Step S1: Based on the test vehicle equipped with 4D radar and lidar systems, data is collected by 4D radar and lidar in different scenarios and stored in ROS2 Bag format; after the collection is completed, the collected data is uploaded to the edge server through an automated script, and the CI / CD pipeline tool triggers the downstream pipeline after detecting new data, and further uploads the data to the control terminal; Step S2: After receiving the uploaded data file, the terminal controller processes the collected data based on the CI / CD pipeline; Step S3: Based on the results of step S2, a test report is automatically generated through the CI / CD pipeline tool and sent to relevant personnel.

2. The 4D radar function testing method based on CI / CD according to claim 1, characterized in that: In step S1, the data collected by the 4D radar is the perception data, and the data collected by the laser radar is the true value.

3. The CI / CD-based 4D radar function testing method according to claim 2, characterized in that: The test scenarios include but are not limited to highways, tunnels, under bridges, and closed test sites; the collected data include but are not limited to target distance, speed, azimuth and pitch angle.

4. The CI / CD-based 4D radar function testing method according to claim 3, characterized in that: In step S2, the CI / CD pipeline processes the collected data, specifically including the following steps: Step S2.1: The control terminal receives the uploaded data file, unpacks and preliminarily parses it, and converts it into a unified structured format; Step S2.2: extract the key fields of the perception data collected by the 4D radar and the true value data collected by the lidar, and align the timestamps and spatial coordinates of the two to ensure the consistency of the data collected by the two sensors; Step S2.3: According to the alignment processing data in step S2.2, the perception data collected by the 4D radar is compared with the corresponding true value data collected by the lidar, and the performance indicators such as the detection rate, false detection rate, and target position error of the 4D radar are calculated; Step S2.4: Compare the calculation result of each performance indicator with the set corresponding threshold value. When the calculation result is less than the corresponding threshold value, identify and extract the corresponding data segment in the perception data stream, and analyze the problem. Step S2.5: Based on the analysis results of steps S2.3 and S2.4, display the results and provide feedback on the issues.

5. The 4D radar function testing method based on CI / CD according to claim 4 is characterized in that: In step S2.4, during the data processing, when the automated script detects that a certain performance indicator is abnormal, the following steps are performed: 1) Identify the problem: based on the set threshold, automatically identify the test data segment corresponding to the abnormal indicator, and classify the problem type through the rule engine; 2) Record the problem: Generate a problem log to record the specific time, scenario and corresponding parameters of the problem; associate the problem tag with the corresponding data fragment.

6. The 4D radar function testing method based on CI / CD according to claim 5 is characterized in that: The step S2.5 specifically includes: visually displaying the test results through a visual interface, reviewing and replaying the data segments where the problems are located, and the testers marking the problems found on the interface, forming a closed-loop feedback to promote the R&D team to make corrections and optimizations.

7. The CI / CD-based 4D radar function testing method according to claim 5, characterized in that: In step S3, the test report includes statistical results such as detection rate, false detection rate, main error scenarios, and visual charts such as target trajectory comparison chart and error distribution chart; the test report is sent to relevant personnel via email, instant messaging tools or internal systems.

8. The CI / CD-based 4D radar function testing method according to claim 7, characterized in that: The terminal controller is deployed in the cloud.

9. The 4D radar functional test system based on CI / CD is characterized by: include: The data acquisition module is used to collect 4D radar perception data and lidar ground truth data through the test vehicle equipped with 4D radar and lidar systems, and store them in ROS2 Bag format; The data transmission module is used to upload the collected data to the edge server, and the CI / CD pipeline tool detects new data and triggers the downstream pipeline to further upload the data to the terminal controller; The terminal controller includes a data import and processing unit, a perception problem extraction unit, and a data display and playback unit; wherein the data import and processing unit is used to receive uploaded data files, unpack, preliminarily parse, convert formats, extract key fields, and perform coordinate conversion; and perform frame-level matching of data collected by 4D radar and lidar according to timestamps; the perception problem extraction unit is used to automatically identify multiple potential perception problems of 4D radar; The data display and playback unit displays the target distribution and track overlap of 4D radar and lidar detection through a visual interface, supports focused viewing and playback of problem segments, and testers can mark, annotate or manually correct problems on the interface; The test report generation module is used to automatically generate test reports through CI / CD pipeline tools. The report contains test statistics, visual charts and problem lists, and sends the report to relevant personnel via email, instant messaging tools or internal systems.

Images