Parking perception fusion fault analysis verification method and system

By collecting and analyzing real-vehicle data during parking, generating and correcting perception results, and establishing a scenario library for verification, the problem of parking perception fusion abnormal analysis in automatic parking assist technology is solved, and the failure analysis speed and software update efficiency are improved. .

CN119028170APending Publication Date: 2024-11-26DONGFENG MOTOR GRP
View PDF 0 Cites 2 Cited by

Patent Information

Application Number
CN202411134242.X
Authority / Receiving Office
CN · China
Patent Type
Applications(China)
Current Assignee / Owner
Filing Date
2024-08-19
Publication Date
2024-11-26

AI Technical Summary

Technical Problem

In automatic parking assist technology, it is difficult for the existing technology to quickly locate the source of parking perception fusion abnormalities, resulting in slow progress in failure analysis and it is difficult to intuitively compare the effects of old and new versions of software version update verification.

Method used

By collecting real vehicle data during parking, including target truth value, ultrasonic raw data and visual raw data, visual perception results, ultrasonic perception results and fusion results are generated, and abnormal analysis and correction are carried out, and scene database is established for truth value annotation. and data storage for re-injection verification and evaluation of software version updates.

Benefits of technology

It quickly locates the source of parking perception fusion abnormalities, simplifies the fault analysis process, improves the efficiency and accuracy of software version updates, reduces real-time verification work, and promotes rapid iterative software release.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN119028170A_ABST
    Figure CN119028170A_ABST
Patent Text Reader

Abstract

The invention discloses a parking perception fusion fault analysis verification method and system, and the method comprises the steps: collecting the data of a real vehicle in a parking process; whether the parking process is abnormal or not is judged; when abnormity occurs in the parking process, the ultrasonic original data and the visual data are sensed and fused, and a visual sensing result, an ultrasonic sensing result and a fusion result are generated respectively; wherein the visual perception result, the ultrasonic perception result and the fusion result are provided with a plurality of groups of nodes for observing abnormal types according to a preset standard; performing anomaly analysis on each node in the visual perception result, the ultrasonic perception result and the fusion result in sequence, and when an anomaly occurs, finishing the analysis and correcting the anomaly; and after correction, jumping to judge whether the parking process is abnormal or not until the parking process is not abnormal. According to the invention, the problem output source of parking perception fusion in simulation or real vehicle verification can be visually positioned, the fault reason is analyzed, and the fault solving progress is accelerated.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present invention belongs to the technical field of automatic parking assistance, and specifically relates to a parking perception fusion fault analysis verification method and system. Background Art

[0002] Automatic parking assistance is increasingly being mass-produced and installed, becoming one of the representative technologies of current smart cars. From the initial spatial parking space parking (APA) based on ultrasonic radar, it has developed into the mainstream visual space parking space fusion automatic parking assistance (FAPA). In the parking space search in the FAPA solution, a fusion parking space search method is adopted. However, in the process of verification or use of the function, once problems related to parking space search occur, such as failure to identify the parking space, large deviation in the parking space recognition position, large deviation in the parking space recognition angle, overlapping parking spaces, parking space attribute status errors, etc. Whether it is a problem with the original data of the sensor, the quality of the surround view stitching, the problem of model detection of the visual perception module, the problem of model post-processing, the problem of the radar module, or the problem of the fusion algorithm, it is difficult for problem analysts to intuitively and quickly find the cause of the problem, which seriously affects the progress of problem analysis.

[0003] Currently, problem analysts mainly make inferences based on experience, reproduce abnormal problems and scenarios, record data, and then replay them. They consider as many causes of the problem as possible, conduct investigations and analyses one by one until the problem is finally found, and conduct repeated tests until the problem is finally solved.

[0004] However, the current solution makes it difficult to conduct intuitive problem location analysis, and it is impossible to quickly determine whether the perception fusion anomaly is caused by an abnormal raw data input, an abnormal ultrasonic perception, an abnormal visual perception, or an abnormal fusion, etc., which leads to the final parking perception fusion anomaly. In addition, the current solution's test verification of software version changes also makes it difficult to directly see the differences in the various effects of the original software version and the updated software version in perception fusion. Finally, the current development test solution also makes it difficult to use the true value scenario library to conduct an overall evaluation and comparison of the new and old versions of parking perception fusion, and automatically output a quantitative comparison test analysis report for the scenario library. Summary of the invention

[0005] The purpose of the present invention is to provide a parking perception fusion fault analysis and verification method and system, which can intuitively locate the problem output source of parking perception fusion in simulation or actual vehicle verification, analyze the cause of the fault, and speed up the progress of fault resolution.

[0006] In order to solve the above technical problems, the technical solution of the present invention is: a parking perception fusion fault analysis and verification method, comprising:

[0007] During the parking process, real vehicle data is collected; the real vehicle data at least includes: target true value, ultrasonic raw data, visual raw data and vehicle related information;

[0008] Determine whether the parking process is abnormal;

[0009] When an abnormality occurs during parking, the ultrasonic raw data and visual data are sensed and fused to generate visual perception results, ultrasonic perception results, and fusion results respectively; among which, the visual perception results, ultrasonic perception results, and fusion results are used to set multiple groups of nodes for observing abnormality types according to preset standards;

[0010] The visual perception results, ultrasonic perception results and each node in the fusion results are analyzed for abnormalities in sequence. When an abnormality occurs, the analysis ends and the abnormality is corrected; after the correction, the process jumps to determine whether the parking process is abnormal until there is no abnormality in the parking process.

[0011] It also includes: using the data after abnormal analysis correction and the visual perception results, ultrasonic perception results and fusion results before correction as visual perception historical version results, ultrasonic perception historical version results and fusion historical version results respectively, and using multiple groups of visual perception historical version results, ultrasonic perception historical version results and fusion historical version results as a parking perception fusion fault analysis verification method to perform re-injection verification data after the software version is updated.

[0012] It also includes: marking the true value of real vehicle data, data after abnormal analysis correction, and visual perception results, ultrasonic perception results and fusion results before correction, assigning specific numerical values ​​to each data to calculate the data error of each node between the historical software version and the updated software version, and determine whether the updated software version meets the expected optimization effect.

[0013] It also includes: setting up a scene library for storing real vehicle data after true value annotation, data after abnormal analysis correction and visual perception results before correction, ultrasonic perception results and fusion results.

[0014] It also includes: classifying the database scenes into scene library categories according to key scene attributes; the key scene attributes at least include lighting conditions, ground materials and parking space line types;

[0015] For the scenario categories corresponding to the abnormal problems solved by the software version update, one or more scenarios of similar scenario categories in the scenario library are screened out for re-injection to verify the optimization effect of the software under similar scenario categories, and quantitative indicator statistical analysis is performed based on the annotated true values; comparative analysis is performed on the re-injection indicators of the same scenario data of the software versions before and after the update, and a similar scenario re-injection verification evaluation report is generated;

[0016] Conduct back-injection verification and evaluation on all scenarios in the scenario library for versions of the software before and after the update, perform quantitative indicator statistical analysis, generate a full-scenario back-injection verification and evaluation report, and speed up version update verification and release.

[0017] A parking perception fusion fault analysis and verification system is also provided, which includes at least a real vehicle data collection and display function module and a host computer perception fusion abnormality analysis function module, wherein:

[0018] The real vehicle data collection and display function module is used to collect the real vehicle data during the parking process; the real vehicle data at least includes: target true value, ultrasonic raw data, visual raw data and vehicle related information;

[0019] The host computer perception fusion abnormal analysis function module is used to determine whether the parking process is abnormal;

[0020] When an abnormality occurs during parking, the ultrasonic raw data and visual data are sensed and fused to generate visual perception results, ultrasonic perception results, and fusion results respectively; among which, the visual perception results, ultrasonic perception results, and fusion results are used to set multiple groups of nodes for observing abnormality types according to preset standards;

[0021] The visual perception results, ultrasonic perception results and each node in the fusion results are analyzed for abnormalities in sequence. When an abnormality occurs, the analysis ends and the abnormality is corrected; after the correction, the process jumps to determine whether the parking process is abnormal until there is no abnormality in the parking process.

[0022] It also includes: an updated re-injection verification function module, which is used to use the data after abnormal analysis correction and its visual perception results, ultrasonic perception results and fusion results before correction as visual perception historical version results, ultrasonic perception historical version results and fusion historical version results respectively, and use multiple groups of visual perception historical version results, ultrasonic perception historical version results and fusion historical version results as a parking perception fusion fault analysis verification method to perform re-injection verification data after the software version is updated.

[0023] It also includes: a perception fusion index evaluation function module, which is used to mark the true value of real vehicle data, data after abnormal analysis correction and its visual perception results, ultrasonic perception results and fusion results before correction, and assign specific numerical values ​​to each data to calculate the data error of each node between the historical software version and the updated software version, and determine whether the updated software version meets the expected optimization effect.

[0024] It also includes: a scene library, which is used to store real vehicle data after true value annotation, data after abnormal analysis correction and visual perception results, ultrasonic perception results and fusion results before correction.

[0025] It also includes: scenario library recharge verification and evaluation function module, which is divided into three submodules: database scenario classification submodule, similar scenario recharge submodule and full scenario recharge submodule; among them,

[0026] The database scene classification submodule is used to classify the database scenes into scene library categories according to key scene attributes; the key scene attributes include at least lighting conditions, ground materials and parking space line types;

[0027] The same-type scenario re-injection submodule is used to select one or more scenarios of similar scenario categories in the scenario library for re-injection for the scenario categories corresponding to the abnormal problems solved by the software version update, verify the optimization effect of the software under similar scenario categories, and perform quantitative indicator statistical analysis based on the annotated true values; compare and analyze the re-injection indicators of the same-type scenario data in the software versions before and after the update, and generate a similar scenario re-injection verification evaluation report;

[0028] The full-scenario reinjection submodule is used to perform reinjection verification and evaluation on all scenarios in the scenario library of the software versions before and after the update, conduct quantitative indicator statistical analysis, generate a full-scenario reinjection verification and evaluation report, and speed up version update verification and release.

[0029] Compared with the prior art, the present invention has the following beneficial effects:

[0030] 1. The present invention formulates data nodes containing rich information of perception fusion information and supports customized selection of all or part of the information to be released. It can not only intuitively monitor the result status of each node in the perception fusion operation, but also perform customized data recording containing rich node information.

[0031] 2. The present invention can conveniently analyze problems and quickly identify the source of problems by perceiving and fusing the recorded data of abnormal situations, because it contains rich node information. After the software is updated, the re-injection verification of all key nodes can be quickly performed.

[0032] 3. After marking the true value of the data packet, the present invention can also accumulate and establish a scenario library with rich scenarios including common scenarios. The updated software version can be used for batch scene re-injection. The scenario verification evaluation indicators including key node information can be calculated in combination with the true value, and a report can be automatically generated, thereby reducing the actual vehicle verification work after the software version is updated and helping to quickly iterate and release the software version. BRIEF DESCRIPTION OF THE DRAWINGS

[0033] Figure 1 A schematic diagram of a flow chart of an embodiment of the present invention;

[0034] Figure 2This is a schematic diagram of the structure and function of the host computer perception fusion abnormality analysis function module in an embodiment of the present invention;

[0035] Figure 3 The figure is a schematic diagram of the workflow of the host computer perception fusion anomaly analysis function module in an embodiment of the present invention. DETAILED DESCRIPTION

[0036] In order to make the purpose, technical solutions and advantages of the present invention more clearly understood, the present invention is further described in detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present invention and are not intended to limit the present invention. In addition, the technical features involved in the various embodiments of the present invention described below can be combined with each other as long as they do not conflict with each other.

[0037] The technical solution of the present invention is:

[0038] Embodiment 1:

[0039] A parking perception fusion fault analysis and verification method, such as Figure 1 As shown, including:

[0040] S1. During the parking process, real vehicle data is collected; the real vehicle data at least includes: target true value, ultrasonic raw data, visual raw data and vehicle related information;

[0041] Generally speaking, the true value of the measurement target will be collected for typical scenarios and used to construct typical scenarios in the scenario library, which is mainly used for indicator evaluation during the re-injection of typical scenarios.

[0042] The collected target true values ​​include two categories: parking spaces and obstacles. The first category is the parking space true value, which contains information including (corner0, corner1, corner2, corner3, confidence, ps_type, ps_curb, ps_position), where corner0, corner1, corner2, corner3 represent the four corner coordinates of the parking space in the vehicle coordinate system (x0, y0), (x1, y1), (x2, y2), (x3, y3), the data type is float, the unit is m, the origin of the vehicle coordinate system is the center point of the rear axle of the vehicle, the x-axis direction is to the right, and the y-axis direction is toward the front of the vehicle body; confidence represents confidence, the data type is float, and the value range is (0,1]; ps_type represents the parking space type, the data type is uint8_t, and the value range is (0,1,2,3), which respectively represent the default value, vertical parking space, horizontal parking space, and oblique parking space;

[0043] ps_curb represents the curb attribute of the horizontal parking space. The data type is uint8_t. The value range is (0 or 1), which means there is no curb and there is a curb respectively. ps_position represents the relative position of the parking space to the vehicle. The data type is uint8_t. The value range is (0,1,2), which means the default value, unknown position, position on the left or position on the right respectively.

[0044] The second category is the true value of the obstacle, which contains information including (corner0, corner1, confidence, obs_type), where corner0 and corner1 represent the endpoint coordinates (x0, y0) and (x1, y1) of the grounding line segment of the obstacle target in the vehicle coordinate system, the data type is float, and the unit is m; confidence represents the confidence, the data type is float, and the value range is (0,1]; obs_type represents the obstacle type, the data type is uint8_t, and the value range is (0,1,2,3,4,5), which respectively represent unknown type, cone barrel, wheel stop, ground lock open, ground lock closed, and curb.

[0045] The ultrasonic raw data contains the following information (range0.range1.range2.range3.range4.range5.range6,range7,range8,range9,range10,range11), which respectively represent the distance information detected by the 12 ultrasonic radars of the vehicle in the current frame. The data type is float and the unit is m.

[0046] The original visual data is the AVM image used for FAPA visual parking space detection after fisheye image stitching and correction. The image format is jpg or png, and the image size is usually 1280*720 or 960*720.

[0047] S2. determining whether the parking process is abnormal;

[0048] S3. When an abnormality occurs during parking, the ultrasonic raw data and the visual data are sensed and fused to generate visual perception results, ultrasonic perception results, and fusion results, respectively; wherein the visual perception results, ultrasonic perception results, and fusion results are all used to set multiple groups of nodes for observing abnormality types according to preset standards;

[0049] S4. Perform abnormal analysis on each node in the visual perception results, ultrasonic perception results and fusion results in sequence. When an abnormality occurs, the analysis ends and the abnormality is corrected; after the correction, jump to judging whether the parking process is abnormal until there is no abnormality in the parking process.

[0050] It also includes: using the data after abnormal analysis correction and the visual perception results, ultrasonic perception results and fusion results before correction as visual perception historical version results, ultrasonic perception historical version results and fusion historical version results respectively, and using multiple groups of visual perception historical version results, ultrasonic perception historical version results and fusion historical version results as a parking perception fusion fault analysis verification method to perform re-injection verification data after the software version is updated.

[0051] It also includes: marking the true value of real vehicle data, data after abnormal analysis correction, and visual perception results, ultrasonic perception results and fusion results before correction, assigning specific numerical values ​​to each data to calculate the data error of each node between the historical software version and the updated software version, and determine whether the updated software version meets the expected optimization effect.

[0052] It also includes: setting up a scene library for storing real vehicle data after true value annotation, data after abnormal analysis correction and visual perception results before correction, ultrasonic perception results and fusion results.

[0053] It also includes: classifying the database scenes into scene library categories according to key scene attributes; the key scene attributes at least include lighting conditions, ground materials and parking space line types;

[0054] For the scenario categories corresponding to the abnormal problems solved by the software version update, one or more scenarios of similar scenario categories in the scenario library are screened out for re-injection to verify the optimization effect of the software under similar scenario categories, and quantitative indicator statistical analysis is performed based on the annotated true values; comparative analysis is performed on the re-injection indicators of the same scenario data of the software versions before and after the update, and a similar scenario re-injection verification evaluation report is generated;

[0055] Conduct back-injection verification and evaluation on all scenarios in the scenario library for versions of the software before and after the update, perform quantitative indicator statistical analysis, generate a full-scenario back-injection verification and evaluation report, and speed up version update verification and release.

[0056] The above embodiments may be stored in a computer-readable storage medium, which stores executable instructions. When the instructions are executed by a processor, the processor implements the method described in the above embodiments.

[0057] Embodiment 2:

[0058] A parking perception fusion fault analysis and verification system is also provided, which includes at least a real vehicle data collection and display function module and a host computer perception fusion abnormality analysis function module, wherein:

[0059] The real vehicle data collection and display function module is used to collect real vehicle data during the parking process; during the operation of the parking function, the key node information of each submodule in the perception fusion can be published, displayed and recorded. In this module, the information input source and subnodes of the parking perception fusion module are first confirmed. For example, the commonly used key information nodes of visual perception include original image data input information, surround image preprocessing result output information, visual model reasoning result output information, visual post-processing result output information, visual tracking result output information, etc. The commonly used key information nodes of ultrasonic perception include ultrasonic original signal input information, ultrasonic construction obstacle result output information, ultrasonic construction space parking result output information, etc. The commonly used key information nodes of fusion include ultrasonic obstacle and visual obstacle fusion result output information, ultrasonic parking space and visual parking space fusion result output information, and fusion obstacle and fusion parking space fusion result output information. Vehicle-related information includes parking system status, global odometer information, etc. Secondly, this information is displayed and collected in real time. The data marshaling library (such as LCM, etc.) is used to publish key node information. The automatic parking perception fusion analysis and evaluation system can freely select the key node information that needs to be subscribed for real-time display and data collection and storage. For the real-time display function, a host computer is developed to carry out data communication, and the node information such as surround images, parking spaces, obstacles, etc. in the data are processed in real time, and mapped to the same spatial scale in the host computer for display. For the parking perception fusion module, the commonly used coordinate systems are generally the vehicle coordinate system and the global coordinate system. The present invention uses multiple views to uniformly map the scales from different coordinate systems. For different key node information, in order to perform intuitive analysis when displaying, different colors and line types are selected for display, and legends are added to facilitate intuitive viewing and analysis; the actual vehicle data includes at least: target true value, ultrasonic raw data, visual raw data and vehicle-related information;

[0060] The host computer perception fusion anomaly analysis function module is used to determine whether the parking process is abnormal. This module is used to quickly locate the source of common problems in the automatic parking perception fusion module. Common anomalies in automatic parking perception fusion include failure to identify parking spaces and obstacles, misidentification of parking spaces and obstacles, inaccurate angles of parking space and obstacle identification positions, etc. The main causes of the anomaly are: abnormal parking system status, abnormal raw image data input, abnormal image preprocessing, abnormal visual model reasoning results, abnormal visual post-processing, abnormal visual tracking, abnormal ultrasonic raw data, abnormal ultrasonic space parking space, abnormal ultrasonic obstacle, abnormal fused obstacle, abnormal fused parking space, abnormal parking space and obstacle fusion, abnormal global odometer status, and other anomalies. Based on the key node information in the LCM data and the cause of the problem analyzed by the perception fusion anomaly analysis function module, software developers can quickly optimize and update the software in a targeted manner;

[0061] Perception fusion anomaly analysis is divided into two methods:

[0062] The first method is manual judgment. Through the real-time display of the perception fusion results, it is judged whether there are problems of missed detection, false detection, and wrong detection of parking spaces and obstacles. If such problems exist, it is manually judged that there is an abnormality in the parking perception fusion results.

[0063] The second is the automatic judgment method of perception fusion abnormal analysis.

[0064] It mainly uses the high-precision perception fusion feedback module in the abnormal judgment system to perform real-time feedback of the input of the perception fusion module, and compares and judges the output fusion parking space and fusion obstacle with the perception fusion results output by the vehicle's actual vehicle domain control end.

[0065] The host computer perception fusion abnormal analysis function module solution is as follows Figure 2 As shown, it includes two sub-modules, namely the vehicle-side domain control perception fusion module and the high-precision perception fusion feedback module.

[0066] The input data of the perception fusion module include the stitched and corrected AVM surround view image data with a frame rate of ≥25 frames; ultrasonic original distance data, ultrasonic parking space and ultrasonic obstacles; and vehicle information including speed, heading angle, odometer and other necessary information.

[0067] The similarities between the vehicle-side domain control perception fusion module and the high-precision perception fusion recharge module are that the output data interface format is consistent, both of which are fused parking space information structures and fused obstacle structures. The fused parking space data structure includes but is not limited to (corner0, corner1, corner2, corner3, confidence, ps_type, ps_curb, ps_position), which are used to represent the parking space corner coordinates, confidence, parking space type, parking space curb attributes, and the relative position of the parking space to the vehicle. The fused obstacle structure includes but is not limited to (corner0, corner1, confidence, obs_type), which respectively represent the grounding point position of the obstacle, the confidence of the obstacle, and the obstacle type.

[0068] The difference is that the vehicle-side domain control perception fusion module is limited by the computing power of the embedded platform. In order to ensure the minimum frame rate requirement of parking space detection (generally 3 frames), the vehicle-side domain control perception model will use a lightweight model backbone network with a large number of parameters, such as replacing it with the backbone network of mobilenetV2 to reduce the amount of model inference calculation. Compared with the conventional backbone network, the lightweight backbone network will bring certain performance losses. In terms of the model inference input image size, in order to reduce the amount of calculation, the image inference input size is transformed to a lower size, such as 300*300, which will further lose image details. In terms of model parameter quantization, since it needs to be deployed and run on an embedded mobile terminal, the model needs to be quantized, such as converting parameters from floating point to fixed point, which will also cause a certain decrease in accuracy. Finally, the visual perception results and ultrasonic perception results are fused to output the fused parking space and obstacle result information.

[0069] The high-precision perception fusion module uses a more accurate method in the perception model, model reasoning input size, and model parameter quantization on the basis of ensuring the consistency of the input and output interfaces. The CenterNet deep learning model used is run on a high-computing industrial computer with sufficient computing resources. The backbone network of the CenterNet model selects ResNet18 or DLA-34, and there is no need to perform model lightweight operations. At the input end of the model reasoning, the input image is transformed to a larger size for model reasoning, for example, the input image size is transformed to 760*760 or above. Since it runs on the industrial computer, there is no need to perform quantization operations such as floating point to fixed point conversion on the model. Then the visual perception results are fused with the ultrasonic perception results to output the parking space and obstacle result information with higher accuracy and frame rate after fusion.

[0070] In the comparison and judgment of the fusion results, the operation results on the vehicle side and the operation results on the industrial computer side are matched by position. If there are unmatched parking spaces or obstacles, it is judged that a suspected missed detection anomaly has occurred. For the matched targets, the attributes such as categories are compared. If the attributes are inconsistent, it is judged that a suspected misdetection anomaly has occurred. Finally, the key point position distance of the matched targets with the same category is judged. If it is greater than a certain threshold, such as 15cm, it is judged that a suspected position recognition anomaly has occurred.

[0071] When an abnormality occurs during parking, the ultrasonic raw data and visual data are sensed and fused, and the visual perception results, ultrasonic perception results and fusion results are generated by the back-injection method. Among them, the visual perception results, ultrasonic perception results and fusion results are set as multiple groups of nodes for observing abnormal types according to preset standards;

[0072] In the abnormal judgment module, the feedback of the vehicle-side perception fusion module only publishes the fused parking space and fused obstacles of the overall output of perception fusion through LCM. When it is determined that an abnormality has occurred, the feedback of the vehicle-side perception fusion module switches to the key node full release mode, and the key node results are released through LCM. The feedback of the high-precision perception fusion module on the industrial control computer side also enters the key node full distribution mode, and publishes the key nodes through LCM. Key nodes include but are not limited to image preprocessing result node information, model reasoning result node information, visual post-processing result node information, visual tracking result node information, ultrasonic perception result information, obstacle fusion result information, parking space fusion result information, and parking space and obstacle fusion node information.

[0073] In addition, when the high-precision perception fusion module on the industrial computer side is fed back to start the full-node release mode, the drivable area recognition module will also be turned on. The drivable area recognition module uses the RepVGG deep learning model to detect the drivable area within the surround image range and save the result as the drivable area node information.

[0074] The visual perception results, ultrasonic perception results and each node in the fusion results are analyzed for abnormalities in sequence. When an abnormality occurs, the analysis ends and the abnormality is corrected; after the correction, the process jumps to determine whether the parking process is abnormal until there is no abnormality in the parking process.

[0075] The perception fusion anomaly analysis system divides abnormal situations into 14 categories, namely parking system status abnormality, original image input abnormality, image preprocessing abnormality, visual model reasoning result abnormality, visual post-processing result abnormality, visual tracking result abnormality, ultrasonic raw data abnormality, ultrasonic space parking space abnormality, ultrasonic obstacle abnormality, fused obstacle abnormality, fused parking space abnormality, parking space and obstacle fusion abnormality, global odometer status abnormality, and other abnormalities;

[0076] Abnormal source analysis can be done manually or automatically.

[0077] The first method is manual analysis, which involves mapping the key node information of perception fusion in the image coordinate system, vehicle coordinate system and global coordinate system respectively. The user can analyze the source of the anomaly through the synchronous display of the key node information.

[0078] For the automated perception fusion anomaly analysis method, the analysis basis of various anomaly causes is as follows:

[0079] The analysis basis of parking system status abnormality is: verify whether the parking system state machine signal value is the expected value. If it is not the expected value or the perception fusion is not started, the analysis result is parking system status abnormality. The analysis basis of raw image input abnormality is: verify whether the size, number of channels and channel order of the raw image are consistent with the expected value. If not, the analysis result is raw image input abnormality.

[0080] The analysis basis for image preprocessing anomalies is: verify whether the size, number of channels and channel order of the image after image preprocessing are consistent with the expected values. If not, the analysis result is image preprocessing anomalies. The analysis basis for visual model reasoning anomalies is: match the vehicle-side visual model reasoning results with the high-precision perception fusion module visual model reasoning results fed back by the industrial computer by position. If there are unmatched parking spaces or obstacles, the matched target categories and other attributes are inconsistent, or the distance between key point coordinates exceeds 15cm, the analysis result is visual model reasoning results anomalies.

[0081] The analysis basis for abnormal visual post-processing results is as follows: match the vehicle-side visual post-processing results with the post-processing results of the high-precision perception fusion module fed back by the industrial computer by position. If there are unmatched parking spaces or obstacles, inconsistent attributes such as the matched target category, or the distance between the key point coordinates exceeds 15 cm, the analysis result is abnormal visual post-processing results;

[0082] The analysis basis for abnormal visual tracking results is as follows: the visual tracking results of the vehicle end are matched with the visual tracking results of the high-precision perception fusion recharge module recharged by the industrial computer through position. If there are unmatched parking spaces or obstacles, inconsistent attributes such as the matched target category, or the distance between the key point coordinates exceeds 15 cm, the analysis result is that the visual tracking results are abnormal;

[0083] The analysis basis of the abnormal ultrasonic raw data is: analyzing the range and frame rate of the ultrasonic raw data distance value. If it exceeds the normal range, the analysis result is that the ultrasonic raw data is abnormal.

[0084] The ultrasonic parking space abnormality is judged by: the ultrasonic parking space result on the vehicle side is overlapped with the drivable area result of the high-precision perception fusion recharge module of the industrial computer recharge operation. If the overlap between the ultrasonic parking space and the drivable space is less than a certain threshold, the analysis result is that the ultrasonic parking space result is abnormal.

[0085] The ultrasonic obstacle abnormality is judged by: the ultrasonic obstacle result on the vehicle side is overlapped with the drivable area result of the high-precision perception fusion recharge module of the industrial computer recharge operation. If the overlap between the ultrasonic parking space and the drivable space is greater than a certain threshold, the analysis result is that the ultrasonic obstacle result is abnormal.

[0086] The judgment method of fusion obstacle abnormality is: match the vehicle-side fusion obstacle result with the high-precision perception fusion module fusion obstacle result fed back by the industrial computer by position. If there are unmatched obstacles, the attributes such as the matched target category are inconsistent, or the key point coordinate distance exceeds 15cm, the analysis result is that the fusion obstacle result is abnormal;

[0087] The judgment method of fusion parking space abnormality is: match the fusion parking space result of the vehicle end with the fusion parking space result of the high-precision perception fusion module fed back by the industrial control computer through position. If there are unmatched parking spaces, inconsistent attributes such as the matched parking space category, or the distance between the key point coordinates exceeds 15cm, the analysis result is that the fusion parking space result is abnormal;

[0088] The judgment method for abnormal fusion of parking spaces and obstacles is as follows: the fusion results of parking spaces and obstacles on the vehicle side are matched with the fusion results of parking spaces and obstacles by the high-precision perception fusion module fed back by the industrial computer through position. If there are unmatched parking spaces, inconsistent attributes such as the types of matched parking spaces, or the distance between the key point coordinates exceeds 15 cm, the analysis result is that the fusion results of parking spaces and obstacles are abnormal;

[0089] The method for judging the abnormal state of the global odometer is as follows: if the tilt angles of the parking spaces on the same side of the fusion output parking spaces mapped to the global coordinate system are inconsistent or there are intersections of parking spaces, then the global odometer state is analyzed as abnormal;

[0090] Other abnormalities: When parking abnormalities are determined manually or by the system, and the system analysis results do not belong to the previous 13 abnormal situations, the analysis results are other abnormalities.

[0091] It also includes: an update re-injection verification function module, which is used to use the data corrected by the abnormal analysis and the visual perception results, ultrasonic perception results and fusion results before correction as the visual perception historical version results, ultrasonic perception historical version results and fusion historical version results respectively, and use multiple groups of visual perception historical version results, ultrasonic perception historical version results and fusion historical version results as a parking perception fusion fault analysis verification method to perform re-injection verification data after the software version is updated; in this module, for the updated software version, the data recorded by the real vehicle data acquisition and display function module when the abnormal problem occurs, such as the LCM data packet, the data contains the original sensor data of the perception fusion module, and the results of each key node of the perception fusion module, and the updated new version of the software and the LCM data packet are used for re-injection, and the difference between the node information obtained by the improved version of the software re-injection and the results of the historical version recorded in the LCM data packet after mapping in the upper computer can be seen in the global coordinate system and the vehicle coordinate system, and the site selection value can be configured to map the information node you are concerned about, so as to intuitively see the optimization effect of the abnormal scene and confirm whether the perception fusion abnormal problem in the LCM data scene has been solved.

[0092] It also includes: a perception fusion index evaluation function module, which is used to annotate the real vehicle data, the data corrected by abnormal analysis and the visual perception results before correction, the ultrasonic perception results and the fusion results, and assign specific values ​​to each data to calculate the data error of each node between the historical software version and the updated software version, and judge whether the updated software version meets the expected optimization effect; in this module, the true value of the data is firstly annotated, and the true value information can be obtained by the true value tool or even manual measurement. The true value information of the perception fusion module contains the parking space and obstacle feature information in the scene, including but not limited to the parking space corner point coordinates and the obstacle ground point coordinates. Through the true value coordinate information, the module can calculate the error of each key node node information of the historical software version and the updated version in this scene, and make quantitative comparison of the errors, such as parking space corner point error, parking space angle error, obstacle position error, etc., and through the difference comparison of the new and old versions of each information node, it is also convenient to know whether the effect of the new version on the improvement of abnormal problems and the impact on the results of other key link information nodes meet expectations. Through this module, combined with the scene true value information, the perception index and fusion index of the perception fusion updated version software can be quantitatively obtained.

[0093] It also includes: a scene library, which is used to store real vehicle data after true value annotation, data after abnormal analysis correction and visual perception results, ultrasonic perception results and fusion results before correction.

[0094] It also includes: scenario library recharge verification and evaluation function module, which is divided into three submodules: database scenario classification submodule, similar scenario recharge submodule and full scenario recharge submodule; among them,

[0095] The database scene classification submodule is used to classify the database scenes into scene library categories according to key scene attributes; the key scene attributes include at least lighting conditions, ground materials and parking space line types;

[0096] The same-type scenario re-injection submodule is for the scenario category corresponding to the abnormal problem solved by this software version update. One or more scenarios of similar scenario categories in the scenario library are screened out for re-injection, verifying the optimization effect of the software in similar scenarios, and conducting quantitative indicator statistical analysis. The statistical indicators include not only the final results of the perception fusion module, but also the indicators of key nodes. The key node indicators can be used for regression test analysis of the new version of the software to verify the impact of the updated and optimized code modules on the various modules of perception fusion. The data re-injection indicators of the same scenario of the new and old software versions are compared and analyzed, and a verification and evaluation report for the same scenario re-injection is automatically generated;

[0097] In the full-scenario reinjection sub-module, this module will conduct reinjection verification and evaluation on the old and new versions of the software in all scenario libraries, conduct quantitative statistical analysis on key node indicators and final indicators, automatically generate a full-scenario reinjection verification and evaluation report, and accelerate the verification and release of the new version.

[0098] It will be easily understood by those skilled in the art that the above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions and improvements made within the spirit and principles of the present invention should be included in the protection scope of the present invention.

Claims

1. A parking perception fusion fault analysis and verification method, characterized in that: include: During the parking process, real vehicle data is collected; The real vehicle data includes at least: target true value, ultrasonic raw data, visual raw data and vehicle related information; Determine whether the parking process is abnormal; When an abnormality occurs during parking, the ultrasonic raw data and visual data are sensed and fused to generate visual perception results, ultrasonic perception results, and fusion results respectively; among which, the visual perception results, ultrasonic perception results, and fusion results are used to set multiple groups of nodes for observing abnormality types according to preset standards; The visual perception results, ultrasonic perception results and each node in the fusion results are analyzed for abnormalities in sequence. When an abnormality occurs, the analysis ends and the abnormality is corrected; after the correction, the process jumps to determine whether the parking process is abnormal until there is no abnormality in the parking process.

2. The parking perception fusion fault analysis and verification method according to claim 1 is characterized in that: It also includes: using the data after abnormal analysis correction and the visual perception results, ultrasonic perception results and fusion results before correction as visual perception historical version results, ultrasonic perception historical version results and fusion historical version results respectively, and using multiple groups of visual perception historical version results, ultrasonic perception historical version results and fusion historical version results as a parking perception fusion fault analysis verification method to perform re-injection verification data after the software version is updated.

3. The parking perception fusion fault analysis and verification method according to claim 2 is characterized in that: Also includes: The real vehicle data, the data after abnormal analysis correction, and the visual perception results, ultrasonic perception results, and fusion results before correction are annotated with true values, and each data is assigned a specific numerical value to calculate the data error of each node between the historical software version and the updated software version, and to determine whether the updated software version meets the expected optimization effect.

4. The parking perception fusion fault analysis and verification method according to claim 3 is characterized in that: Also includes: A scene library is set up to store the real vehicle data after truth value annotation, the data after abnormal analysis correction and the visual perception results, ultrasonic perception results and fusion results before correction.

5. The parking perception fusion fault analysis and verification method according to claim 4 is characterized in that: Also includes: Classify the database scenes into scene library categories based on key scene attributes; Key scene attributes include at least lighting conditions, ground material, and parking line type; For the scenario categories corresponding to the abnormal problems solved by the software version update, one or more scenarios of similar scenario categories in the scenario library are screened out for re-injection to verify the optimization effect of the software under similar scenario categories, and quantitative indicator statistical analysis is performed based on the annotated true values; comparative analysis is performed on the re-injection indicators of the same scenario data in the software versions before and after the update, and a similar scenario re-injection verification evaluation report is generated; Conduct back-injection verification and evaluation on all scenarios in the scenario library for versions of the software before and after the update, perform quantitative indicator statistical analysis, generate a full-scenario back-injection verification and evaluation report, and speed up version update verification and release.

6. A parking perception fusion fault analysis and verification system, characterized in that: It at least includes the real vehicle data collection and display function module and the host computer perception fusion abnormality analysis function module, among which, The real vehicle data collection and display function module is used to collect the real vehicle data during the parking process; the real vehicle data at least includes: target true value, ultrasonic raw data, visual raw data and vehicle related information; The host computer perception fusion abnormal analysis function module is used to determine whether the parking process is abnormal; When an abnormality occurs during parking, the ultrasonic raw data and visual data are sensed and fused to generate visual perception results, ultrasonic perception results, and fusion results respectively; among which, the visual perception results, ultrasonic perception results, and fusion results are used to set multiple groups of nodes for observing abnormality types according to preset standards; The visual perception results, ultrasonic perception results and each node in the fusion results are analyzed for abnormalities in sequence. When an abnormality occurs, the analysis ends and the abnormality is corrected; after the correction, the process jumps to determine whether the parking process is abnormal until there is no abnormality in the parking process.

7. The parking perception fusion fault analysis and verification system according to claim 6 is characterized in that: Also includes: An updated re-injection verification function module is used to use the data after abnormal analysis correction and its visual perception results, ultrasonic perception results and fusion results before correction as visual perception historical version results, ultrasonic perception historical version results and fusion historical version results respectively, and use multiple sets of visual perception historical version results, ultrasonic perception historical version results and fusion historical version results as a parking perception fusion fault analysis verification method to perform re-injection verification data after the software version is updated.

8. The parking perception fusion fault analysis and verification system according to claim 7 is characterized in that: Also includes: The perception fusion index evaluation function module is used to annotate the true value of real vehicle data, data after abnormal analysis correction, and visual perception results, ultrasonic perception results, and fusion results before correction, and assign specific numerical values ​​to each data to calculate the data error of each node between the historical software version and the updated software version, and determine whether the updated software version meets the expected optimization effect.

9. The parking perception fusion fault analysis and verification system according to claim 8, characterized in that: Also includes: The scene library is used to store real vehicle data after true value annotation, data after abnormal analysis correction, and visual perception results, ultrasonic perception results, and fusion results before correction.

10. The parking perception fusion fault analysis and verification system according to claim 5, characterized in that: Also includes: The scenario library recharge verification and evaluation function module is divided into three submodules: database scenario classification submodule, similar scenario recharge submodule and full scenario recharge submodule; among them, The database scene classification submodule is used to classify the database scenes into scene library categories according to key scene attributes; the key scene attributes include at least lighting conditions, ground materials and parking space line types; The same-type scenario re-injection submodule is used to select one or more scenarios of similar scenario categories in the scenario library for re-injection for the scenario categories corresponding to the abnormal problems solved by the software version update, verify the optimization effect of the software under similar scenario categories, and perform quantitative indicator statistical analysis based on the annotated true values; compare and analyze the re-injection indicators of the same-type scenario data in the software versions before and after the update, and generate a similar scenario re-injection verification evaluation report; The full-scenario reinjection submodule is used to perform reinjection verification and evaluation on all scenarios in the scenario library of the software versions before and after the update, conduct quantitative indicator statistical analysis, generate a full-scenario reinjection verification and evaluation report, and speed up version update verification and release.

Citation Information

Cited By

  • Vehicle parking test method, parking abnormity management method and computer equipment

    CN120689803A

  • Automatic parking method and system combined with digital road

    CN121214712A