External damage hidden danger identification method and system based on AI image and radar dual verification

Abstract

The invention discloses an external damage hidden danger identification method and system based on AI image and radar dual verification, and relates to the technical field of artificial intelligence and multi-source perception fusion, and the method comprises the following steps: extracting the motion trail features of perception data in a target region, solving a coordinate transformation matrix through an iterative nearest point algorithm, carrying out the time sequence alignment, and carrying out the recognition of the motion trail features of the perception data; obtaining the calibrated sensing data; taking the calibrated image data and the calibrated radar data as input, and outputting a radar detection result and a visual identification result; projecting radar coordinates to an image coordinate system according to a coordinate transformation matrix based on a visual identification result and a radar detection result, outputting a space overlapping degree, and generating a matching result set; carrying out confidence fusion on the matching result set through a Bayesian probability model, and judging an external damage hidden danger level in combination with a radar detection result; according to the invention, through the AI vision and radar dual verification fusion technology, the problems of inconsistent multi-source perception and low external damage hidden danger identification precision are solved.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of artificial intelligence and multi-source perception fusion, and more particularly, relates to an external damage hidden danger identification method and system based on AI image and radar double verification. BACKGROUND

[0002] With the continuous expansion of the power system scale and the continuous extension of the transmission line, external damage (referred to as "external damage") has become one of the main hidden dangers threatening the safe operation of the power grid. External damage hidden dangers usually come from external factors such as construction machinery intrusion, vehicle approach, falling object impact, vegetation intrusion or human damage. If these hidden dangers cannot be identified and disposed in time, they will easily cause transmission line breakage, equipment damage, power supply interruption, and even cause large-scale power failure accidents. In recent years, with the rapid development of artificial intelligence and sensing technology, the power operation and maintenance department has gradually introduced an external damage monitoring system based on AI visual recognition, trying to realize automatic identification and early warning of external damage targets through image analysis. However, in the complex field operation environment, the single visual perception mode still faces many technical bottlenecks, and the stability and reliability of its identification are difficult to meet the safety monitoring needs of the power system all-weather and all-scenarios.

[0003] For example, the invention patent with publication number CN118898810B discloses a transmission cable external damage hidden danger identification method, device, electronic equipment and storage medium. The method comprises: acquiring a monitoring image for monitoring construction machinery and transmission cables, the monitoring image including a first image area where the construction machinery is located and a second image area where the transmission cables are located; if the first image area and the second image area are at the same relative depth in the monitoring image, then the actual distance between the construction machinery and the transmission cables is calculated according to the first image area and the second image area; the relative depth represents the front-back relationship between the construction machinery in the first image area and the transmission cables in the second image area in the monitoring image; if the actual distance is less than a preset threshold, it is determined that the transmission cables have external damage hidden dangers. The method can improve the accuracy of transmission cable external damage hidden danger identification and accurately alarm the transmission cable external damage hidden danger.

[0004] For example, the invention patent with publication number CN118885880A discloses a power distribution network external damage hidden danger intelligent identification method and device, relating to power grid safety management technical field, the method comprises: using millimeter wave radar dynamic monitoring the spatial environment of the predetermined power distribution network, obtaining real-time external damage hidden danger detection information; extract the first real-time hidden danger information in the real-time external damage hidden danger detection information, judge whether it conforms to the predetermined tracking constraint; if it conforms, activate the laser radar to track monitoring according to the analyzed monitoring strategy, obtain the first tracking monitoring information; read the predetermined external damage hidden danger characteristic, match the characteristic in the detection information and the tracking monitoring information, generate hidden danger characteristic parameter; input the characteristic parameter to the integrated intelligent model, identify the danger category; combine the hidden danger characteristic parameter and the predicted category to generate danger warning information, and start the emergency warning. Further, the technical effect of improving the identification accuracy and real-time is achieved.

[0005] The above disclosed technical solution at least has the following technical problems: Traditional radar detection of external damage hidden dangers usually adopts fixed threshold or single-frame clustering algorithm, which is difficult to distinguish targets from noise in complex background, lacks inter-frame trajectory correlation mechanism, and the matching of radar and image usually relies on simple geometric overlap or nearest point strategy, which is difficult to cope with complex scenes such as multi-target intersection and inconsistent scale. In view of the above problems, the present application provides a solution. SUMMARY

[0006] In order to overcome the above-mentioned defects of the prior art, the embodiments of the present application provide an external damage hidden danger identification method and system based on AI image and radar double verification, which solves the problems of multi-source perception inconsistency and low external damage hidden danger identification accuracy through AI vision and radar double verification fusion technology.

[0007] To achieve the above-mentioned purpose, the present application provides the following technical solutions: The external damage hidden danger identification method based on AI image and radar double verification comprises: extracting the motion trajectory features of the perception data in the target area, obtaining the coordinate transformation matrix through the iterative nearest point algorithm, and performing time sequence alignment to obtain the calibrated perception data, the perception data including image data and radar data; taking the calibrated image data as input, adjusting the pre-trained general vision model through transfer learning to obtain an external damage identification vision model, and performing inference on the input image to output the vision recognition result; taking the calibrated radar data as input, performing constant false alarm rate detection clustering and trajectory tracking processing to output the radar detection result; projecting the radar coordinates to the image coordinate system according to the coordinate transformation matrix based on the vision recognition result and the radar detection result, outputting the spatial overlap degree, and generating a matching result set; performing confidence fusion on the matching result set through the Bayesian probability model, and combining the radar detection result to determine the external damage hidden danger level.

[0008] In a preferred embodiment, the motion trajectory feature of the perception data in the extraction target region is obtained by an iterative closest point algorithm to obtain a coordinate transformation matrix, and time sequence alignment is performed to obtain calibrated perception data, which is specifically as follows: obtaining the time stamp corresponding to the perception data, calculating the time difference between adjacent frames through a unified clock reference, regarding the frames with a time difference between adjacent frames below a preset threshold as synchronous matching samples, and generating a synchronous data set; taking the synchronous data set as input, performing motion detection on each image frame through a background difference method, extracting the center point coordinate set of the motion region as the image target point set; extracting the point cloud center coordinates of the radar frame with a reflection intensity higher than a set threshold through a target detection and clustering algorithm to obtain a radar target point set; according to the time index matching relationship, associating the image target points with the radar target points at the frame level to obtain a time matching point pair set; taking the time matching point pair set as input, converting each radar target point into a Cartesian coordinate point, establishing a planar coordinate representation of the radar target point, and obtaining a radar planar point set; obtaining an initial rotation matrix and translation vector according to the historical calibration results of the device, and constructing an extrinsic parameter matrix; projecting the radar planar point set to the image coordinate system through the extrinsic parameter matrix by the camera intrinsic parameter matrix to obtain a projection point set, and corresponding the projection point set and the image target point set according to the time index to form a coarse registration data set; taking the coarse registration data set as input, constructing an optimization objective function of the iterative closest point algorithm, and solving the rotation matrix and the translation vector through the least square method to output an accurate spatial registration matrix; taking the synchronous data set and the spatial registration matrix as input, constructing a time sequence filtering model, dynamically correcting the time index, and obtaining a time correction parameter; adjusting the time index of the image frame and the radar frame according to the time correction parameter to generate a data set corrected by time sequence filtering; performing spatial coordinate transformation on the corrected radar frame according to the spatial registration matrix to obtain an aligned radar frame, and synchronously retaining the corresponding image frame to form a registration frame pair; summarizing all registration frame pairs to generate a final calibrated data set.

[0009] In a preferred embodiment, the time sequence filtering model is constructed by taking the synchronous data set and the spatial registration matrix as input, dynamically correcting the time index, and obtaining a time correction parameter, which is specifically as follows: mapping the radar points to the image coordinate system through the spatial registration matrix, and calculating the spatial coincidence error between each frame of radar data and the corresponding image data; obtaining the time sequence observation value of the frame pair based on the average point-to-point distance deviation between the radar target point set and the image target point set; associating the time sequence observation value with its corresponding time stamp to form a time sequence error sequence, and denoising the time sequence error sequence through a sliding window mechanism; predicting and updating the denoised time sequence error sequence through Kalman filtering, and the first component of the state quantity of the filter after updating is the current time correction parameter.

[0010] In a preferred embodiment, the pre-trained general visual model is adjusted by transfer learning using the calibrated image data as input to obtain an external damage identification visual model, and the input image is inferred to output a visual recognition result, specifically as follows: a training data set is constructed using the calibrated image data and labeled external damage target information, and the output layer of the pre-trained visual model is replaced with a classification layer for external damage target identification; the training data set is input into the modified visual model, and the error between the predicted result and the true label is output, and the model parameters are updated by an optimization algorithm to obtain a preliminary external damage identification model; the preliminary external damage identification model is trained in stages, and the model performance is regulated by a validation set during the training process; new calibrated image data is input into the trained external damage identification visual model for forward propagation to obtain an initial visual recognition result for each image frame; the initial visual recognition result is subjected to confidence screening and time sequence post-processing to generate a final corrected visual recognition result set.

[0011] In a preferred embodiment, the calibrated radar data is input to perform constant false alarm rate detection clustering and trajectory tracking processing to output a radar detection result, specifically as follows: the calibrated radar data frame is subjected to signal preprocessing; each range-Doppler cell in the preprocessed radar data is detected by a constant false alarm rate detection algorithm, the background noise power is dynamically output, and an adaptive detection threshold is calculated, and when the signal power exceeds the threshold, it is determined that there is a potential external damage target echo, which is marked as a target candidate point; the target candidate points are clustered according to spatial proximity or velocity similarity, and if the distance or velocity difference between adjacent points is below a preset threshold, they are classified into the same target cluster, and a target cluster set is output; the target clusters in consecutive radar frames are matched between frames, and the clusters that meet the consistency conditions in spatial position and velocity change in adjacent frames are matched as the same physical target trajectory, and in the case where they cannot be matched, the trajectory is newly created or terminated to obtain an initial trajectory set; the target position and velocity of the initial trajectory set are predicted and corrected by a Kalman filter algorithm to obtain a filtered trajectory set; the filtered trajectory set is subjected to stability analysis, and trajectories that do not meet the stability threshold are removed to form a stable target trajectory set; the trajectory data in the stable target trajectory set is summarized to form a radar detection result set.

[0012] In a preferred embodiment, the step of projecting the radar coordinates to the image coordinate system based on the coordinate transformation matrix of the visual recognition results and radar detection results, outputting the spatial overlap, and generating a matching result set is as follows: Each target position in the radar detection result set is projected to the image coordinate system through a matrix to obtain a radar projection point set; the visual detection box of each frame is traversed, and projection points falling into the region of the detection box are retrieved from the projection point set to form candidate matching groups; for each candidate matching group, the spatial overlap between the visual detection box region and the region where the projection point is located is calculated; if multiple radar projection points fall into the same detection box, the distribution density of the projection points is used as a supplementary weight to obtain a corrected spatial overlap; the corrected spatial overlap is weighted proportionally with the radar confidence to output a matching confidence; if the matching confidence is lower than a preset threshold, it is marked as no match or low-confidence match; otherwise, the radar target projection points of the visual detection box are determined to be valid matching pairs; valid matching pairs of all frames are collected to construct a matching result set.

[0013] In a preferred embodiment, the step of fusing confidence scores on the matching result set using a Bayesian probability model and combining this with radar detection results to determine the level of external damage hazard is as follows: Detection features of the corresponding targets are extracted from the radar detection results, and these features are correlated with the confidence scores and locations in the visual recognition results to form a fused input dataset. Using this fused input dataset as input, a joint confidence fusion model is constructed using a Bayesian probability model. In the joint confidence fusion model, the visual recognition results are considered prior observations, and the radar detection results are considered independent observation evidence. The probability of target existence is dynamically corrected using the Bayesian probability update principle. Based on the corrected probability of target existence, a fused confidence score value is obtained for each externally damaged target. The fused confidence score and radar distance information are used together as input features, and the hazard level of the externally damaged target is determined based on preset confidence and distance thresholds. The hazard level results for all targets are summarized to form a set of external damage hazard level determination results.

[0014] The system for identifying external damage hazards based on AI-powered image and radar dual verification includes a data acquisition module, a visual recognition module, a radar detection module, a matching module, and a judgment module, with interconnections between the modules. The data acquisition module extracts the motion trajectory features of the perceived data in the target area, obtains the coordinate transformation matrix through an iterative nearest-point algorithm, and performs temporal alignment to obtain calibrated perceived data, which includes image data and radar data. The visual recognition module uses the calibrated image data as input, performs transfer learning to adjust a pre-trained general visual model to obtain an external damage identification visual model, and infers from the input image to output the visual recognition result. The radar detection module uses the calibrated radar data as input, performs constant false alarm rate detection clustering and trajectory tracking processing, and outputs the radar detection result. The matching module projects the radar coordinates to the image coordinate system based on the coordinate transformation matrix, outputs the spatial overlap, and generates a matching result set. The judgment module performs confidence fusion on the matching result set using a Bayesian probability model and, combined with the radar detection result, determines the level of external damage hazard.

[0015] The technical effects and advantages of the present invention, a method and system for identifying external damage hazards based on AI image and radar dual verification, are as follows: 1. This invention introduces the Iterative Closest Point (ICP) algorithm to establish a precise coordinate transformation matrix between radar and image data, and combines this with a unified clock reference to achieve timing alignment, ensuring consistency of spatial and temporal data from different sensors within the same reference coordinate system. This process not only eliminates errors caused by sampling delays and geometric distortions between sensors, but also dynamically corrects timing errors through Kalman filtering, achieving high-precision synchronization of both spatial and temporal calibrations, laying a solid mathematical foundation for subsequent multi-source information fusion. Compared with traditional static calibration methods, this method has the advantages of adaptability and dynamic updating, maintaining stable system registration even under environmental changes or slight equipment offsets.

[0016] 2. This invention employs an adaptive detection mechanism combining constant false alarm rate (CFAR) detection with clustering and trajectory tracking at the radar detection and trajectory analysis level. This mechanism can dynamically determine the threshold of target signals in complex background noise environments and extract stable targets through clustering and multi-frame trajectory correlation. Furthermore, Kalman filtering is used to predict and correct target position and velocity, effectively eliminating false alarms and noisy trajectories, ensuring the accuracy and continuity of radar detection results. This dynamic detection and prediction mechanism enables the system to track the motion characteristics of potential targets in real time, providing reliable physical support for determining the level of external damage. Attached Figure Description

[0017] Figure 1This is a schematic diagram of the external damage hazard identification method based on AI image and radar dual verification according to the present invention.

[0018] Figure 2 This is a schematic diagram of the system structure of the external damage hazard identification method based on AI image and radar dual verification of the present invention. Detailed Implementation

[0019] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those of ordinary skill in the art without creative effort are within the scope of protection of the present invention.

[0020] Example 1, Figure 1 This invention presents a method for identifying external damage hazards based on dual verification of AI images and radar, including: S1, extract the motion trajectory features of the sensing data in the target area, obtain the coordinate transformation matrix through the iterative nearest point algorithm, and perform temporal alignment to obtain the calibrated sensing data, which includes image data and radar data; In this embodiment, the motion trajectory features of the sensing data in the target area are extracted, the coordinate transformation matrix is ​​obtained through the iterative nearest point algorithm, and temporal alignment is performed to obtain the calibrated sensing data, as follows: The timestamps corresponding to the sensing data are obtained. The time difference between adjacent frames is calculated using a unified clock reference (such as GPS timing or system synchronization signal). Frame pairs with a time difference between adjacent frames that is lower than a preset threshold are regarded as synchronization matching samples, and a synchronization dataset is generated. Using a synchronous dataset as input, motion detection is performed on each image frame using the background subtraction method to extract the set of center point coordinates of the motion region as the image target point set, which is used to describe the planar position distribution of the externally damaged target; The radar target point set is obtained by extracting the center coordinates of point clouds in radar frames where the reflection intensity is higher than a set threshold through target detection and clustering algorithms (such as CFAR+DBSCAN). Based on the time index matching relationship, the image target points and radar target points are associated at the frame level to obtain a set of time matching point pairs; Using the time-matching point pair set as input, each radar target point is converted into a Cartesian coordinate point, and a planar coordinate representation of the radar target point is established to obtain the radar planar point set; Based on the historical calibration results of the equipment, the initial rotation matrix and translation vector are obtained, and the extrinsic parameter matrix is ​​constructed; The radar plane point set is projected onto the image coordinate system through the camera intrinsic parameter matrix and the extrinsic parameter matrix to obtain the projected point set. The camera intrinsic parameter matrix is ​​a set of key parameters that describe the camera imaging geometry and is used to project three-dimensional spatial points onto a two-dimensional image plane. The projection point set and the image target point set are mapped according to time index to form a coarse registration dataset; Using the coarse registration dataset as input, we construct the optimization objective function of the iterative nearest point algorithm, and iteratively solve the rotation matrix and translation vector using the least squares method to output the accurate spatial registration matrix. Using the synchronized dataset and spatial registration matrix as input, a time-series filtering model (Kalman filter or particle filter) is constructed to dynamically correct the time index and obtain the time correction parameters. Adjust the time index of image frames and radar frames according to the time correction parameters to generate a dataset after time-series filtering correction. The corrected radar frame is transformed into spatial coordinates based on the spatial registration matrix to obtain the aligned radar frame, and the corresponding image frame is retained simultaneously to form a registration frame pair. All registered frame pairs are aggregated to generate the final calibrated dataset.

[0021] The objective function of the iterative nearest point algorithm is as follows: In the formula: It is an optimization error function, representing the registration error between the image and the radar point set in the current iteration. The optimization objective is to minimize this value. It is the position vector of the i-th image target point in the camera coordinate system. It is a rotation matrix, representing the rotation relationship from the radar coordinate system to the camera coordinate system. It is the position vector of the i-th radar target point in the radar coordinate system. It is a spatial translation vector, and i is the index of the matching point.

[0022] In this embodiment, a temporal filtering model is constructed using the synchronized dataset and spatial registration matrix as input, and the time index is dynamically corrected to obtain the time correction parameters, as detailed below: The radar points are mapped to the image coordinate system using a spatial registration matrix, and the spatial overlap error between each frame of radar data and the corresponding image data is calculated. The temporal observation values ​​of the frame pair are obtained based on the average point-to-point distance deviation between the radar target point set and the image target point set; By associating time-series observations with their corresponding timestamps, a time-series error sequence is formed; Denoising of the temporal error sequence is achieved using a sliding window mechanism (e.g., the most recent N frames). The denoised time-series error sequence is predicted and updated using Kalman filtering. The first component of the updated state variable is the current time correction parameter.

[0023] S2, using the calibrated image data as input, performs transfer learning to adjust the pre-trained general visual model to obtain the external damage recognition visual model, and performs reasoning on the input image to output the visual recognition result; In this embodiment, the calibrated image data is used as input, and the pre-trained general visual model is adjusted through transfer learning to obtain an external damage recognition visual model. The model then performs inference on the input image and outputs the visual recognition result, as detailed below: A training dataset was constructed using calibrated image data and labeled external damage target information; The output layer of the pre-trained visual model is replaced with a classification layer suitable for external damage target recognition, so that the output can represent the external damage category or the probability of its existence, while retaining the weights of the original feature extraction layer to ensure the transfer learning effect. The training dataset is input into the modified visual model, the error between the predicted result and the true label is output, and the model parameters are updated by optimization algorithm (such as gradient descent) to obtain a preliminary external damage recognition model. The initial external damage identification model is trained in stages: first, the original feature extraction layer is fixed, and only the classification layer is trained to quickly adapt to the external damage target; then, the network parameters are finely adjusted as a whole to enable the model to fully learn the external damage features. During training, the model performance is monitored using a validation set. If the validation error reaches the set convergence condition or an overfitting trend occurs, the training process is optimized by pausing or adjusting the learning rate to ensure model stability. The new calibrated image data is input into the trained external damage recognition visual model and forward propagation is performed to obtain the initial visual recognition result for each image frame. The initial visual recognition result includes the target category, the probability of existence, and the possible detection location. The initial visual recognition results are subjected to confidence screening and temporal post-processing: low confidence detections are filtered based on the predicted probability threshold, and temporal consistency checks or tracking smoothing are performed on the visual recognition results of consecutive frames to reduce false detections and missed detections, and finally, a set of corrected visual recognition results is generated.

[0024] S3 takes the calibrated radar data as input, performs constant false alarm rate detection clustering and trajectory tracking processing, and outputs the radar detection results; In this embodiment, calibrated radar data is used as input to perform constant false alarm rate (CFAR) detection clustering and trajectory tracking processing, and the radar detection results are output as follows: The calibrated radar data frames are preprocessed, including noise suppression, range-Doppler transformation and amplitude normalization, in order to remove environmental interference and unify the energy scale of different frames. The constant false alarm rate (CFAR) detection algorithm is used to detect each range-Doppler cell in the preprocessed radar data, dynamically output the background noise power, and calculate the adaptive detection threshold. When the signal power exceeds the threshold, it is determined that there is a potential external target echo and it is marked as a target candidate point. The target candidate points are clustered according to spatial proximity or velocity similarity. If the distance or velocity difference between adjacent points is less than a preset threshold, they are classified into the same target cluster and the target cluster set is output. Each cluster contains the target's center position, average velocity, and scattering intensity. Inter-frame matching is performed on target clusters in consecutive radar frames. Clusters whose spatial position and velocity changes in adjacent frames meet the consistency condition are matched as the same physical target trajectory. In the case where no matching is possible, a new trajectory is created or terminated to obtain the initial trajectory set. The target position and velocity of the initial trajectory set are predicted and corrected by the Kalman filter algorithm, which smooths random fluctuations and eliminates noise interference, resulting in a filtered trajectory set. Stability analysis is performed on the filtered trajectory set. The velocity continuity, reflection intensity change rate and trajectory duration of each trajectory in the time series are calculated. Trajectories that do not meet the stability threshold are removed, and a set of reliable stable target trajectories is retained. All trajectory data in the stable target trajectory set are aggregated to form a radar detection result set containing the target's spatial position, velocity, and confidence level.

[0025] S4, based on the coordinate transformation matrix, projects the radar coordinates to the image coordinate system using the visual recognition results and radar detection results, outputs the spatial overlap, and generates a matching result set; In this embodiment, the visual recognition results and radar detection results are projected onto the image coordinate system using a coordinate transformation matrix, the spatial overlap is output, and a matching result set is generated, as follows: Each target location in the radar detection result set is projected onto the image coordinate system using a matrix to obtain the radar projection point set; The visual detection box of each frame image is traversed, and the projection points that fall into the region of the detection box are retrieved from the projection point set to form a candidate matching group. The visual detection box refers to the rectangular region output in the image that is used to characterize the spatial location of the external target. For each candidate matching group, calculate the spatial overlap between the visual detection box region and the region where the projection point is located; When multiple radar projection points fall into the same detection frame, the distribution density of the projection points is used as a supplementary weight to obtain the corrected spatial overlap. The corrected spatial overlap is weighted proportionally with the radar confidence level to output the matching confidence level; If the matching confidence is lower than the preset threshold, it is marked as no match or low confidence match; otherwise, the radar target projection point of the visual detection box is determined to be a valid matching pair. Collect all valid matching pairs from all frames and construct a matching result set, which includes visual recognition results and radar detection results.

[0026] S5 uses a Bayesian probability model to fuse the confidence scores of the matching result set and combines them with radar detection results to determine the level of external damage hazards; In this embodiment, the confidence level of the matching result set is fused using a Bayesian probability model, and combined with radar detection results, the level of external damage hazard is determined, as follows: The detection features of the corresponding target are extracted from the radar detection results, and the detection features are correlated with the confidence and position in the visual recognition results to form a fusion input dataset. The detection features include distance information, reflection intensity and target movement trend. Using the fused input dataset as input, a joint confidence fusion model is constructed through a Bayesian probability model; In the joint confidence fusion model, visual recognition results are regarded as prior observations, radar detection results are regarded as independent observation evidence, and the probability of target existence is dynamically corrected through the Bayesian probability update principle. Based on the corrected probability of the target's existence, the fusion confidence value of each external damage target is obtained, which represents the credibility of the system's comprehensive judgment that the target is a real external damage hazard. The system uses both confidence level and radar distance information as input features, and determines the risk level of external targets based on preset confidence and distance thresholds. When the fusion confidence level is high and the radar ranging results show that the target is close to the safety boundary of the power line, it is judged as a high-level external damage hazard. When the fusion confidence level is moderate or the target is far from the safety boundary, it is judged as a medium-level hidden danger; When the confidence level is low or the detection results are unstable, it is judged as a low-level hidden danger or a false alarm; The results of the hazard levels of all targets are summarized to form a set of external damage hazard level judgment results.

[0027] The process of using a fused input dataset as input and constructing a joint confidence fusion model through a Bayesian probability model is as follows: The radar detection results are used to obtain the detection threshold through a constant false alarm rate algorithm, and the radar detection confidence is output based on the ratio of the target echo signal to the threshold. Visual confidence is obtained by estimating the distribution of visual detection scores on a labeled validation set under conditions of target presence and absence, and mapping the current score to a posterior probability based on Bayes' theorem. Based on the confidence scores of visual recognition, radar detection, and matching, a joint confidence fusion model is constructed using a Bayesian probability model.

[0028] The joint confidence fusion model is as follows: In the formula: It is the joint confidence level. It is the confidence level of visual recognition. It is the confidence level of radar detection. It is the matching confidence level.

[0029] Example 2, Figure 2 The present invention provides a system for identifying external damage hazards based on AI image and radar dual verification, comprising a data acquisition module, a visual recognition module, a radar detection module, a matching module, and a judgment module, with connections between the modules; The data acquisition module is used to extract the motion trajectory features of the sensing data in the target area, obtain the coordinate transformation matrix through the iterative nearest point algorithm, and perform time-series alignment to obtain calibrated sensing data, which includes image data and radar data. The visual recognition module is used to take calibrated image data as input, perform transfer learning on a pre-trained general visual model to obtain an external damage recognition visual model, and perform inference on the input image to output the visual recognition result. The radar detection module is used to perform constant false alarm rate detection, clustering, and trajectory tracking processing on calibrated radar data as input, and output radar detection results. The matching module is used to project the radar coordinates to the image coordinate system based on the coordinate transformation matrix, which combines the visual recognition results and radar detection results, outputs the spatial overlap, and generates a matching result set. The judgment module is used to perform confidence fusion on the matching result set through a Bayesian probability model and combine it with radar detection results to determine the level of external damage risks.

[0030] The above formulas are all dimensionless calculations. The formulas are derived from software simulations based on a large amount of collected data to obtain the most recent real-world results. The preset parameters in the formulas are set by those skilled in the art according to the actual situation.

[0031] The above embodiments can be implemented, in whole or in part, by software, hardware, firmware, or any other combination thereof. When implemented using software, the above embodiments can be implemented, in whole or in part, in the form of a computer program product.

[0032] Those skilled in the art will recognize that the modules and algorithm steps of the various examples described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are implemented in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of this application.

[0033] In addition, the functional modules in the various embodiments of this application can be integrated into one processing module, or each module can exist physically separately, or two or more modules can be integrated into one module.

[0034] The above description is merely a specific embodiment of this application, but the scope of protection of this application is not limited thereto. Any variations or substitutions that can be easily conceived by those skilled in the art within the scope of the technology disclosed in this application should be included within the scope of protection of this application. Therefore, the scope of protection of this application should be determined by the scope of the claims.

[0035] In conclusion, the above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the protection scope of the present invention.

Claims

1. A method for identifying external damage hazards based on dual verification of AI images and radar, characterized in that, include: The motion trajectory features of the sensing data in the target area are extracted, the coordinate transformation matrix is ​​obtained by the iterative nearest point algorithm, and time alignment is performed to obtain the calibrated sensing data, which includes image data and radar data. Using calibrated image data as input, the pre-trained general visual model is adjusted through transfer learning to obtain an external damage recognition visual model, and inference is performed on the input image to output the visual recognition result. Using calibrated radar data as input, constant false alarm rate detection clustering and trajectory tracking processing are performed, and radar detection results are output. The visual recognition results and radar detection results are combined, and the radar coordinates are projected onto the image coordinate system according to the coordinate transformation matrix. The spatial overlap is output, and a matching result set is generated. The confidence level of the matching result set is fused using a Bayesian probability model, and combined with radar detection results, the level of external damage hazard is determined.

2. The method for identifying external damage hazards based on dual verification of AI image and radar as described in claim 1, characterized in that, The motion trajectory features of the sensed data in the target area are extracted, the coordinate transformation matrix is ​​obtained through the iterative nearest point algorithm, and temporal alignment is performed to obtain the calibrated sensed data, as follows: The timestamps corresponding to the perceived data are obtained. The time difference between adjacent frames is calculated using a unified clock reference. Frame pairs with a time difference between adjacent frames that is lower than a preset threshold are regarded as synchronization matching samples, and a synchronization dataset is generated. Using a synchronized dataset as input, motion detection is performed on each image frame using the background subtraction method to extract the set of center point coordinates of the motion region as the image target point set; The radar target point set is obtained by extracting the center coordinates of point clouds in radar frames where the reflection intensity is higher than a set threshold through target detection and clustering algorithms. Based on the time index matching relationship, the image target points and radar target points are associated at the frame level to obtain a set of time matching point pairs; Using the time-matching point pair set as input, each radar target point is converted into a Cartesian coordinate point, and a planar coordinate representation of the radar target point is established to obtain the radar planar point set; Based on the historical calibration results of the equipment, the initial rotation matrix and translation vector are obtained, and the extrinsic parameter matrix is ​​constructed; The radar plane point set is projected onto the image coordinate system through the camera intrinsic parameter matrix and the image target point set to obtain the projected point set. The projected point set is then matched with the image target point set according to the time index to form a coarse registration dataset. Using the coarse registration dataset as input, we construct the optimization objective function of the iterative nearest point algorithm, and iteratively solve the rotation matrix and translation vector using the least squares method to output the accurate spatial registration matrix. Using the synchronized dataset and spatial registration matrix as input, a time-series filtering model is constructed, and the time index is dynamically corrected to obtain the time correction parameters; Adjust the time index of image frames and radar frames according to the time correction parameters to generate a dataset after time-series filtering correction. The corrected radar frame is transformed into spatial coordinates based on the spatial registration matrix to obtain the aligned radar frame, and the corresponding image frame is retained simultaneously to form a registration frame pair. All registered frame pairs are aggregated to generate the final calibrated dataset.

3. The method for identifying external damage hazards based on dual verification of AI image and radar according to claim 2, characterized in that, The process involves constructing a time-series filtering model using a synchronized dataset and a spatial registration matrix as inputs, dynamically correcting the time index, and obtaining time correction parameters, as detailed below: The radar points are mapped to the image coordinate system using a spatial registration matrix, and the spatial overlap error between each frame of radar data and the corresponding image data is calculated. The temporal observation values ​​of the frame pair are obtained based on the average point-to-point distance deviation between the radar target point set and the image target point set; The time series observations are correlated with their corresponding timestamps to form a time series error sequence, and the time series error sequence is denoised using a sliding window mechanism; The denoised time-series error sequence is predicted and updated using Kalman filtering. The first component of the updated state variable is the current time correction parameter.

4. The method for identifying external damage hazards based on dual verification of AI image and radar according to claim 3, characterized in that, The process involves using calibrated image data as input, performing transfer learning adjustments on a pre-trained general visual model to obtain an external damage recognition visual model, and then inferring from the input image to output the visual recognition result, as detailed below: A training dataset was constructed using calibrated image data and labeled external damage target information, and the output layer of the pre-trained visual model was replaced with a classification layer for external damage target recognition. The training dataset is input into the modified visual model, the error between the predicted result and the true label is output, and the model parameters are updated by the optimization algorithm to obtain a preliminary external damage recognition model. The initial external damage identification model is trained in stages, and the model performance is adjusted through the validation set during the training process. The new calibrated image data is input into the trained external damage recognition visual model and forward propagated to obtain the initial visual recognition result for each image frame. The initial visual recognition results are subjected to confidence screening and time-series post-processing to generate the final corrected set of visual recognition results.

5. The method for identifying external damage hazards based on dual verification of AI image and radar according to claim 4, characterized in that, The process involves using calibrated radar data as input, performing constant false alarm rate (CFAR) detection clustering and trajectory tracking, and outputting radar detection results, as detailed below: Perform signal preprocessing on the calibrated radar data frames; The constant false alarm rate (CFAR) detection algorithm is used to detect each range-Doppler cell in the preprocessed radar data, dynamically output the background noise power, and calculate the adaptive detection threshold. When the signal power exceeds the threshold, it is determined that there is a potential external target echo and it is marked as a target candidate point. The target candidate points are clustered according to spatial proximity or velocity similarity. If the distance or velocity difference between adjacent points is less than a preset threshold, they are classified into the same target cluster, and the target cluster set is output. Inter-frame matching is performed on target clusters in consecutive radar frames. Clusters whose spatial position and velocity changes in adjacent frames meet the consistency condition are matched as the same physical target trajectory. In the case where no matching is possible, a new trajectory is created or terminated to obtain the initial trajectory set. The target position and velocity of the initial trajectory set are predicted and corrected by the Kalman filter algorithm to obtain the filtered trajectory set. Stability analysis is performed on the filtered trajectory set, and trajectories that do not meet the stability threshold are removed to form a stable target trajectory set; The trajectory data from the stable target trajectory set are aggregated to form a radar detection result set.

6. The method for identifying external damage hazards based on dual verification of AI image and radar according to claim 5, characterized in that, The process involves projecting the visual recognition results and radar detection results onto the image coordinate system using a coordinate transformation matrix, outputting the spatial overlap, and generating a matching result set, as detailed below: Each target location in the radar detection result set is projected onto the image coordinate system using a matrix to obtain the radar projection point set; Traverse the visual detection bounding boxes of each frame image and retrieve the projection points that fall into the region of the detection bounding box from the projection point set to form candidate matching groups; For each candidate matching group, calculate the spatial overlap between the visual detection box region and the region where the projection point is located; If multiple radar projection points fall into the same detection frame, the distribution density of the projection points is used as a supplementary weight to obtain the corrected spatial overlap. The corrected spatial overlap is weighted proportionally with the radar confidence level to output the matching confidence level; If the matching confidence is lower than the preset threshold, it is marked as no match or low confidence match; otherwise, the radar target projection point of the visual detection box is determined to be a valid matching pair. Collect all valid matching pairs from all frames and construct a matching result set.

7. The method for identifying external damage hazards based on dual verification of AI image and radar according to claim 6, characterized in that, The confidence level of the matching result set is fused using a Bayesian probability model, and combined with radar detection results to determine the level of external damage hazard, as detailed below: The detection features of the corresponding targets are extracted from the radar detection results, and the detection features are correlated with the confidence and location in the visual recognition results to form a fusion input dataset; Using the fused input dataset as input, a joint confidence fusion model is constructed through a Bayesian probability model; In the joint confidence fusion model, visual recognition results are regarded as prior observations, radar detection results are regarded as independent observation evidence, and the probability of target existence is dynamically corrected through the Bayesian probability update principle. Based on the corrected target existence probability, the fusion confidence value of each externally damaged target is obtained; The system uses both confidence level and radar distance information as input features, and determines the risk level of external targets based on preset confidence and distance thresholds. The results of the hazard levels of all targets are summarized to form a set of external damage hazard level judgment results.

8. A system for identifying external damage hazards using the AI-based image and radar dual-verification method as described in any one of claims 1-7, characterized in that, It includes a data acquisition module, a visual recognition module, a radar detection module, a matching module, and a judgment module, and the modules are interconnected. The data acquisition module is used to extract the motion trajectory features of the sensing data in the target area, obtain the coordinate transformation matrix through the iterative nearest point algorithm, and perform time-series alignment to obtain calibrated sensing data, which includes image data and radar data. The visual recognition module is used to take calibrated image data as input, perform transfer learning on a pre-trained general visual model to obtain an external damage recognition visual model, and perform inference on the input image to output the visual recognition result. The radar detection module is used to perform constant false alarm rate detection, clustering, and trajectory tracking processing on calibrated radar data as input, and output radar detection results. The matching module is used to project the radar coordinates to the image coordinate system based on the coordinate transformation matrix, which combines the visual recognition results and radar detection results, outputs the spatial overlap, and generates a matching result set. The judgment module is used to perform confidence fusion on the matching result set through a Bayesian probability model and combine it with radar detection results to determine the level of external damage risks.

Images