Ship track joint calibration method based on radar and visual technology

Abstract

The invention relates to the technical field of radar and vision combined calibration, in particular to a ship track combined calibration method based on a radar and vision technology. The method comprises the following steps: acquiring radar point cloud data and image data; performing three-dimensional synchronization on the point cloud image data to obtain a plurality of groups of point cloud image data pairs at each moment; performing cross-correlation analysis on the radar point cloud data and the image data of the dynamic target, and combining a plurality of modal function values of a sequence formed by fusing the two kinds of data to determine the track fusion confidence of the dynamic target at each acquisition moment; determining a sea condition stability index of the dynamic target at each acquisition moment according to the flight path of the dynamic target fused with the mean value after confidence coefficient smoothing and the local periodic volatility index; and selecting a calibration method based on the sea condition stability index to complete the calibration of the dynamic target. According to the method, the delay problem of a traditional fusion model is avoided, degradation data pollution of the main sensor is avoided, and continuity and anti-interference capability of track calibration under complex sea conditions are ensured.

Description

Technical Field

[0001] The present application relates to the technical field of radar and vision joint calibration, and in particular to a ship track joint calibration method based on radar and vision technology. Background Art

[0002] With the surge in global maritime traffic density and the escalation of navigation safety requirements, radar has all-weather detection capabilities in complex weather conditions, but its resolution is low and is easily affected by multipath effects. Visual technology can capture rich texture and semantic information, but it is seriously affected by environmental factors such as lighting and haze. With the breakthrough of deep learning and multi-source data fusion technology, joint calibration has become an inevitable choice to improve perception accuracy. In the joint calibration of ship tracks using radar and visual technology, severe weather and complex sea conditions not only lead to the decline of the performance of a single sensor, but also destroy the spatiotemporal consistency and complementarity of radar and visual data, making it difficult for the joint calibration system to achieve reliable fusion. For example, in a strong wave environment, radar generates a large number of false alarm point clouds due to sea clutter, while vision loses target texture features due to rain and fog interference. The two data may generate erroneous tracks due to differences in noise distribution when they are aligned in time and space. Existing technologies mainly address the above problems through cross-modal consistency verification. However, existing adaptive algorithms are mostly offline training models, which have delayed responses when facing sudden environmental changes (such as instantaneous strong light or surges) and cannot meet the timing synchronization requirements of real-time joint calibration. It is urgent to develop environmental perception online learning and multi-source redundant fusion mechanisms. Summary of the invention

[0003] In order to solve the technical problem that timing synchronization is difficult to complete due to sudden environmental changes, this application provides a ship track joint calibration method based on radar and vision technology. The technical solution adopted is as follows: This application proposes a ship track joint calibration method based on radar and vision technology, which includes the following steps: The radar point cloud data and image data at each moment are obtained through radar and visual technology respectively; based on the high-precision GPS module and atomic clock, the radar point cloud data and image data are synchronized in three dimensions to obtain multiple sets of point cloud image data pairs at each moment, and the target tracking algorithm is used to make the point cloud image data pairs of dynamic targets at different moments correspond one to one; For the dynamic target, the cross-correlation analysis of the point cloud data and image data at the preset time before each moment is performed to obtain the cross-correlation function peak value and time delay; the fusion sequence of the dynamic target is formed based on the different point cloud image data of the dynamic target at the preset time before each moment; the fusion sequence is decomposed into several modal subsequences using the empirical mode decomposition method, and the track fusion confidence of the dynamic target at each acquisition moment is calculated based on the modal function value, cross-correlation function peak value and time delay of the modal subsequence; The mean value of the track fusion confidence of the dynamic target smoothed at a preset moment before each moment is combined with the volatility index in the preset sliding window to determine the sea condition stability index of the dynamic target at each acquisition moment; Based on the comparison between the sea condition stability index and the preset error, a calibration method is selected to complete the calibration of the dynamic target.

[0004] In the above solution, the present application first uses cross-correlation analysis to quantify the spatio-temporal matching degree of two types of sensor data, combines empirical mode decomposition to distinguish transient interference from stable features, and through spatio-temporal alignment and interference separation, improves the consistency of radar and visual data under complex sea conditions and reduces the target misdetection rate; then, aiming at the interference problem of the mirror result caused by the noise of surge mutation and instantaneous strong light, exponential weighted moving average filtering is used, combined with the standard deviation analysis of the sliding window to comprehensively consider long-term stability and local mismatch risk, eliminate the influence of transient interference such as rain, fog, and surge on the confidence, and identify the risk of track mismatch caused by sensor desynchronization or environmental mutation; then, the sea condition stability index is used to dynamically quantify the system reliability, quickly identify extreme environmental interference (such as strong light, surge), avoid the delay problem of the traditional fusion model, directly switch to the inertial navigation system, avoid the degradation data pollution of the main sensor, and ensure the continuity and anti-interference ability of track calibration under complex sea conditions.

[0005] In one embodiment, the step of obtaining a multi-group of point cloud image data pairs at each moment by three-dimensionally synchronizing radar point cloud data and image data based on a high-precision GPS module and an atomic clock is as follows: The centroid points and three-dimensional coordinates of the radar point cloud data are obtained through a clustering algorithm; The image data is used as the input of the neural network, the minimum circumscribed rectangle of each dynamic target is obtained, and the three-dimensional coordinates of its center point are obtained; Based on the high-precision GPS module and the atomic clock, the three-dimensional coordinates of the radar point cloud data and the image data are mapped to obtain the synchronized three-dimensional coordinate data of the radar and the vision, complete the one-to-one matching of the radar point cloud data and the image data, and record the matched pair of radar point cloud data and image data as a point cloud image data pair.

[0006] In one embodiment, the step of obtaining the peak value and time delay of the cross-correlation function through the cross-correlation analysis of the point cloud data and the image data at the preset moment before each moment for the dynamic target is as follows: For each moment, a preset number of moments before it are used as the target time period. The radar point cloud data in the target time period and the image data in the target time period of each group of point cloud image data points are subjected to cross-correlation analysis to obtain the peak value and time delay of the cross-correlation function. The cross-correlation analysis presets a time window and a maximum lag compensation.

[0007] In one embodiment, the step of forming a fusion sequence of the dynamic target from different data at a preset moment before each moment based on the dynamic target is as follows: Take the mean of the corresponding data points of the radar and vision in each quadrant of the coordinate system in the point cloud image data pair at each moment as the fusion data points; for each moment, sort the different fusion data points of the dynamic target in the target time period in chronological order, where the three-dimensional data of each fusion data point are respectively used as the three values of the sequence to form the fusion sequence of the dynamic target.

[0008] In one embodiment, the modal function value is the sum of the absolute values of all sequence values in each modal subsequence.

[0009] In one embodiment, the track fusion confidence is positively correlated with the negative exponential mapping of the modal function and the peak value of the cross-correlation function, and negatively correlated with the time delay.

[0010] In one embodiment, the sea state stability index is positively correlated with the mean value of the track fusion confidence and negatively correlated with the mean value of the volatility indexes of all sliding windows in the target time period.

[0011] In one embodiment, the step of obtaining the mean value of the track fusion confidence is as follows: Take all the track fusion confidences of the dynamic target in the target time period before each moment as the input, use the exponentially weighted moving average filter, output the smoothed track fusion confidence, and calculate its mean value.

[0012] In one embodiment, the step of obtaining the volatility index is as follows: Calculate the standard deviation of all the track fusion confidences in the sliding window as the volatility index of the sliding window; Each sliding window contains a preset number of time windows.

[0013] In one embodiment, the step of selecting a calibration method based on the comparison between the sea state stability index and a preset error is as follows: For the dynamic target, normalize the sea state stability indexes calculated at all acquisition moments. When the normalized sea state stability index continuously falls below the error threshold within a preset time, immediately stop using the data of the radar and vision, and switch to the inertial navigation system for route calibration; if there is no continuous fall below the error threshold within the preset time, continue with the joint calibration by the radar and vision.

[0014] The beneficial effects of this application are: This application first uses cross-correlation analysis to quantify the spatio-temporal matching degree of two types of sensor data, combines empirical mode decomposition to distinguish transient interference from stable features, and through spatio-temporal alignment and interference separation, improves the consistency of radar and visual data under complex sea conditions and reduces the target misdetection rate. Then, for the problem of mirror surface result interference caused by noise such as surge mutations and instantaneous strong light, exponential weighted moving average filtering is adopted, combined with sliding window standard deviation analysis to comprehensively consider long-term stability and local mismatch risk, eliminate the influence of transient interferences such as rain, fog, and surge on confidence, and identify the risk of track mismatch caused by sensor desynchronization or environmental mutations. Then, the system reliability is dynamically quantified through the sea condition stability index, extreme environmental interferences (such as strong light and surge) are quickly identified, the delay problem of traditional fusion models is avoided, and the system is directly switched to the inertial navigation system to avoid the pollution of degraded data of the main sensor and ensure the continuity and anti-interference ability of track calibration under complex sea conditions. BRIEF DESCRIPTION OF THE DRAWINGS

[0015] To more clearly illustrate the technical solutions and advantages in the embodiments of the present application or the prior art, the following will briefly introduce the drawings required for the description of the embodiments or the prior art. Obviously, the drawings in the following description are only some embodiments of the present application. For those of ordinary skill in the art, without creative efforts, other drawings can be obtained based on these drawings.

[0016] Figure 1 It is a flowchart of a method for jointly calibrating ship tracks based on radar and vision technologies provided by an embodiment of the present application. DETAILED DESCRIPTION OF THE EMBODIMENTS

[0017] To further elaborate on the technical means and effects adopted by the present application to achieve the intended invention purpose, the following, in combination with the drawings and preferred embodiments, details the specific implementation manners, structures, features, and effects of the method for jointly calibrating ship tracks based on radar and vision technologies proposed by the present application. In the following description, different "one embodiment" or "another embodiment" do not necessarily refer to the same embodiment. In addition, the specific features, structures, or characteristics in one or more embodiments can be combined in any suitable form.

[0018] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by those of ordinary skill in the technical field to which this application belongs.

[0019] Embodiment of the method for jointly calibrating ship tracks based on radar and vision technologies: The following specifically describes the specific solution of the method for jointly calibrating ship tracks based on radar and vision technologies provided by the present application in combination with the drawings.

[0020] Please refer to Figure 1, which shows a flowchart of a ship track joint calibration method based on radar and vision technologies provided by an embodiment of the present application. The method includes the following steps: Step S001, obtain radar point cloud data and image data; perform three-dimensional synchronization on them to obtain multiple groups of point cloud image data pairs at each moment.

[0021] Install millimeter-wave radars and high-resolution multispectral cameras on the top of the ship's mast and the top of the sidewalls on both sides of the hull to synchronously collect radar point cloud data and RGB-D depth image data. Among them, the millimeter-wave radar obtains the distance of moving targets around the ship from the ship, the speed and azimuth information of the dynamic targets by transmitting frequency modulation continuously. In addition, for dynamic targets, the millimeter-wave radar obtains their reflection intensity, and each dynamic target is used as a point cloud data to generate a dense point cloud to characterize the dynamic environment around the ship; while the camera relies on wide dynamic imaging technology to capture high-resolution visual data on both sides of the ship, enhancing the target's three-dimensional space perception ability. In this embodiment, the acquisition frequencies of both the millimeter-wave radar and the high-resolution multispectral camera are 10Hz.

[0022] For each moment, take each point cloud data and its corresponding reflection intensity as the input of the clustering algorithm, and output point cloud clusters. In this embodiment, the DBSCAN clustering algorithm is used, the neighborhood radius is 0.5, and the minimum number of point clouds in the point cloud cluster is 10. Determine its centroid point and its three-dimensional coordinates based on the point cloud cluster.

[0023] After that, take the RGB-D image as the input, and use a neural network model to output the three-dimensional coordinates of the center point of the dynamic target bounding box in the hull coordinate system; in this embodiment, the YOLO model is used, and the SGD optimizer is adopted in the training of the YOLOv5 model, and the momentum , the initial learning rate is 0.01; the cross-entropy loss function is adopted. The dynamic target bounding box is the minimum bounding rectangle of the dynamic target. The hull coordinate system is a coordinate system with the ship itself as the origin, which describes the position of the ship.

[0024] To achieve spatio-temporal consistency, integrate a high-precision GPS module and an atomic clock, and eliminate the time delay and pose deviation between sensors through timestamp alignment and geographic coordinate mapping. Eliminating time delay and pose deviation through a high-precision GPS module and an atomic clock is a well-known technology to those skilled in the art, and will not be elaborated here; finally, output the three-dimensional coordinate data of the radar and vision synchronized in time and space.

[0025] It should be noted that the three-dimensional coordinate data output by radar and vision have the same meaning after spatio-temporal alignment and geographical coordinate mapping, both representing the absolute spatial positions of dynamic objects (such as other ships or obstacles) around the ship in a unified geographical coordinate system (such as longitude, latitude, and altitude). Because the system converts the sensor data of the two to the same global coordinate system through high-precision GPS and atomic clocks, and the point cloud and the center point of the bounding box are matched one by one, thus realizing the consistent representation of the target position in the real world. Each point cloud and its corresponding bounding box center point are used as a point cloud image data pair, and multiple groups of point cloud image data pairs at different times are made to correspond one by one based on the target tracking algorithm. This lays a reliable data foundation for the method of jointly calibrating ship tracks based on radar and vision.

[0026] So far, multiple groups of point cloud image data pairs at each moment have been obtained.

[0027] Step S002: Perform cross-correlation analysis on the radar point cloud data and image data of the dynamic object, and determine the track fusion confidence of the dynamic object at each acquisition moment by combining the multiple modal function values of the sequence formed after fusing the two types of data.

[0028] Due to the influence of multipath effects, rain and fog interference, and sea clutter on radar and vision sensors under bad weather and complex sea conditions, the spatio-temporal registration deviation between radar point clouds and vision images increases significantly, the false alarm rate in track calibration increases, and the target tracking accuracy decreases. Especially in scenarios of sudden surges or instantaneous strong light, traditional offline fusion models are difficult to ensure data consistency.

[0029] For each moment, there are multiple groups of point cloud image data pairs, and for one of the point cloud image data pairs, there are also different positions at different times. The moments with a preset length before each moment are used as the target time period; for the dynamic object, all the point cloud image data pairs corresponding to the dynamic object in the target time period are subjected to cross-correlation analysis to obtain the peak value of the cross-correlation function and its corresponding time delay. The peak value size characterizes the matching degree of the two types of sensor data in the spatio-temporal dimension, and the time delay reflects the system clock synchronization error. In this embodiment, the length of the target time period is 5 minutes, and the time window length and the maximum lag compensation when setting the cross-correlation analysis are 0.5 seconds and 10 step lengths respectively.

[0030] After that, for each moment, the mean value of the corresponding data points of radar and vision in each quadrant of the coordinate system in the point cloud image data pair is used as the fused data point. The dimension value on the x-axis of the fused data point is the mean value of the dimension values on the x-axis of the corresponding data points of radar and vision, and the same applies to the y-axis and z-axis.

[0031] For each moment, the fused data points in the target time period at that moment are sorted in time sequence, where the three-dimensional data of each fused data point are respectively used as the three values of the sequence, thus forming a fused sequence for each dynamic target; for example, the dimension value on the x-axis of the fused data point at the first moment is the first value of the fused sequence, the dimension value on the y-axis is the second value of the fused sequence, and the dimension value on the z-axis is the third value of the fused sequence.

[0032] Use the empirical mode decomposition method to divide the fused sequence into several modal subsequences, and take the sum of the absolute values of all sequence values in each modal subsequence as the modal function value. In this embodiment, the number of modal subsequences is 6.

[0033] After decomposing the fused sequence by the empirical mode decomposition method, the larger the modal function value, the higher its frequency, and the smaller the modal function value, the lower its frequency. After performing a negative exponential mapping on the modal function value, it reflects the stability degree of the track signal.

[0034] Calculate the track fusion confidence at each acquisition moment based on the negative exponential mapping of the modal function, the peak value of the cross-correlation function, and the corresponding time delay.

[0035] The track fusion confidence is positively correlated with the negative exponential mapping of the modal function and the peak value of the cross-correlation function, and negatively correlated with the time delay.

[0036] It should be noted that positive correlation means that when one variable increases, the other variable also increases, and the change directions of the two variables are the same. When one variable changes from large to small or from small to large, the other variable also changes from large to small or from small to large; the specific relationship is determined by the actual application, and this application does not make special restrictions.

[0037] It should be noted that negative correlation means that when one variable increases, the other variable decreases, and the change directions of the two variables are opposite. When one variable changes from large to small or from small to large, the other variable also changes from small to large or from large to small; the specific relationship is determined by the actual application, and this application does not make special restrictions.

[0038] Preferably, in this embodiment, the expression of the track fusion confidence is: , represents the modal function value of the i-th modal subsequence, represents the exponential function with the natural constant as the base, represents the peak value of the cross-correlation function, represents the time delay, represents the track fusion confidence at each moment.

[0039] Among them, the peak value of the cross-correlation function reflects the matching degree of the radar and visual sensor data in the time series. Through cross-correlation analysis, the peak value indicates the degree of alignment of the observation data of the ship track of the two types of sensors in the time dimension under a specific time delay. The larger the peak value of the cross-correlation function, the higher the consistency of the spatiotemporal characteristics of the ship track described by the radar point cloud and the visual image. The delay quantifies the time synchronization error of the two types of sensors. The smaller the delay, the higher the system clock synchronization accuracy and the better the spatial consistency of the target position in the track calibration. The negative mapping of the modal function value reflects the intensity of the stable track feature in the signal. The larger the value, the more stable the track signal.

[0040] The track fusion confidence is used to quantify the fusion confidence of radar and visual data in track calibration. The larger the value, the better the spatiotemporal alignment of radar and visual data and the more stable the track characteristics. The higher the accuracy of the fusion track output by the joint calibration method.

[0041] At this point, the fused track confidence of the dynamic target at each moment is obtained.

[0042] Step S003, the sea condition stability index of the dynamic target at each acquisition moment is determined by combining the smoothed mean value of the dynamic target's track fusion confidence and the local periodic volatility index.

[0043] Due to the aggravated temporal and spatial registration deviation of radar point cloud and visual image under severe weather and complex sea conditions, the time series data of track fusion confidence A has high-frequency noise interference and short-term fluctuations, which leads to the joint calibration system having problems such as increased false alarm rate and inaccurate target tracking in surge mutation or instantaneous strong light scenes. This application constructs a stability index based on the fluctuation of track fusion confidence.

[0044] For each moment, all track fusion confidences of dynamic targets in the previous target time period are used as input and filtered using exponential weighted moving average (EWMA) to avoid transient noise interference, and the average of the smoothed track fusion confidences is output. , which represents the long-term stable trend of track fusion confidence, and enhances the system's ability to perceive the stable motion characteristics of the ship by suppressing the influence of transient interference such as rain, fog, and surge on track fusion confidence. In this embodiment, the smoothing factor is 0.2.

[0045] Then, a sliding window is set, and the standard deviation of all track fusion confidences in the sliding window is calculated as the volatility index of the sliding window to quantify the local fluctuation intensity of the track fusion confidence, thereby reflecting the risk of track feature mismatch between radar and visual data in the spatiotemporal dimension due to environmental interference (such as sea clutter, rain and fog scattering) or sensor desynchronization. In this embodiment, the size of the sliding window is 2s, which covers 4 time windows.

[0046] The sea condition stability index is obtained based on the mean of track fusion confidence and all volatility indicators in the target time period.

[0047] The sea state stability index is positively correlated with the mean of track fusion confidence, and negatively correlated with the mean of all volatility indicators in the target time period.

[0048] Preferably, in this embodiment, the expression of the sea state stability index is: , is the mean of the smoothed track fusion confidence, represents the mean of the volatility indicators of all sliding windows, Indicates the adjustment parameter, which is used to prevent the denominator from being 0. Represents the sea state stability index at each moment.

[0049] The mean of the smoothed track fusion confidence retains the long-term trend characteristics of the track fusion confidence (such as the stable navigation mode of the ship), and at the same time filters out the high-frequency jitter caused by environmental transient interference or sensor clock micro-jumps. The larger the value, the stronger the spatiotemporal alignment consistency of the radar and visual data is in the long-term perspective, and the higher the credibility of the track calibration; the mean of the sliding window volatility index is the discrete degree of the track fusion confidence, which can identify the risk of track feature mismatch caused by sea surface clutter, sensor desynchronization or instantaneous strong light. The larger the value, the more violent the matching of the radar and visual data in the spatiotemporal dimension fluctuates, and the system is more significantly affected by multipath effects, rain and fog scattering or surge mutations, reflecting the decreased stability of the track calibration due to environmental interference or sensor abnormalities.

[0050] The sea condition stability index dynamically characterizes the reliability of the radar and vision joint calibration system by fusing smoothing confidence and volatility. A higher value indicates that the track characteristics are well aligned in time and space, the environmental interference is controllable, and the calibration result is highly credible. A lower value indicates that it is necessary to trigger sensor weight adjustment or redundant data fusion mechanism to cope with the perception degradation problem under complex sea conditions.

[0051] At this point, the sea state stability index is obtained.

[0052] Step S004, selecting a calibration method based on the sea condition stability index to complete the calibration of the dynamic target.

[0053] Due to the multipath effect, rain and fog interference and sea clutter in severe weather and complex sea conditions, radar and visual sensors are affected by the time and space registration deviation between radar point cloud and visual image significantly increases, the false alarm rate in track calibration increases and the target tracking accuracy decreases. Especially under sudden environmental interference such as surge mutation or instantaneous strong light, the traditional offline fusion model lacks real-time response capability and is difficult to ensure the time and space consistency of data, resulting in response delay and data mismatch problems in the joint calibration system, which seriously restricts the perception reliability of the ship's track.

[0054] Therefore, firstly, the radar and visual data points are matched one by one for calibration through step S001. When the sea condition stability index does not meet the conditions, the route calibration is performed through the inertial navigation system (INS).

[0055] In order to filter out occasional noise, for dynamic targets, the sea condition stability index calculated at all acquisition moments is normalized. When the normalized sea condition stability index is continuously lower than the error threshold within the preset time, the data of the main sensor (radar or vision) is immediately stopped, and the inertial navigation system is switched to perform route calibration. The inertial navigation system calculates the position through the gyroscope and accelerometer, and provides positioning when the satellite signal is lost, thereby ensuring the continuity and robustness of the track calibration under complex sea conditions, thereby completing the calibration of dynamic targets. The inertial navigation system is a technology well known to those skilled in the art, and will not be elaborated here. In this embodiment, the error threshold is 0.4 and the preset time is 5s.

[0056] Through dynamic monitoring of the sea stability index and hard switching rather than complex fusion, the impact of the main sensor degradation data on the system is avoided, ensuring the continuity and robustness of track calibration in extreme environments. The spatiotemporal registration accuracy and anti-interference ability of the ship track joint calibration system are significantly improved, effectively solving the data mismatch and response delay problems caused by environmental mutations in existing technologies.

[0057] It should be noted that the above-described embodiments are only used to illustrate the technical solutions of the present application, rather than to limit them. Although the present application has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or make equivalent replacements for some of the technical features therein. These modifications or replacements do not deviate the essence of the corresponding technical solutions from the scope of the technical solutions of the embodiments of the present application, and should all be included in the protection scope of the present application.

[0058] The various embodiments in this specification are described in a progressive manner, and the same or similar parts between the various embodiments can be referenced to each other, and each embodiment focuses on the differences from other embodiments.

Claims

1. A method for joint calibration of ship tracks based on radar and vision technologies, characterized in that, The method includes the following steps: Obtain radar point cloud data and image data at each moment through radar and vision technologies respectively; based on a high-precision GPS module and an atomic clock, perform three-dimensional synchronization on the radar point cloud data and the image data to obtain multiple groups of point cloud image data pairs at each moment, and use a target tracking algorithm to make the point cloud image data pairs of the dynamic target at different moments correspond one by one; For the cross-correlation analysis of the point cloud data and the image data at preset moments before each moment of the dynamic target, obtain the peak value of the cross-correlation function and the time delay; based on the different point cloud image data of the dynamic target at preset moments before each moment, form a fusion sequence of the dynamic target; use the empirical mode decomposition method to decompose the fusion sequence into several modal subsequences, and calculate the track fusion confidence of the dynamic target at each acquisition moment based on the modal function values, the peak value of the cross-correlation function, and the time delay of the modal subsequences; Determine the sea condition stability index of the dynamic target at each acquisition moment based on the mean value of the smoothed track fusion confidence of the dynamic target at preset moments before each moment and the volatility index in the preset sliding window; Select a calibration method based on the comparison between the sea condition stability index and the preset error to complete the calibration of the dynamic target.

2. The method for jointly calibrating ship tracks based on radar and vision technologies according to claim 1, characterized in that, The step of performing three-dimensional synchronization on the radar point cloud data and the image data based on the high-precision GPS module and the atomic clock to obtain multiple groups of point cloud image data pairs at each moment is as follows: Obtain the centroid point and three-dimensional coordinates of the radar point cloud data through a clustering algorithm; Take the image data as the input of the neural network, obtain the minimum circumscribed rectangle for each dynamic target, and obtain the three-dimensional coordinates of its center point; Based on the high-precision GPS module and the atomic clock, map the three-dimensional coordinates of the radar point cloud data and the image data to obtain the synchronized three-dimensional coordinate data of the radar and the vision, complete the one-to-one matching of the radar point cloud data and the image data, and record the matched pair of radar point cloud data and image data as a point cloud image data pair.

3. The method for jointly calibrating ship tracks based on radar and vision technologies according to claim 1, wherein The step of performing cross-correlation analysis on the point cloud data and the image data at preset moments before each moment of the dynamic target to obtain the peak value of the cross-correlation function and the time delay is as follows: For each moment, use a preset number of moments before it as the target time period, perform cross-correlation analysis on the radar point cloud data in the target time period and the image data in the target time period for each group of point cloud image data points to obtain the peak value of the cross-correlation function and the time delay, and preset the time window and the maximum lag compensation for the cross-correlation analysis.

4. The method for jointly calibrating ship tracks based on radar and vision technologies according to claim 3, wherein, The step of forming a fusion sequence of the dynamic target based on different data of the dynamic target at preset moments before each moment is as follows: Take the mean value of the corresponding data points of the radar and the vision in each quadrant of the coordinate system in the point cloud image data pair at each moment as the fusion data point; for each moment, sort the different fusion data points of the dynamic target in the target time period in chronological order, and use the three-dimensional data of each fusion data point as the three values of the sequence respectively to form a fusion sequence of the dynamic target.

5. The method for jointly calibrating the ship's track based on radar and vision technologies according to claim 1, wherein The modal function value is the cumulative sum of the absolute values of all sequence values in each modal subsequence.

6. The method for jointly calibrating the ship's track based on radar and vision technologies according to claim 1, characterized in that, The track fusion confidence is positively correlated with the negative exponential mapping of the modal function and the peak value of the cross-correlation function, and negatively correlated with the time delay.

7. The method for jointly calibrating the ship's track based on radar and vision technologies according to claim 3, wherein The sea condition stability index is positively correlated with the mean value of the track fusion confidence, and negatively correlated with the mean value of the volatility index of all sliding windows in the target time period.

8. The method for jointly calibrating the ship's track based on radar and vision technologies according to claim 7, wherein, The steps for obtaining the mean value of the track fusion confidence are as follows: Taking all the track fusion confidences of the dynamic target within the target time period before each moment as input, using exponential weighted moving average filtering, outputting the smoothed track fusion confidence, and calculating its mean value.

9. The method for jointly calibrating ship tracks based on radar and vision technologies according to claim 7, wherein The steps for obtaining the volatility index are as follows: Calculating the standard deviation of all the track fusion confidences in the sliding window as the volatility index of the sliding window; where each sliding window contains a preset number of time windows.

10. The method for jointly calibrating the ship's track based on radar and vision technologies according to claim 1, wherein, The steps for selecting the calibration method based on the comparison between the sea condition stability index and the preset error are as follows: For the dynamic target, normalizing the sea condition stability index calculated at all acquisition moments. When the normalized sea condition stability index continuously falls below the error threshold within the preset time, immediately stop using the data of the radar and vision, and switch to the inertial navigation system for route calibration; if there is no continuous fall below the error threshold within the preset time, continue with the joint calibration through the radar and vision.

Images