Machine learning based radar-optical target alignment method

Abstract

The application discloses a radar photoelectric target alignment method based on machine learning, comprising: acquiring the time sequence space coordinate sequence of the target by the radar and the gimbal pose state of the camera, estimating the motion state parameters of the target, combining the gimbal hardware pose estimation response delay duration, and then calculating the compensation space coordinates of the target at the predicted time; performing space geometry trigonometric function coding and coordinate system interactive coupling on the compensation coordinates to obtain an extended feature vector; inputting the vector into a space mapping model to output initial gimbal control parameters and a pointing confidence parameter representing internal prediction divergence; finally determining a field of view angle adjustment factor and adaptively scaling a focal length parameter according to the pointing confidence parameter to generate a final alignment instruction. The application effectively overcomes the mechanical response delay and the space nonlinear mapping error, and improves the first capture rate and the alignment robustness of the photoelectric system in a complex environment.

Description

Technical Field

[0001] This invention relates to radar and optoelectronic collaborative detection technology, and in particular to a radar optoelectronic target alignment method based on machine learning. Background Technology

[0002] In modern maritime and air surveillance and defense systems, radar possesses all-weather, long-range, and wide-area search capabilities, while electro-optical cameras can provide high-resolution visual detail information. Integrating the detection advantages of both, enabling radar-guided electro-optical equipment to quickly lock onto a predetermined area, is of significant technical importance for improving the situational awareness resolution of electro-optical reconnaissance systems, enhancing target confirmation rates in complex electromagnetic environments, and shortening the overall reaction cycle of sensor networks.

[0003] Current mainstream alignment schemes primarily rely on pre-calibrated fixed geometric coordinate transformation matrices, directly mapping the polar coordinate data detected by radar to the camera's gimbal control parameters using a rigid body transformation formula. However, in actual operation, due to factors such as machining tolerances, long-term operational deformation, and nonlinear changes in atmospheric refractive index, a single rigid mapping is insufficient to adapt to the non-uniform error accumulation across different spatial orientations. Furthermore, when processing information on highly maneuverable targets, existing systems typically use the instantaneous coordinates of the received radar data as the endpoint command for the photoelectric servo actuator, resulting in a shift in the target's actual position by the time the gimbal's mechanical movements are completed.

[0004] Existing methods struggle to effectively overcome the coupling effects of spatial heterogeneity errors and hardware execution delays in complex and dynamic monitoring environments, leading to a significant increase in the randomness of alignment deviations. This is particularly true in long-distance or highly maneuverable scenarios, where initial capture success rate and continuous tracking reliability face severe challenges. Therefore, it is necessary to investigate a method that can improve the robustness and accuracy of multi-sensor collaborative alignment in complex and variable scenarios. Summary of the Invention

[0005] Therefore, a radar electro-optical target alignment method based on machine learning is provided to address the aforementioned problems in existing technologies.

[0006] A machine learning-based radar electro-optical target alignment method includes:

[0007] Acquire the temporal and spatial coordinate sequence generated by the radar's continuous detection of the target, as well as the gimbal pose state of the electro-optical camera;

[0008] The motion state parameters of the target are estimated based on the temporal spatial coordinate sequence, and the response delay is predicted by combining the gimbal pose state. The compensated spatial coordinates of the target at the predicted time are calculated based on the motion state parameters and the response delay.

[0009] Spatial geometric trigonometric function encoding and coordinate system interaction coupling operation are performed on the compensated spatial coordinates to obtain the extended feature vector;

[0010] The extended feature vector is input into the pre-built spatial mapping model, and the output is the initial gimbal control parameters and the pointing confidence parameters extracted based on the internal prediction divergence of the model.

[0011] The field of view adjustment factor is determined using the pointing confidence parameter, and the focal length parameter in the initial gimbal control parameters is adaptively scaled accordingly to output the final gimbal target alignment command.

[0012] Based on the above technical solution, the present invention effectively overcomes mechanical response delay and spatial nonlinear mapping error, and improves the first capture rate and alignment robustness of photoelectric systems in complex environments. Attached Figure Description

[0013] The accompanying drawings, which are included to provide a further understanding of this application and form part of this application, illustrate exemplary embodiments and are used to explain this application, but do not constitute an undue limitation of this application. In the drawings:

[0014] Figure 1 This is a schematic diagram of the overall process of a radar electro-optical target alignment method based on machine learning provided in an embodiment of this application.

[0015] Figure 2 This is a schematic diagram illustrating the hardware execution time required for the computational gimbal to rotate from its current gimbal pose state to the intermediate predicted pose, as provided in this application embodiment.

[0016] Figure 3 This is a schematic diagram of the process of obtaining extended feature vectors by performing spatial geometric trigonometric function encoding and coordinate system interactive coupling operation on the compensated spatial coordinates, as provided in the embodiments of this application.

[0017] Figure 4 This is a schematic diagram illustrating the process of using a weighted dynamic sample set to perform asynchronous retraining on a spatial mapping model, as provided in an embodiment of this application.

[0018] Figure 5 This is a schematic diagram of the error-driven adaptive partitioning logic provided in the embodiments of this application. Detailed Implementation

[0019] Example 1 provides a radar-electro-optical target alignment method based on machine learning, aiming to achieve rapid and accurate alignment of the electro-optical camera after the radar detects a target in complex radar and electro-optical cooperative detection scenarios. This is mainly achieved by using a data-driven machine learning model to overcome pointing deviations caused by installation errors, mechanical delays, and target movement. Figure 1 As shown, the specific steps may include the following:

[0020] Step 101: Obtain the temporal spatial coordinate sequence generated by the radar's continuous detection of the target and the gimbal pose state of the photoelectric camera.

[0021] In this embodiment, the radar is used to acquire the spatial location of the target, and can be an L-band or X-band sea / airspace surveillance radar. The temporal spatial coordinate sequence specifically includes the target azimuth and range acquired by the radar at different detection times, and for airspace targets, it also includes altitude information. Since the radar and the electro-optical camera are installed in different geographical locations or on different platforms, a unified coordinate reference needs to be established before data fusion.

[0022] Specifically, the radar azimuth angle is typically referenced to true north as 0 degrees and increases clockwise, while the horizontal rotation axis of the electro-optical camera, i.e., the Pan axis, is referenced to the installation orientation as 0 degrees. To eliminate systematic deviations caused by mechanical installation, a fixed offset angle needs to be calculated after data acquisition.

[0023] The fixed offset angle is calibrated by selecting K fixed reference objects with known azimuths, and the radar azimuth angle of each reference object and the horizontal rotation angle of the camera during alignment are recorded:

[0024] Δ φ =median(θ ref_k -P ref_k );

[0025] Where, Δ φ The calculated fixed offset angle is θ, where median is the median operation function. ref_k Let P be the radar azimuth angle of the k-th fixed reference point. ref_k This represents the horizontal rotation angle of the camera corresponding to the k-th fixed reference point, where k is the index of the fixed reference point, ranging from 1 to K. Using the median method can suppress the impact of anomalous jumps in individual observation points on calibration accuracy. In subsequent alignment processes, this fixed offset angle is used to perform translation correction on the original data, ensuring the consistency between radar detection coordinates and camera pointing coordinates in geographic space.

[0026] Furthermore, when the method is applied to moving platforms such as shipborne or airborne platforms, step 101 also involves platform attitude compensation. Specifically, the real-time attitude angles of the platform output by the inertial navigation system, including roll angle, pitch angle, and yaw angle, are received in real time. The target coordinates detected by the radar and the gimbal pose state of the camera are uniformly transformed to a stable geodetic coordinate system using a coordinate rotation matrix to avoid alignment failure caused by platform shaking.

[0027] Step 102: Estimate the motion state parameters of the target based on the temporal spatial coordinate sequence, and estimate the response delay duration in combination with the gimbal pose state. Calculate the compensated spatial coordinates of the target at the predicted time based on the motion state parameters and the response delay duration.

[0028] In this step, considering that it takes a certain amount of physical time for the photoelectric gimbal to rotate from its current position to the target's predicted pointing position, a motion compensation mechanism is introduced to prevent the target from shifting after the gimbal has reached its position.

[0029] Specifically, least-squares linear regression analysis is performed using historical position data within the most recent detection time window to estimate the target's azimuth angular velocity and radial velocity, which are then used as motion state parameters. Correspondingly, the response delay is obtained by adding the gimbal rotation time to the mechanical stabilization time.

[0030] Δ t =max(|P init -P cur |,|T init -T cur |) / Ω max +t settle ;

[0031] Where, Δ t To estimate the response delay, max is the maximum value operation function, P init For the preliminary predicted horizontal rotation angle, P cur T represents the current actual horizontal rotation angle. init For the preliminary predicted pitch angle, T cur Ω represents the current actual pitch angle. max t is the maximum angular velocity of the gimbal. settle This refers to the mechanical stabilization time after the gimbal has rotated to its final position.

[0032] After obtaining the response delay duration, the target's azimuth is calculated by extrapolating it based on the target's current angular velocity, thus obtaining the target's compensated spatial coordinates at the predicted time. This compensation mechanism solves the lag problem when aligning high-speed moving targets in a narrow field of view.

[0033] Step 103: Perform spatial geometric trigonometric function encoding and coordinate system interaction coupling operation on the compensated spatial coordinates to obtain the extended feature vector.

[0034] When constructing feature vectors, directly inputting angle values ​​can lead to mathematical discontinuities at the boundary between 0 degrees and 360 degrees. For example, 359 degrees and 1 degree are only 2 degrees apart in physical space, but 358 degrees apart in numerical space. To solve this problem, this embodiment performs trigonometric function encoding on the compensated azimuth angle, mapping it to sine and cosine components.

[0035] Furthermore, in order to capture the spatial coupling effect of azimuth and distance, the distance component is multiplied by the trigonometric function component of the azimuth angle to obtain the corresponding eastward coordinate component and northward coordinate component. Finally, all the above components are concatenated to form an extended feature vector.

[0036] For example, for a maritime target, the extended feature vector can be represented as (sinθ,cosθ,R*sinθ,R*cosθ,θ,R), where θ is the original value of the compensated azimuth component and R is the original value of the compensated range component.

[0037] Step 104: Input the extended feature vector into the pre-built spatial mapping model, and output the initial gimbal control parameters and the pointing confidence parameters extracted based on the internal prediction divergence of the model.

[0038] The spatial mapping model is implemented using a random forest regression model trained in separate regions. The expanded feature vector obtained in step 103 is used as input, and the model outputs the predicted horizontal rotation angle, pitch angle, and focal length multiplier, which serve as the initial gimbal control parameters. Simultaneously, leveraging the ensemble learning characteristics within the random forest, the statistical dispersion of the output results of each decision tree is extracted.

[0039] Specifically, statistical dispersion can be represented by the standard deviation of the predictions from each decision tree. This standard deviation is the indicative confidence parameter, reflecting the reliability of the model's predictions for the current input sample. When the target is located in an area with poor coverage of historical training samples or when there is significant environmental interference, the disagreement among the decision trees will increase, leading to a larger indicative confidence parameter value, indicating a decrease in the reliability of the prediction.

[0040] Step 105: Determine the field of view adjustment factor using the pointing confidence parameter, adaptively scale the focal length parameter in the initial gimbal control parameters based on the field of view adjustment factor, and output the final gimbal target alignment command.

[0041] Specifically, this step can also be implemented as follows: using the pointing confidence parameter to determine the field of view adjustment factor, adaptively scaling the focal length parameter in the initial gimbal control parameters based on the field of view adjustment factor to obtain the final focal length parameter; combining the final focal length parameter with the rotation angle parameter in the initial gimbal control parameters to output the final gimbal target alignment command.

[0042] In this embodiment, in order to ensure that the target can still be captured even when the model prediction confidence is low, the camera's focal length is adjusted in real time using the pointing confidence parameter.

[0043] Specifically, the field-of-view adjustment factor is calculated using a preset exponential decay function. When the discrepancy reflected by the pointing confidence parameter is small, the field-of-view adjustment factor approaches 1, and the camera maintains the high focal length output predicted by the model to obtain target details; when the discrepancy increases, the field-of-view adjustment factor decreases, and the focal length magnification is automatically reduced to expand the camera's real-time field of view.

[0044] The aforementioned adaptive scaling mechanism establishes a correlation between statistical prediction uncertainty and physical field of view, thereby improving the success rate of first target acquisition in complex environments by expanding the field of view.

[0045] The scaled focal length parameters are combined with the predicted rotation parameters to output the final gimbal target alignment command and send it to the camera controller.

[0046] The coordinated operation of the above steps enables effective calibration of sensor installation errors, dynamic compensation for moving targets, continuous encoding of spatial features, and adaptive field-of-view control based on prediction confidence. This improves the response efficiency and alignment accuracy of the electro-optical reconnaissance system in variable maritime and air environments.

[0047] According to another aspect of this application, a machine learning-based radar electro-optical target alignment system is provided, comprising:

[0048] At least one processor; and a memory communicatively connected to the at least one processor; wherein the memory stores instructions executable by the processor for implementing a machine learning-based radar electro-optical target alignment method according to the present invention.

[0049] Example 2, as a further refinement of the motion compensation step in Example 1, describes how the system can offset the impact of gimbal mechanical response delay on alignment accuracy through historical trajectory regression analysis and gimbal physical characteristic modeling.

[0050] One possible implementation includes pre-interception verification logic before performing sequence regression analysis based on historical observation windows in the time-series spatial coordinate sequence.

[0051] The cumulative number of trajectory points detected within the statistical time-series spatial coordinate sequence;

[0052] If the cumulative number of track points detected does not reach the preset cumulative threshold, the current target coordinates will be used as the compensation spatial coordinates, and the subsequent velocity component extraction and time estimation process will be skipped.

[0053] Specifically, if the cumulative number of trajectory points detected does not reach the preset cumulative threshold, the target coordinates at the latest detection time in the temporal spatial coordinate sequence are directly used as the compensation spatial coordinates, and the subsequent velocity component extraction and time estimation process is skipped.

[0054] In this embodiment, a cold start logic determination is performed before motion compensation. The preset cumulative threshold can be set to 5 times. When the radar first detects a target or re-acquires a target after loss, if the number of valid trajectory points in the acquired temporal spatial coordinate sequence is less than 5, it is considered that the current sample size is insufficient to support high-precision motion trend fitting. At this time, it is determined that the random error of the regression analysis is too large. In order to avoid the gimbal pointing off from the actual field of view of the target due to incorrect compensation, the subsequent prediction logic will be intercepted, and the target azimuth and distance detected at the current moment will be directly used as the input for the subsequent feature encoding stage.

[0055] One possible implementation involves estimating the target's motion state parameters based on a temporal spatial coordinate sequence and combining this with the gimbal pose state to predict the response delay, including the following process:

[0056] Step 201: Perform sequence regression analysis based on the historical observation window in the time-series spatial coordinate sequence to extract the azimuth angular velocity component and radial velocity component of the target to form motion state parameters.

[0057] Once the cumulative number of track points detected meets the preset cumulative threshold, the most recent 5-10 track points are extracted to form a historical observation window. For the azimuth sequence and the detection timestamp sequence, the least squares linear regression method is used for slope estimation.

[0058] Ω A =∑((t p -t mean )*(A p -A mean )) / ∑((t p -t mean ) 2 );

[0059] Among them, Ω A The calculated azimuth angular velocity component; t p t is the timestamp of the p-th probe within the historical observation window; mean A is the time average of all detection times within the observation window; p Let A be the target azimuth angle value detected in the p-th test. mean This is the average azimuth angle of all targets within the observation window.

[0060] Similarly, the radial velocity component v of the target in the range dimension is calculated using the same regression logic. R .

[0061] In some alternative implementations, if the target exhibits obvious nonlinear motion characteristics, the sequence regression analysis can also use second-order polynomial fitting to obtain the target's angular acceleration parameters by extracting the quadratic term coefficients, thereby further improving the compensation accuracy of the maneuvering target.

[0062] Step 202: Use the currently detected target coordinates to pre-obtain the corresponding intermediate predicted pose, calculate the hardware execution time required for the gimbal to rotate from the current gimbal pose state to the intermediate predicted pose, and obtain the response delay duration.

[0063] Extract the target coordinates of the latest detection time from the temporal spatial coordinate sequence and use them as the target coordinates of the current detection. Perform spatial geometric trigonometric function encoding on the target coordinates of the current detection to obtain the preliminary input feature vector. Input the preliminary input feature vector into the spatial mapping model to perform preliminary prediction and obtain the corresponding intermediate prediction pose. Calculate the hardware execution time required for the gimbal to rotate from the current gimbal pose state to the intermediate prediction pose to obtain the response delay duration.

[0064] In some embodiments, this step may also involve performing preliminary model inference on the uncompensated version of the extended feature vector using the currently detected target coordinates to obtain the corresponding preliminary predicted pose, and calculating the hardware execution time required for the gimbal to rotate from the current gimbal pose state to the preliminary predicted pose in order to obtain the response delay duration.

[0065] Furthermore, the hardware execution time required for the gimbal to rotate from its current pose state to the intermediate predicted pose is calculated, such as... Figure 2 As shown, the process includes the following:

[0066] Determine the spatial pointing deviation between the intermediate predicted pose and the gimbal pose state;

[0067] The mechanical rotation time is obtained by calculating the ratio of the spatial pointing deviation value to the pre-configured maximum angular velocity of the gimbal.

[0068] The time consumed by mechanical rotation is added to the pre-configured mechanical stabilization time of the gimbal to obtain the hardware execution time, which is used as the response delay duration.

[0069] The target coordinates detected by the radar at the current moment are directly input into the spatial mapping model without motion compensation, resulting in a rough intermediate predicted pose, including the intermediate predicted horizontal rotation angle P. init With intermediate predicted pitch angle T init This intermediate predicted pose is only used to estimate the physical time required for the gimbal to reach the initial pointing position of the target, and is not the final alignment command.

[0070] To accurately predict the gimbal's arrival time, a physical model of the gimbal is constructed. Using the currently detected target coordinates as initial input, a coarse intermediate predicted pose is obtained through a mapping model. The spatial pointing deviation is specifically the absolute difference between the predicted horizontal rotation angle in the intermediate predicted pose and the actual horizontal rotation angle in the currently fed-back gimbal pose state.

[0071] In some embodiments, the spatial pointing deviation value Δε between the intermediate predicted pose and the current gimbal pose is determined, specifically as the larger of the absolute values ​​of the difference in horizontal rotation angle and the absolute values ​​of the difference in pitch angle:

[0072] Δε=max(|P init -P cur |,|T init -T cur |).

[0073] Among them, P init / T init These are the intermediate predicted horizontal and pitch angles (target position), P cur / T cur These are the current actual horizontal angle and pitch angle (current position) of the gimbal, respectively.

[0074] Furthermore, the calculation process for the hardware execution time is as follows:

[0075] Δ t =(Δε / Ω max )+t settle ;

[0076] Where, Δ t The calculated hardware execution time is the response latency; Δε is the calculated spatial pointing deviation value; Ω max The pre-configured maximum angular velocity of the gimbal reflects the limiting rotational speed of the gimbal motor under rated load; t settle The pre-configured gimbal mechanical stabilization time is used to characterize the damped oscillation time required for the gimbal to decelerate and stop until the image stabilizes.

[0077] After obtaining the response delay duration, a linear extrapolation is performed to generate the compensated coordinates. Specifically, the target azimuth angle A detected at the current moment is... now The azimuth component at the predicted time is obtained by adding the product of the azimuth angular velocity component and the response delay duration.

[0078] A comp =A now +Ω A *Δ t ;

[0079] Among them, A compThis refers to the azimuth term in the calculated compensated spatial coordinates.

[0080] Similarly, the radial velocity component is used to advance the target distance.

[0081] Therefore, the generated compensation spatial coordinates reflect the expected position of the target at the moment the gimbal's physical action is completed, enabling the center of the field of view of the photoelectric camera to accurately cover the dynamic target.

[0082] In some embodiments, the system also monitors changes in response latency in real time. In actual deployments, if a response latency Δ is detected... t If the preset safety threshold is exceeded, the compensation amount can be reduced or an alarm can be triggered according to actual needs. The specific anomaly handling strategy can be determined according to the actual deployment requirements.

[0083] Example 3 refines the feature engineering and mapping process in Example 1, describing how to use nonlinear feature transformation to solve the numerical jump problem in polar coordinates and enhance the physical representation of spatial features during model splitting.

[0084] In one possible implementation, spatial geometric trigonometric function encoding and coordinate system interaction coupling operations are performed on the compensated spatial coordinates to obtain an extended feature vector, such as... Figure 3 As shown, the specific steps include the following:

[0085] Step 301: Extract the azimuth and distance components from the compensated spatial coordinates, and perform sine and cosine operations on the azimuth components respectively to generate an orthogonal angular mapping feature that eliminates periodic discontinuities.

[0086] Step 302: Multiply the angle orthogonal mapping feature with the distance component to generate a Cartesian mapping feature that represents the spatial coupling relationship between azimuth and distance.

[0087] Step 303: Concatenate the angle orthogonal mapping features, Cartesian mapping features, and the original spatial coordinate sub-items to form an extended feature vector.

[0088] Specifically, the angle orthogonal mapping features and Cartesian mapping features are concatenated with the original spatial coordinate components to form an extended feature vector. In other words, the angle orthogonal mapping features and Cartesian mapping features are concatenated with the original values ​​of the azimuth and range components to form an extended feature vector.

[0089] In practical engineering applications of radar detection and electro-optical alignment, the original azimuth angle values ​​typically cycle within the range of 0 to 360 degrees. Because the node splitting mechanism of the random forest regression model is based on a threshold judgment of feature values, when the target moves within a small range near the 0-degree boundary, the original angle value will exhibit a jump from 359 degrees to 1 degree. To eliminate artificially introduced numerical abrupt changes, a dual-channel conversion of the azimuth angle is performed.

[0090] Specifically, the angular orthogonal mapping feature is composed of the sine and cosine values ​​of the azimuth angle. For example, when the azimuth angle is 359 degrees, its sine and cosine values ​​are very close to the corresponding values ​​when the azimuth angle is 1 degree, maintaining the continuity of spatially nearby targets in the vector space. Furthermore, the angular component and the range component are nonlinearly coupled to construct a more geometrically representative feature term.

[0091] The Cartesian mapping features correspond to the target's eastward and northward offsets on the horizontal plane.

[0092] x coord =R*sin(θ*π / 180); y coord =R*cos(θ*π / 180);

[0093] Where, x coord The characteristic of eastward offset, y coord R represents the northward offset characteristic, R represents the distance component in the compensated spatial coordinates, and θ represents the azimuth component in the compensated spatial coordinates.

[0094] By introducing such spatial interaction terms, the model can more accurately identify the heterogeneity of alignment errors across different spatial blocks. The resulting extended feature vector is a multidimensional vector.

[0095] As an example, in a sea-alignment scenario, the vector includes six feature dimensions: azimuth sine, azimuth cosine, eastward offset, northward offset, original azimuth value, and original distance value.

[0096] As an optional approach, the initial gimbal control parameters are output, including the following process:

[0097] The rotation angle orthogonal prediction component is received based on the extended feature vector output of the spatial mapping model; the rotation angle orthogonal prediction component is normalized and decoded using arctangent operation logic to recover continuous horizontal rotation angle values, which constitute the initial gimbal control parameters.

[0098] The process can be further implemented as follows: receiving the rotation angle orthogonal prediction component output by the spatial mapping model based on the extended feature vector, as well as the pitch angle prediction component and focal length prediction component directly output; using arctangent operation logic to perform directional normalization decoding on the rotation angle orthogonal prediction component to recover continuous horizontal rotation angle values; combining the horizontal rotation angle values, pitch angle prediction component and focal length prediction component to form the initial gimbal control parameters.

[0099] Not only is there a discontinuity issue at the input end, but the camera's horizontal rotation angle, or Pan angle, also exhibits periodic jumps during a continuous 360-degree rotation. If the regression model is directly driven to predict angle values, when targets close to 360 degrees and close to 0 degrees exist simultaneously in the training sample set, the model will attempt to take the arithmetic mean between the two, resulting in absurd predictions pointing in the 180-degree direction.

[0100] In this embodiment, the output layer of the spatial mapping model is also orthogonalized. The model no longer directly outputs a single angle value, but instead outputs two orthogonal components: the rotation angle orthogonal prediction component. These two components represent the sine and cosine components of the predicted angle, respectively. Accordingly, when training the model using this orthogonalized output design, the horizontal rotation angle values ​​in the training labels need to be pre-converted into their sine and cosine values ​​as the corresponding dimension's supervision target. Those skilled in the art will understand that such label format transformation is a conventional data preprocessing method.

[0101] After obtaining the model prediction results, the decoding operation is performed. Direction-normalized decoding is implemented using the four-quadrant arctangent function:

[0102] P final =atan2(sin P_pred cos P_pred )*180 / π;

[0103] Among them, P final To recover the obtained continuous horizontal rotation angle values, in degrees; atan2 is the standard four-quadrant arctangent operation function; sin P_pred The sine term in the orthogonal prediction component of the rotation angle of the spatial mapping model output; cos P_pred The cosine term in the orthogonal prediction component of the rotation angle output by the spatial mapping model.

[0104] Since the average of the predictions from each decision tree in the ensemble learning model may not strictly fall on the mathematical unit circle (i.e., the sum of the squares of the sine and cosine predictions may not equal 1), the atan2 function determines a unique direction solely based on the ratio of these two components. Therefore, this decoding method exhibits strong engineering robustness, ensuring smooth and continuous pointing motion of the gimbal when crossing the 0-degree boundary.

[0105] As an alternative, for aerial scenes, the original spatial coordinates also include the target's height component. The height component is retained directly as an independent feature term in the extended feature vector. Since height has a linear dominance in predicting the camera's pitch angle, retaining this original physical quantity can accelerate the model's convergence speed in the vertical dimension.

[0106] Example 4: Based on the above examples, this example further describes how to eliminate local alignment deviations caused by geographical heterogeneity and solve the gimbal jitter problem when the target crosses the grid boundary by using refined spatial segmentation and boundary transition zone fusion algorithms.

[0107] One possible implementation involves inputting the extended feature vector into a pre-built spatial mapping model, specifically including the following process:

[0108] Based on the compensated spatial coordinates and the preset spatial block division rules, the spatial block to which the target belongs is determined;

[0109] Determine whether a local model matching the spatial block exists in the pre-built local spatial mapping model library;

[0110] If it exists, the matched local model is used as the calling object, and the extended feature vector is input into it to perform prediction, so as to output the initial gimbal control parameters and the pointing confidence parameters.

[0111] If it does not exist, the model fallback mechanism is triggered, the pre-built global spatial mapping model is used as the calling object, and the extended feature vector is input into it to perform prediction, so as to output the initial gimbal control parameters and the pointing confidence parameters.

[0112] The preset spatial block division rules specifically include gridded partitioning of the azimuth and distance dimensions. For example, the 360-degree range is uniformly divided into NA azimuth sectors in the azimuth direction, and the range is divided into NR distance rings from the minimum to the maximum detection distance in the distance direction. Each grid is a spatial block, which can be represented as B. ij (The intersection of the i-th orientation sector and the j-th distance ring). The local spatial mapping model library pre-stores random forest models trained individually for predetermined spatial blocks.

[0113] In the specific execution process, the azimuth and distance components in the compensation spatial coordinates are first extracted. For example, if the current target's azimuth is 45 degrees and the distance is 5000 meters, its index value is calculated according to the division rules. If enough historical alignment samples have been accumulated within the block corresponding to the index, then that block has its own local model. By calling the local model, mapping deviations caused by mechanical installation deformation or local magnetic field interference specific to that area can be captured.

[0114] Furthermore, when the target is located in the edge region of radar detection or in an unpopular location where few targets appear, the corresponding spatial block may not meet the minimum training sample size requirement, leading to missing local models. In this case, an automatic model fallback mechanism is triggered. The model fallback mechanism refers to seamlessly switching the model input to a global spatial mapping model covering the entire airspace. The global model uses all historical data to learn general alignment rules, ensuring that the system can output effective alignment commands at any location.

[0115] In one possible implementation, after determining the spatial block to which the target belongs and calling the matching local model, the step of outputting the initial gimbal control parameters specifically includes:

[0116] Step 401: Calculate the shortest spatial distance from the compensated spatial coordinates to the nearest boundary of the spatial block;

[0117] Step 402: Determine whether the shortest spatial distance falls within the preset boundary transition zone;

[0118] If it does not fall into the space, the output of the local model corresponding to the spatial block will be used as the initial gimbal control parameters.

[0119] If it falls into the range, the local model of the adjacent spatial block on the opposite side of the nearest boundary is obtained, and the adjacent prediction parameters are output based on the extended feature vector. The distance decay weight is constructed based on the shortest spatial distance.

[0120] Step 403: Using distance attenuation weights, a weighted fusion operation is performed on the output of the local model corresponding to the spatial block and the adjacent prediction parameters to obtain the initial gimbal control parameters after smooth transition. Specifically, the pointing confidence output of the local model corresponding to the spatial block and the pointing confidence output of the local models of adjacent spatial blocks are weighted and fused using distance attenuation weights to obtain the fused pointing confidence parameters.

[0121] To avoid abrupt changes in output parameters when the target switches between different block models, this embodiment introduces a soft fusion algorithm. The shortest spatial distance refers to the minimum vertical distance among the four boundary lines of the grid to the target's current position coordinates. The boundary transition zone refers to a buffer layer inside the grid near the boundary.

[0122] For example, assuming the width of a spatial block is normalized to 1.0, the preset boundary transition band width Δ b Set to 0.2. When the shortest spatial distance d from the target to the boundary is calculated... b When d is 0.3, the target is determined to be in the center region of the block, and the prediction of the current local model is directly executed. If the target continues to move, causing d to... b If the value is reduced to 0.1, the fusion logic will be triggered.

[0123] When the fusion logic is triggered, the distance decay weight is first constructed.

[0124] η=d b / Δ b ;w main =(1+η) / 2;w neighbor =(1-η) / 2;

[0125] Where η is the normalized distance scaling factor, and its value ranges from 0 to 1; w main Weight components of the local model of the main block; w neighbor These are the weight components of the local model for adjacent blocks. Perform a weighted fusion operation:

[0126] y final =w main *y local +w neighbor *y neighbor ;

[0127] Among them, y final The initial gimbal control parameters after a smooth transition in the final output, y local The original output of the main block model, y neighbor This is the output of the adjacent block model. When the target is exactly on the boundary line, d... b When η is 0, the weights of both models are 0.5, achieving an equivalent connection of physical orientation. As the target penetrates deeper into the main block, η increases, and the weight of the main block model gradually becomes dominant.

[0128] Furthermore, as an alternative, for nearshore environments, the distance dimension can be divided using a logarithmic distribution to provide finer partitioning accuracy in the near-field region and better overcome alignment errors caused by near-field multipath effects.

[0129] In another alternative implementation, the spatial mapping model can also be implemented using Gradient Boosting Regression Tree (GBDT). GBDT gradually corrects alignment errors by iteratively fitting the prediction residuals from previous iterations, exhibiting good fitting accuracy in scenarios where the error distribution within each spatial block has a continuously varying characteristic. In this implementation, the pointing confidence parameter can be obtained by statistically estimating the variance of the prediction residuals over multiple iterations, and its calculation method is similar to the statistical dispersion extraction logic in the random forest implementation. The remaining processing flow remains consistent with the implementation using random forest.

[0130] Example 5: Based on the above examples, this example further describes how to use the statistical characteristics inside the mapping model to perceive the uncertainty of the alignment result in real time, and dynamically adjust the field of view of the photoelectric camera accordingly, so as to compensate for the pointing error by expanding the physical field of view in scenarios where the prediction accuracy decreases, thereby improving the success rate of the first target acquisition.

[0131] In one possible implementation, the spatial mapping model is an ensemble learning model composed of multiple basic tree models; the output is a direction confidence parameter extracted based on the internal prediction divergence of the model, including the following process:

[0132] Step 501: Input the extended feature vectors into each of the base tree models contained in the ensemble learning model in parallel to obtain the independent prediction components of the independent operation output of each base tree model.

[0133] Step 502: Calculate the statistical dispersion of all independent prediction components in the spatial dimension, and use the statistical dispersion as the directional confidence parameter to characterize the degree of prediction divergence.

[0134] Specifically, this process can be implemented as follows: inputting the extended feature vectors into each of the base tree models included in the ensemble learning model in parallel, and obtaining the independent prediction components output by each base tree model; performing mean aggregation on all independent prediction components in the corresponding dimension, and using the aggregation result as the initial gimbal control parameters; calculating the statistical dispersion of all independent prediction components in the spatial dimension, performing reference normalization on the statistical dispersion, and using the normalized dimensionless value as the directional confidence parameter representing the degree of prediction divergence.

[0135] In this embodiment, the spatial mapping model is constructed using the random forest algorithm, consisting of a predetermined number of base tree models, such as 300 independent decision trees. After obtaining the extended feature vector, it is simultaneously distributed to each decision tree for computation. Each decision tree outputs a set of corresponding horizontal rotation angle and pitch angle prediction values, i.e., independent prediction components, based on its internally defined splitting logic. The degree of dispersion of the prediction values ​​in the spatial dimension is calculated to quantify the model's grasp of the current input.

[0136] The specific calculation logic for the confidence parameter is as follows: the variance of the predicted values ​​from each decision tree is used as an evaluation index of uncertainty. The calculation process is as follows:

[0137] σ PT =sqrt((1 / NT)*∑((P n -P avg ) 2 +(T n -T avg ) 2 ));

[0138] Where, σ PT Here, NT represents the confidence parameter, i.e., the statistical dispersion, and P represents the total number of basic tree models. n T is the predicted horizontal rotation angle component output by the nth base tree model. n P is the pitch angle prediction component output by the nth base tree model. avg T is the mean horizontal rotation angle output by all basic tree models. avg The average pitch angle output by all base tree models.

[0139] Before mapping the pointing confidence parameter to the field of view adjustment factor, σ PT Perform reference normalization to obtain the normalized pointing confidence parameter σ. PT_norm :

[0140] σ PT_norm =σ PT / σ ref ;

[0141] Where, σ ref The pre-configured reference dispersion benchmark value can be determined based on σ in the historical validation set. PT The statistical median is determined so that the normalized σ PT_norm Under normal operating conditions, the values ​​are in the range [0, 5]. The attenuation sensitivity coefficient λ and the lower bound convergence boundary α in subsequent steps... min Both act on the normalized σ PT_norm , λ and σ PT_norm All are dimensionless quantities. σ ref The specific value can be obtained by statistically analyzing σ in all alignment tasks during the initial deployment phase (within the first month of the operating cycle). PT The median can be determined, or a value that maximizes the first capture rate can be selected on the cross-validation set.

[0142] Furthermore, the field-of-view adjustment factor is determined using the pointing confidence parameter. Based on the field-of-view adjustment factor, the focal length parameter in the initial gimbal control parameters is adaptively scaled, and the final gimbal target alignment command is output. Specifically, this may include the following process:

[0143] The pointing confidence parameter is mapped and transformed using a pre-configured nonlinear attenuation mapping logic to obtain the field of view adjustment factor; wherein, the field of view adjustment factor decreases monotonically as the prediction divergence represented by the pointing confidence parameter increases, and is not lower than the preset lower limit convergence boundary.

[0144] Multiply the focal length parameter in the initial gimbal control parameters with the field of view adjustment factor to obtain the final focal length parameter after performing the field of view expansion and magnification reduction operation;

[0145] The final focal length parameter is combined with the rotation angle parameter, which remains unchanged in the initial gimbal control parameters, to form the final gimbal target alignment command.

[0146] In this embodiment, the focal length parameter in the initial gimbal control parameters is directly predicted and output by the spatial mapping model. This focal length parameter is multiplied by the field of view adjustment factor to obtain the scaled final focal length parameter. The calculated final focal length parameter is then repackaged with the originally predicted rotation angle parameters (horizontal rotation angle and pitch angle) to generate a gimbal target alignment command containing complete PTZ (horizontal rotation angle / pitch angle / zoom magnification) control information, which is then sent to the camera controller to drive the gimbal action.

[0147] This adjustment logic establishes a feedback loop between model statistical uncertainty and the physical field of view. When the model has a high degree of certainty in predicting the target's motion compensation or spatial position, i.e., the pointing confidence parameter is small, a high focal length is maintained to obtain high-resolution details. When the target appears for the first time or the detection noise is large, causing the model confidence to decrease, i.e., the pointing confidence parameter is large, the field of view is automatically expanded to ensure that the target can fall within the photoelectric image range, and then the position is fine-tuned through the closed-loop self-supervised mechanism in subsequent embodiments.

[0148] In one possible implementation, the pointing confidence parameter is mapped and transformed using pre-configured nonlinear attenuation mapping logic to obtain the field of view adjustment factor. This process specifically includes the following steps:

[0149] Obtain the pre-configured attenuation sensitivity coefficient, and multiply the attenuation sensitivity coefficient with the pointing confidence parameter to obtain the attenuation index;

[0150] Using the natural constant e as the base and the negative value of the decay exponent as the power, perform exponential operations to calculate the exponential decay assessment value;

[0151] Perform a numerical comparison operation between the exponential decay assessment value and the preset lower limit convergence boundary;

[0152] Extract the larger value between the exponential decay assessment value and the preset lower limit convergence boundary, and use the larger value as the field of view adjustment factor after truncation protection.

[0153] After obtaining the pointing confidence parameter, a nonlinear transformation is performed. Since a larger pointing confidence parameter indicates greater disagreement in the prediction results, meaning the predicted pointing is less reliable, the focal length needs to be reduced accordingly to expand the field of view. To prevent image resolution loss due to excessively small focal length scaling in cases of extreme uncertainty, a truncation protection mechanism is introduced.

[0154] α Z =max(α min ,exp(-λ*σ PT_norm ));

[0155] Where, α Z α is the field-of-view adjustment factor, max is the comparison function for extracting larger values, and α min The lower bound convergence boundary is set to 0.4 in this embodiment. exp is an exponential function with the natural constant as the base, and λ is a pre-configured attenuation sensitivity coefficient. Its value must match the magnitude of the index confidence parameter. In this embodiment, λ is set to 0.5. Engineers can adjust λ on the validation set to observe the balance between the field of view expansion triggering timing and the first capture rate, and determine the value suitable for the actual deployment environment.

[0156] After obtaining the pointing confidence parameter, a nonlinear transformation is performed: using the pre-configured attenuation sensitivity coefficient λ and σ PT_norm The product of the two factors is used as the attenuation exponent, and the exponential attenuation assessment value is calculated using an exponential function. Then, the larger value between this assessment value and the preset lower limit convergence boundary is taken as the field of view adjustment factor after truncation protection. When the prediction results of each decision tree are highly consistent and the divergence is low, the field of view adjustment factor approaches 1, and the camera maintains high magnification to acquire target details. When the divergence of each tree increases, the field of view adjustment factor automatically decreases, and the camera moves to a wider field of view, effectively improving the first acquisition success rate, and not falling below the preset lower limit convergence boundary to ensure the minimum imaging resolution.

[0157] Example 6 further describes the visual feedback-driven closed-loop annotation process, which utilizes the image processing capabilities of the photoelectric camera itself to evaluate the alignment effect in real time and generate weighted training samples to drive the continuous evolution of the model, while providing an automated search and capture strategy in the case of target miss.

[0158] In one possible implementation, a pre-built spatial mapping model is constructed and continuously updated through the following self-supervised evolution process:

[0159] Step 601: Obtain pre-collected historical radar spatiotemporal coordinates and gimbal pose pairing data, and construct a basic training set to train and obtain an initial spatial mapping model.

[0160] Step 602: During the online operation of the model and the execution of the alignment task, obtain closed-loop verification data driven by visual confirmation.

[0161] Step 603: Based on the closed-loop verification data, the basic training set is supplemented with samples and weighted with confidence to generate a weighted dynamic sample set.

[0162] Step 604: Utilize a weighted dynamic sample set to perform an asynchronous retraining mechanism on the spatial mapping model to achieve online evolution of the spatial mapping model.

[0163] The historical radar spatiotemporal coordinates and gimbal pose pairing data are acquired during the initial calibration phase after system deployment. Specifically, multiple known location targets or cooperative targets covering different azimuth angles and distance ranges are selected. The radar detects and obtains their spatiotemporal coordinates, while simultaneously, a manual operator aligns the electro-optical camera with the target and records the gimbal's pose parameters after positioning, forming an input-output pairing sample. The number and distribution of calibration targets can be determined based on the spatial coverage requirements of the deployment environment.

[0164] In this embodiment, an initial spatial mapping model is established using historical alignment data accumulated offline. Once in online operation, it no longer relies solely on manual annotation but achieves self-evolution through a closed-loop feedback mechanism.

[0165] Specifically, the visual confirmation result after each alignment command is executed is used as a supervision signal. The basic training set refers to the static sample library stored in the system's backend database. The closed-loop verification data refers to the target's actual position information confirmed by the image algorithm and the corresponding radar feature input. By continuously injecting closed-loop verification data into the sample library, the training set is dynamically expanded.

[0166] Furthermore, to differentiate between automatically labeled samples of varying quality, a confidence weighting mechanism is introduced. Confidence weighting involves calculating a weight value between 0 and 1 for each new sample based on the magnitude of the angular deviation in visual evaluation. This weight value reflects the reliability of the sample as a true alignment benchmark. Through this method, high-quality samples can be automatically selected for subsequent model retraining, ensuring that the online evolution process is not contaminated by image detection errors.

[0167] According to one aspect of this application, during the initial construction of the spatial mapping model, the model uses the extracted extended feature vector as the input tensor and manually calibrated horizontal rotation angle, pitch angle, and focal length parameters as supervised output labels. Since the alignment task is a continuous variable regression problem, the optimization objective of model training is set to minimize the mean squared error (MSE) between the predicted output and the true label. The mean squared error loss function for each sample's corresponding dimension is calculated, and each decision tree uses the variance reduction as the node splitting criterion to recursively fit the training data. The mean squared error (MSE) between the predicted output and the true label for each sample on the validation set is used as the overall accuracy evaluation index. After confirming that the model's prediction accuracy on the validation set meets the requirements, the model parameters are solidified, establishing the nonlinear mapping relationship between the input spatial coordinates and the output gimbal pose.

[0168] In one possible implementation, visual verification-driven closed-loop validation data is obtained, and the basic training set is supplemented with samples and weighted with confidence levels based on the closed-loop validation data to generate a weighted dynamic sample set, including the following process:

[0169] Acquire the current image frame after the photoelectric camera has rotated into position according to the final gimbal target alignment command;

[0170] Perform target detection on the current image frame, extract the target pixel offset between the target center and the image reference center, and map the target pixel offset into an angle deviation value;

[0171] Based on the magnitude range of the angle deviation values, sample confidence weights with non-linear decay are assigned to the final gimbal target alignment command.

[0172] The final gimbal target alignment command carrying sample confidence weights and the extended feature vector associated with this prediction are used as closed-loop verification data and appended to a pre-established sample buffer queue to form a weighted dynamic sample set.

[0173] During the acquisition of closed-loop verification data, the target detection algorithm is run through the image processing unit integrated within the optoelectronic system. Target detection can employ conventional deep convolutional target detection methods in the field, such as two-stage detectors or single-stage detectors based on candidate regions. The relevant network can be trained using publicly available radar-guided optoelectronic target datasets or manually annotated local datasets. Engineers can select appropriate detection methods based on the actual deployed computing resources and target types. The training data for the target detection model can be obtained from public datasets or through manual annotation in actual deployment scenarios. The specific scale and annotation granularity of the training data can be adjusted according to the actual detection accuracy requirements.

[0174] An image processing unit integrated within the optoelectronic system runs a target detection algorithm on the current image frame. The algorithm takes the image frame as input and outputs the target bounding box coordinates; it extracts the center pixel coordinates (u, v) of the bounding box as the target center. The target pixel offset is the two-dimensional difference between the target center pixel coordinates (u, v) and the image center reference coordinates (u0, v0). Using the camera's horizontal and vertical field of view, as well as the image resolution, this pixel offset is linearly converted into the corresponding angular deviation value.

[0175] In this embodiment, the allocation logic of sample confidence weights adopts an exponential decay model:

[0176] ;

[0177] Where w is the assigned sample confidence weight; exp is the natural exponential function; Δθ norm For reference field of view θ fov Normalized dimensionless angular deviation, Δθ norm =Δθ / θ fov θ fovThe current field of view of the camera can be half of the angle, where Δθ is the angle deviation value before normalization; α is a preset attenuation coefficient used to control the rate at which the weight decreases as the deviation increases. The specific value can be determined through experiments on the validation set according to the requirements of the annotation accuracy in the actual application scenario.

[0178] When the target accurately hits the center of the image, Δθ norm When the value approaches 0, the sample confidence weight w is 1.0; when the target deviates from the center but is within the field of view, the weight decreases exponentially with the square of the deviation. This nonlinear weighting strategy ensures that samples with more accurate alignment have a higher say in model updates, guiding the model to converge toward the global optimum.

[0179] As an optional approach, if no valid target is extracted in the current image frame during the target detection step, an anomaly detection process is executed:

[0180] Using the gimbal pose corresponding to the final gimbal target alignment command as the origin, drive the photoelectric camera to perform a local spiral expansion scan within a preset range;

[0181] If the target is recaptured during the local spiral expansion scan, the actual alignment parameters at this time are recorded, the downgraded sample is given a confidence weight, and the actual alignment parameters and associated input features are used as closed-loop verification data.

[0182] If no capture is found after traversing the scan range, the alignment sample record for the current cycle is discarded.

[0183] According to one aspect of this application, the process may further include: using the gimbal pose corresponding to the final gimbal target alignment command as the origin, driving the photoelectric camera to perform a local spiral expansion scan within a preset range; continuously performing target detection on image frames acquired at each scanning position during the local spiral expansion scan; if the target is recaptured during the local spiral expansion scan, recording the actual alignment parameters at this time, assigning a pre-configured downgraded sample confidence weight, and using the actual alignment parameters and the expansion feature vector associated with this alignment as closed-loop verification data; if the target is still not captured after traversing the scanning range, discarding the alignment sample record for the current period.

[0184] Considering that initial alignment may miss the target under certain extreme weather conditions or high-maneuver scenarios, i.e., the target detector fails to detect the target in the current image frame, this embodiment provides a fallback search mechanism. Local spiral extended scanning refers to the gimbal performing incremental rotational scanning radially outward from the predicted position along a preset Archimedean spiral path. The relationship between the angular radius of the spiral path and the cumulative rotation angle can be expressed as:

[0185] r=k'*φ';

[0186] Where r is the angular radius relative to the initial pointing at the current moment; φ' is the cumulative rotation angle of the gimbal; and k' is the path growth rate coefficient, the value of which is determined by the size of the current field of view of the camera to ensure that the scanning path can achieve complete coverage of the overlapping area.

[0187] If the target is recaptured during the scanning process, the current actual pose of the gimbal is recorded as the actual alignment parameters. Since these parameters include time delays and accumulated mechanical errors from the scanning process, their confidence level is lower than that of directly hit samples. Therefore, a fixed, downgraded sample confidence weight is assigned to these samples, for example, set to 0.3. If the target is still not captured after scanning within a preset range, such as 3 to 5 degrees, the alignment task is considered a failure, related data is automatically removed, and the background sample buffer (sample buffer queue) is not updated, ensuring the purity of the training data.

[0188] As an optional implementation, for fast-moving targets appearing in close-range scenes, the target's angular velocity parameter is also considered when calculating the sample confidence weights. For samples with high angular velocities, even if the angular deviation is small, the weight component is appropriately reduced to eliminate the potential adverse effects of motion blur on annotation accuracy.

[0189] Example 7, as a further refinement of the weighted sample application step in Example 6, describes how an engineered hot update mechanism is used to address the technical constraint that standard random forests do not support incremental updates, and ensures that the radar alignment system achieves continuous self-evolution of accuracy while operating without interruption.

[0190] One possible implementation involves using a weighted, dynamic sample set to perform an asynchronous retraining mechanism on the spatial mapping model, such as... Figure 4 As shown, the process includes the following:

[0191] Step 701: When the cumulative number of new samples in the weighted dynamic sample set reaches the pre-configured update trigger threshold, perform weighted probability sampling with replacement based on the confidence weight of each sample in the weighted dynamic sample set to construct a retraining subset.

[0192] Step 702: Without blocking the current online prediction task, train and generate candidate mapping models in an independent background thread using a retraining subset;

[0193] Step 703: After the candidate mapping model is trained, the spatial mapping model currently called by the online environment is seamlessly replaced with the candidate mapping model using atomic pointer switching logic.

[0194] In this embodiment, since the random forest model is a static ensemble learning architecture, it is difficult to perform online weight updates for individual samples using gradient descent like a neural network. To enable the system to absorb the generated loop closure verification data, a first-in-first-out (FIFO) sample buffer is maintained for each local spatial mapping model and the global spatial mapping model. The sample buffer is set to a maximum capacity N. max For example, it can be set to 500 samples. When the buffer is full, the oldest sample will be automatically removed.

[0195] Real-time monitoring of the number of samples added to the buffer since the last training session. A pre-configured update trigger threshold Δ. n The number of samples can be dynamically adjusted based on real-time computing resource load, with a preferred value of 20. When the cumulative number of new samples reaches 20, the retraining condition is deemed met. To enable the model to focus on learning high-confidence alignment experiences, weighted sampling logic is executed. When constructing the retraining subset for training each decision tree in the random forest, the probability of each sample being selected is proportional to its confidence weight.

[0196] P k =w k / ∑ w ;

[0197] Among them, P k w represents the probability that the k-th sample is selected into the retraining subset. k ∑ is the confidence weight corresponding to this sample. w This is the sum of the weights of all samples in the current sample buffer.

[0198] The sampling method with replacement ensures that high-weight, high-quality samples appear multiple times in the retraining subset, thereby enhancing the model's learning strength for high-precision alignment patterns without changing the algorithm structure.

[0199] To ensure the real-time performance of the online inference service and avoid delays in alignment command issuance due to excessive processing resource consumption during training, a separate background thread is initiated to perform the model refitting process. Meanwhile, the main thread continues to provide alignment services for radar targets using the existing spatial mapping model. Once the background thread completes the construction of the candidate mapping model, it performs an atomic pointer switching operation.

[0200] Specifically, a global memory pointer is maintained, pointing to the currently active model. The atomic pointer switching logic utilizes underlying atomic operation instructions to switch the pointer's address from the old model to the candidate mapping model within a short clock cycle. This operation is atomic, ensuring that no inference requests access illegal memory addresses during the switch. After seamless replacement, the memory resources occupied by the old model are released.

[0201] Furthermore, as an optional implementation, a multi-level triggering strategy can be adopted for command and control center platforms with strong computing power. Specifically, for local spatial mapping models, the update triggering threshold is set to a smaller value, such as 10, to ensure that the system can quickly capture local environmental drift in the predetermined spatial domain; while for global spatial mapping models, the update triggering threshold is set to a larger value, such as 100, to reduce the computational overhead caused by full training of large amounts of data, thus achieving a balance between update timeliness and computational cost.

[0202] Example 8: Based on the above examples, this example further describes how, during the offline construction phase, the spatial grid is adaptively subdivided and merged by using both predicted residuals and data sparsity as indicators, thereby generating a preset spatial block partitioning rule and local model library that highly matches the error distribution of the physical environment.

[0203] One possible implementation is, such as Figure 5 As shown, the preset spatial block partitioning rules and the local spatial mapping models included in the local spatial mapping model library are pre-generated through the following error-driven adaptive partitioning logic:

[0204] Step 801: Obtain the initial coarse-grained spatial grid and calculate the comprehensive pointing error index of the prediction output on the reserved validation set corresponding to each grid.

[0205] Specifically, this step can also involve: obtaining an initial coarse-grained spatial grid; spatially partitioning the historical sample data according to the initial coarse-grained spatial grid, dividing each grid into a training subset and a reserved validation subset, and training the initial local spatial mapping model corresponding to each grid based on the training subset corresponding to each grid; and calculating the comprehensive pointing error index of the prediction output of the initial local spatial mapping model on the reserved validation subset corresponding to each grid.

[0206] In this embodiment, the basic spatial layout is initialized during the initial construction phase. Centered on the origin of the radar coordinate system, the three-dimensional surveillance airspace is projected onto a two-dimensional plane and divided into multiple independent blocks based on azimuth and range dimensions, forming an initial coarse-grained spatial grid. The number of azimuth sectors can be set to 8, and the number of range rings can be set to 4.

[0207] The boundary points of the azimuth sector are calculated as follows: A i =(360 / M0)*i;

[0208] Among them, A i Let M0 be the starting boundary angle of the i-th azimuth sector, M0 be the total number of azimuth sectors, and i be the sector index, which is an integer between 0 and M0.

[0209] The boundary points of the distance loop are calculated as follows: D j=D min +(D max -D min ) / N0*j;

[0210] Among them, D j D is the starting boundary distance of the j-th distance ring. min D represents the minimum detection range of the radar. max N0 is the maximum detection range of the radar, N0 is the set total number of range rings, and j is the range ring index with an integer value between 0 and N0.

[0211] After initial grid partitioning and training of the initial regression model for each grid, the prediction accuracy is evaluated on their respective independent reserved validation sets. The overall pointing error index is quantified using the average absolute value of the prediction error.

[0212] MAE ij =∑(sqrt((P pred_k -P true_k ) 2 +(T pred_k -T true_k ) 2 )) / V ij ;

[0213] Among them, MAE ij V is the comprehensive pointing error index of the grid where the i-th azimuth sector intersects with the j-th distance ring. ij P represents the total number of samples in the reserved validation set corresponding to this grid. pred_k P is the predicted value of the horizontal rotation angle for the k-th sample. true_k T represents the true labeled value of the horizontal rotation angle of the k-th sample. pred_k T represents the predicted pitch angle value for the k-th sample. true_k ∑ represents the true labeled pitch angle of the k-th sample, and ∑ represents the summation operation performed on all independent samples in the reserved validation set.

[0214] Step 802: Determine whether the comprehensive pointing error index of each grid is greater than the preset subdivision trigger threshold, and whether the sample retention within the grid meets the minimum subdivision limit.

[0215] The accuracy and data sufficiency of each grid are assessed simultaneously. The preset partitioning trigger threshold can be set to 2.0 degrees. When the overall pointing error index exceeds this threshold, it indicates that there is severe environmental heterogeneity within the current grid. For example, some areas may experience multipath effects due to mountain shading, while other areas may have open views. A single basic tree model cannot simultaneously account for the nonlinear mapping of these two physical states, and the spatial resolution needs to be improved.

[0216] Simultaneously, a minimum subdivision limit is introduced as a constraint. This minimum subdivision limit can be set to 50 samples. This value ensures that the subdivided sub-blocks still possess the minimum data scale required to train the base tree model. Its specific value can be determined through pre-experiments based on parameters such as the number of sub-blocks in the subdivision method, the model tree depth, and the number of decision trees. If the sample size of the current grid is lower than this limit, forced subdivision will result in insufficient data within the generated sub-grids to support model fitting, leading to severe overfitting.

[0217] Step 803: Perform a recursive sub-block subdivision operation on the spatial dimension for the grid that meets the above conditions, and train and assign an independent local spatial mapping model for each subdivided sub-block.

[0218] For grids that simultaneously meet the conditions of exceeding error limits and having sufficient data, a quadtree partitioning algorithm is used to perform a cross-section along the median of the azimuth dimension and the median of the distance dimension, dividing the original parent grid into four sub-blocks proportionally. After the spatial range is fragmented, the database is re-traversed, and historical training samples falling within the coordinate range of each sub-block are clustered and extracted. Using the extracted subset data, a random forest regression model is refitted for each sub-block.

[0219] Optionally, during model training, the number of decision trees is set to 300, and the minimum number of leaf node samples is set to 10. Hyperparameters such as the number of decision trees and the minimum number of leaf node samples can be determined using conventional hyperparameter tuning methods such as cross-validation.

[0220] Furthermore, when training the local spatial mapping model separately for each sub-block, due to the differences in feature distribution within each sub-block, each decision tree uses variance reduction as the splitting criterion for feature selection when splitting nodes. That is, at each internal node, the variance reduction of all candidate splitting features is calculated, and the feature that minimizes the sum of the variances of the target values ​​of the left and right child node samples after the split, along with its corresponding threshold, is selected as the splitting point.

[0221] Meanwhile, in order to ensure the generalization ability of the ensemble model and the differences between the trees, only a subset of features is randomly selected from all available features to participate in the optimal split evaluation each time a node splits. The number of features selected is preferably the square root of the total feature dimension or one-third of the total feature dimension, which ensures the model's accurate fitting of the local spatial heterogeneity mapping relationship at the algorithm level.

[0222] Step 804: Repeatedly perform error calculation and subdivision operations until the comprehensive pointing error index of all blocks converges to within the subdivision trigger threshold, thus solidifying the preset spatial block division rules.

[0223] Specifically, this step can also involve repeatedly performing error calculation and subdivision operations until the comprehensive pointing error index of all blocks converges to within the subdivision trigger threshold, or the spatial span of the sub-blocks has reached the preset physical control limit resolution, thus solidifying the preset spatial block division rules, and storing the local spatial mapping model corresponding to each block into the local spatial mapping model library.

[0224] The aforementioned spatial adaptive partitioning is a multi-round recursive process. For each newly generated sub-block, the validation set partitioning, error calculation, and threshold judgment logic are repeatedly executed. The termination conditions for this recursive process include two aspects: first, the calculated result of the comprehensive pointing error index of the sub-block is strictly less than the preset partitioning trigger threshold; second, continuous partitioning causes the azimuth span or distance span of the sub-block to reach the preset physical device control limit resolution. When all grids in the entire domain satisfy either termination condition and the partitioning stops, the boundary coordinate set of all current grids is extracted and stored as a preset spatial block partitioning rule.

[0225] In one possible implementation, the error-driven adaptive partitioning logic also includes grid merging rules based on data sparsity:

[0226] Step 805: Extract two spatially adjacent grids, calculate the absolute error difference between the comprehensive pointing error indices of the two grids, and count the sample retention of each grid.

[0227] In addition to the error-driven subdivision operation, a reverse data sparsity optimization mechanism was designed. The current fixed spatial layout topology is scanned to identify adjacent grid pairs sharing boundary coordinates. For any pair of adjacent grids, their respective comprehensive pointing error indices are retrieved, the difference is calculated, and the absolute value is taken to generate an absolute error difference. Simultaneously, the number of valid labeled data entries within each grid is counted to obtain their respective sample retention parameters.

[0228] Step 806: If the absolute error difference is less than the preset error similarity judgment threshold, and the sample retention of both grids is lower than the preset sparsity judgment threshold, then the two grids are merged into a single expanded grid in the spatial dimension.

[0229] The preset error similarity threshold can be set to 0.5 degrees, and the preset sparsity threshold can be set to 50 samples. When the absolute error difference between two adjacent grids is within the error similarity threshold range, it indicates that the electromagnetic wave propagation paths and optical mapping laws in the two spatial regions exhibit high homogeneity.

[0230] If, under these conditions, neither of the two grids has reached the minimum data volume required for independently training a robust model, a reverse merging instruction is executed. In the memory logic, the common boundary line between the two grids is erased, and their spatial coverage areas are merged to generate a single expanded grid with a wider boundary range.

[0231] Step 807: Aggregate the reserved validation set corresponding to the single expanded grid, and retrain the single local spatial mapping model for the merged region. Update the single local spatial mapping model and store it in the local spatial mapping model library, and update the two grid boundaries corresponding to the preset spatial block division rules to the boundaries of the single expanded grid.

[0232] After the spatial extent is merged, the underlying data is integrated simultaneously. The training data points originally belonging to two independent grids and the reserved validation set are concatenated and aggregated. Using the expanded training dataset, the algorithm engine is launched to refit a local spatial mapping model covering the entire single expanded grid. This merging mechanism effectively compensates for the model unavailability problem caused by fixed grid division in areas with uneven data distribution.

[0233] In some alternative implementations, for typical marine surveillance environments, target tracks are dense and multipath interference is frequent in the near-range area, while targets at the sea level are sparse and the background is open at a distance. In this case, the initial distance grid division method in step 801 no longer adopts an equidistant linear uniform distribution, but is replaced by a logarithmic distribution calculation model.

[0234] D j =D min *(D max / D min ) (j / N0) ;

[0235] Among them, D j D is the starting boundary distance of the j-th distance ring in the logarithmic distribution pattern. min The minimum detection range set for the radar, D max The maximum detection range is set for the radar, N0 is the total number of range rings, and j is the corresponding range ring index value.

[0236] By using a nonlinear space pre-allocation scheme, a denser grid resource is configured for the near-distance high-frequency dynamic space during the initialization phase, reducing the recursive rounds of subsequent adaptive partitioning and lowering the overall computational cost of construction.

[0237] According to one aspect of this application, a temporal spatial coordinate sequence generated by continuous radar detection of a target and the gimbal pose state of an electro-optical camera are obtained, as follows:

[0238] In practical deployments, if the photoelectric alignment system is installed on a non-fixed base such as a ship or vehicle-mounted platform, the platform's real-time attitude changes will directly cause the geometric mapping between the radar detection coordinates and the camera's physical pointing to fail. In this embodiment, before acquiring data, an inertial navigation system is connected to obtain the platform's roll, pitch, and pitch angles in real time in the geographic coordinate system.

[0239] Specifically, a coordinate transformation matrix is ​​used to transform the relative azimuth and range output by the radar to the geodetic coordinate system, specifically the NE-G coordinate system. Let the platform's current roll angle be φ, pitch angle be β, and heave angle be ψ. The calculation logic of the coordinate transformation matrix M is as follows:

[0240] M=M z (ψ)*M y (β)*M x (φ);

[0241] Where M is the composite coordinate rotation matrix, M z M y M x These are rotation sub-matrices around the vertical, horizontal, and vertical axes, respectively. By multiplying each detection point in the radar temporal spatial coordinate sequence with the rotation matrix M, the virtual motion component introduced by platform sway is eliminated, ensuring that subsequent sequence regression analysis can extract the target's true motion state parameters. Correspondingly, the camera's gimbal pose state is also mapped to a stable geodetic coordinate system through this matrix, making the training and inference of the mapping model based on a unified and stable spatiotemporal reference.

[0242] According to one aspect of this application, a field-of-view adjustment factor is determined using a pointing confidence parameter, and the focal length parameter in the initial gimbal control parameters is adaptively scaled based on the field-of-view adjustment factor to output the final gimbal target alignment command.

[0243] In some optional embodiments of this example, for complex scenarios where there are multiple targets to be detected on the sea surface or in the airspace, priority scheduling logic is added before outputting the final gimbal target alignment command. Since the camera is a narrow field-of-view sensor, it can only be aligned with a single target at a time. The threat level or observation value of the target is evaluated based on the motion state parameters of each target acquired by the radar, and a priority queue is maintained.

[0244] Specifically, a comprehensive score is calculated for each objective.

[0245] Score=(w1×V target / V ref +w2×(D ref / D target )+w3×σ PT_norm );

[0246] Where Score is the target overall score, w1, w2, and w3 are preset weighting coefficients, and V target D represents the target's velocity. target V is the physical distance to the target from the sensor. ref For the pre-configured reference speed base value, D ref Using the reference distance baseline, targets are extracted from the queue in descending order of score and the alignment process is executed.

[0247] By employing the above methods, we can ensure that the most threatening or the most uncertain targets are captured first in a multi-target concurrent environment, thereby improving overall reconnaissance efficiency.

[0248] According to one aspect of this application, a pre-configured nonlinear attenuation mapping logic is used to map and transform the pointing confidence parameter to obtain the field of view adjustment factor. As another implementation of the focal length control logic in Embodiment 5, this embodiment provides a focal length prediction strategy based on a distance baseline to improve the robustness of the focal length parameters during the initial gimbal control parameter generation stage. In this method, the absolute focal length is not directly predicted by the model; instead, a baseline focal length based on the physical distance is calculated first.

[0249] Z base =Z ref *ln(D / D near +1) / ln(D cal / D near +1);

[0250] Among them, Z base Let Z be the calculated focal length baseline value, ln be the natural logarithm function, and Z be the focal length baseline value. ref D is the calibration focal length multiplier at the preset reference distance, and D is the current detection distance. cal To calibrate the reference distance, D near A preset near-range bias constant is used to prevent near-range numerical singularities.

[0251] In this implementation, the mapping model only needs to learn the focal length residual correction Δ caused by atmospheric refraction or mounting deviation. Z The initial focal length parameter is determined by Z. base With Δ Z The sum is obtained. By introducing a physical baseline, the difficulty of fitting the model to the focal length dimension is significantly reduced. Especially when aligning distant targets, this method can more stably maintain an effective field of view magnification. Combined with the secondary scaling of the field of view adjustment factor, it further enhances the system's acquisition probability across the entire range.

[0252] In summary, through the multi-scenario adaptation mechanism provided in this embodiment, the present invention can effectively address the technical challenges brought about by platform shaking, multi-target competition, and large-span distance changes, ensuring the high availability and robustness of the alignment method in complex engineering environments.

[0253] In typical maritime surveillance scenarios, compared to traditional methods using fixed geometric coordinate transformation matrices, the method of this invention significantly improves the first-shot acquisition rate for fast-moving targets with speeds exceeding 30 knots. In long-range target alignment scenarios exceeding 15 km, the alignment angle deviation is significantly lower than that of traditional methods. In continuous operation scenarios, the self-supervised evolution mechanism continuously improves alignment accuracy over time. It should be understood that the specific improvement may vary depending on the application scenario and target characteristics.

[0254] This application employs a motion state extraction and response time-based physical modeling method based on temporal regression to achieve forward compensation of the target prediction time coordinates. This method transforms the alignment lag defect of highly maneuverable targets into lead lock, effectively reducing the phenomenon that the target has moved out of the field of view after the gimbal has reached its position from a physical mechanism perspective.

[0255] The adoption of double-ended trigonometric coding effectively suppressed model numerical failure and gimbal swaying caused by periodic jumps. At the same time, the introduction of spatial adaptive regional modeling and boundary distance weighted soft fusion effectively smoothed out local alignment errors in complex environments, achieving a smooth and jitter-free transition when the gimbal crosses different monitoring areas.

[0256] Uncertainty is quantified by extracting the prediction divergence degree within the ensemble learning model, and this drives the nonlinear adaptive scaling of the focal length parameter. When the prediction confidence is low, the physical field of view is automatically expanded to offset the risk of missed detection. On this basis, a self-supervised verification and asynchronous hot-switching mechanism based on visual pixel deviation attenuation weighting is established, which gives the system the ability to automatically correct deviations and evolve online, and improves the robustness of the equipment in the whole life cycle collaborative detection in the absence of human intervention.

[0257] The preferred embodiments of the present invention have been described in detail above. However, the present invention is not limited to the specific details in the above embodiments. Within the scope of the technical concept of the present invention, various equivalent transformations can be made to the technical solutions of the present invention, and these equivalent transformations all fall within the protection scope of the present invention.

Claims

1. A machine learning based radar-optical target alignment method, characterized in that, include: Acquire the temporal and spatial coordinate sequence generated by the radar's continuous detection of the target, as well as the gimbal pose state of the electro-optical camera; The motion state parameters of the target are estimated based on the temporal spatial coordinate sequence, and the response delay is predicted by combining the gimbal pose state. The compensated spatial coordinates of the target at the predicted time are calculated based on the motion state parameters and the response delay. Spatial geometric trigonometric function encoding and coordinate system interaction coupling operation are performed on the compensated spatial coordinates to obtain the extended feature vector; The extended feature vector is input into the pre-built spatial mapping model, and the output is the initial gimbal control parameters and the pointing confidence parameters extracted based on the internal prediction divergence of the model. The field of view adjustment factor is determined by the pointing confidence parameter, and the focal length parameter in the initial gimbal control parameters is adaptively scaled accordingly to output the final gimbal target alignment command. Spatial geometric trigonometric function encoding and coordinate system interaction coupling operations are performed on the compensated spatial coordinates to obtain extended feature vectors, including: The azimuth and range components in the compensated spatial coordinates are extracted, and sine and cosine operations are performed on the azimuth components respectively to generate an angle orthogonal mapping feature that eliminates periodic discontinuity. Multiply the angle orthogonal mapping feature with the distance component to generate a Cartesian mapping feature that represents the coupling relationship between azimuth and distance space; The angle orthogonal mapping feature, the Cartesian mapping feature, and the original spatial coordinate sub-item are concatenated to form an extended feature vector; Input the extended feature vector into the pre-built spatial mapping model, including: Based on the compensated spatial coordinates and the preset spatial block division rules, the spatial block to which the target belongs is determined; Determine whether a local model matching the spatial block exists in the pre-built local spatial mapping model library; If it exists, the matched local model is used as the calling object, and the extended feature vector is input into it to perform prediction; if it does not exist, the model fallback mechanism is triggered, the pre-built global space mapping model is used as the calling object, and the extended feature vector is input into it to perform prediction. Output the initial gimbal control parameters and pointing confidence parameters.

2. The method of claim 1, wherein, The motion state parameters of the target are estimated based on the temporal spatial coordinate sequence, and the response delay is predicted by combining the gimbal pose state, including: Based on the historical observation windows in the time-series spatial coordinate sequence, sequence regression analysis is performed to extract the azimuth angular velocity component and radial velocity component of the target to form motion state parameters; By using the currently detected target coordinates to pre-obtain the corresponding intermediate predicted pose, the hardware execution time required for the gimbal to rotate from the current gimbal pose state to the intermediate predicted pose is calculated to obtain the response latency.

3. The method of claim 2, wherein, The hardware execution time required for the gimbal to rotate from its current pose state to the intermediate predicted pose includes: Determine the spatial pointing deviation between the intermediate predicted pose and the gimbal pose state; The mechanical rotation time is obtained by calculating the ratio of the spatial pointing deviation value to the pre-configured maximum angular velocity of the gimbal. The time consumed by mechanical rotation is added to the pre-configured mechanical stabilization time of the gimbal to obtain the hardware execution time, which is used as the response latency.

4. The method of claim 1, wherein, The spatial mapping model is an ensemble learning model composed of multiple basic tree models; the output is a direction confidence parameter extracted based on the internal prediction divergence of the model, including: The extended feature vectors are input in parallel into each of the base tree models contained in the ensemble learning model to obtain the independent prediction components of the independent operation output of each base tree model. Calculate the statistical dispersion of all independent prediction components in the spatial dimension, and use the statistical dispersion as a directional confidence parameter to characterize the degree of prediction divergence.

5. The method of claim 1, wherein, The pre-built spatial mapping model is constructed and continuously updated through the following self-supervised evolution process: Acquire pre-collected historical radar spatiotemporal coordinates and gimbal pose pairing data, construct a basic training set to train and obtain an initial spatial mapping model; During the online operation of the model and the execution of alignment tasks, visual verification-driven closed-loop verification data is acquired. Based on the closed-loop verification data, the basic training set is supplemented with samples and weighted with confidence to generate a weighted dynamic sample set. An asynchronous retraining mechanism is used to perform the spatial mapping model on a weighted dynamic sample set to achieve online evolution of the spatial mapping model.

6. The method of claim 5, wherein, An asynchronous retraining mechanism is used to perform training on the spatial mapping model using a weighted dynamic sample set, including: When the cumulative number of new samples in the weighted dynamic sample set reaches the pre-configured update trigger threshold, weighted probability sampling with replacement is performed based on the confidence weight of each sample in the weighted dynamic sample set to construct a retraining subset. Without blocking the current online prediction task, candidate mapping models are trained and generated in an independent background thread using a retraining subset. After the candidate mapping model is trained, the atomic pointer switching logic is used to seamlessly replace the spatial mapping model called by the current online environment with the candidate mapping model.

7. The method of claim 2, wherein, Before performing sequence regression analysis based on historical observation windows in the time-series spatial coordinate sequence, pre-interception verification logic is also included: The cumulative number of trajectory points detected within the statistical time-series spatial coordinate sequence; If the cumulative number of track points detected does not reach the preset cumulative threshold, the current target coordinates will be used as the compensation spatial coordinates, and the subsequent velocity component extraction and time estimation process will be skipped.

8. A machine learning based radar-optical target alignment system, characterized by, include: At least one processor; as well as, A memory communicatively connected to at least one processor; wherein the memory stores instructions executable by the processor to implement the method of any one of claims 1 to 7.

Images