Optoelectronic sighting method and system based on predictive compensation

Abstract

The application relates to the technical field of photoelectric aiming, in particular to a photoelectric aiming method and system based on prediction compensation, which comprises the following steps: collecting spectral image data of a work target, various environment data in a work environment and distance data of a work device to the work target in real time; obtaining three-dimensional space coordinates of the work target for a single time; dividing a preset time period before single photoelectric aiming into response windows of single environment data for a single photoelectric aiming process of the work target; obtaining effective correlation response values between single environment data and the three-dimensional space coordinates; obtaining an effective cycle length for position prediction of the work target; predicting the three-dimensional space coordinates of the work target, and correcting an aiming line to perform photoelectric aiming work. The application aims to predict the actual position of the work target at a future time, thereby effectively compensating for the position deviation of the work target caused by environment changes in the work process, and improving the photoelectric aiming work precision.

Description

Technical Field

[0001] This application relates to the field of optoelectronic aiming technology, specifically to an optoelectronic aiming method and system based on predictive compensation. Background Technology

[0002] Photoelectric aiming technology is a core technology for high-precision positioning and operation. It boasts advantages such as non-contact aiming, fast response speed, and high positioning accuracy, and is widely used in laser weeding robots, intelligent agricultural equipment, industrial inspection, and security monitoring. By deeply integrating photoelectric aiming technology with optical sensing and mechanical control, it can improve the stability and accuracy of applications in complex scenarios, thereby enhancing the precision and automation level of photoelectric aiming systems.

[0003] However, in practical applications, photoelectric aiming systems are easily affected by environmental factors, making it difficult for photoelectric aiming to compensate for dynamic errors during operation through real-time feedback control alone. This results in a large deviation between the working position of the equipment and the actual position of the target, leading to a decrease in the accuracy of photoelectric aiming operations. Summary of the Invention

[0004] In view of the above, it is necessary to provide an optoelectronic aiming method and system based on prediction compensation. Compared with traditional optoelectronic aiming methods based on prediction compensation, this method effectively compensates for the target position deviation caused by environmental changes during the operation by predicting the actual position of the target at future moments, thereby improving the accuracy of optoelectronic aiming. In a first aspect, embodiments of this application provide an optoelectronic aiming method based on prediction compensation, the method comprising the following steps: Synchronously and in real time, it collects spectral image data of the target, various environmental data in the working environment, and distance data between the working equipment and the target; For a single moment, a probabilistic feature map is obtained by fusing spectral image data and distance data, and then the target area is segmented to extract the two-dimensional pixel coordinates of the target. The three-dimensional spatial coordinates of the target are then obtained through projection transformation. For a single photoelectric aiming process of the target, the cycle duration of a single environmental data is obtained by observing the changes in a single environmental data within a preset time period before the single photoelectric aiming, so that the preset time period is divided into response windows for the single environmental data; by observing the correlation between the single environmental data and the three-dimensional spatial coordinates in each response window, and the distribution of the change in the single environmental data within the preset time period, the effective correlation response value between the single environmental data and the three-dimensional spatial coordinates is obtained. By integrating the effective correlation response value with the period duration of all environmental data, the effective period duration for predicting the position of the target is obtained; by extracting the three-dimensional spatial coordinates and all environmental data within the effective period duration before a single photoelectric aiming, the three-dimensional spatial coordinates of the target are predicted, and the aiming line is then corrected to perform the photoelectric aiming operation.

[0005] In one embodiment, the environmental data includes: wind speed data, wind direction data, light intensity data, temperature data, and humidity data.

[0006] In one embodiment, the process of dividing the response window is as follows: For a single type of environmental data, calculate the difference between any two adjacent time points. Use the autocorrelation function to perform autocorrelation analysis on the difference between any two adjacent time points within the preset time period. Extract the first peak point on the autocorrelation function curve after removing the zero delay point. Record the lag time corresponding to the first peak point as the period duration. The preset time period is evenly divided into response windows of equal length to the cycle duration.

[0007] In one embodiment, the process of obtaining the valid associated response value is as follows: By analyzing the correlation between changes in individual environmental data and three-dimensional spatial coordinates in each response window, short-term correlation response values ​​between individual environmental data and three-dimensional spatial coordinates are obtained. By analyzing the distribution of changes in a single type of environmental data across all response windows within the preset time period, the change response value of the single type of environmental data in each response window is obtained. The effective associated response value is the weighted sum of the short-term associated response values ​​in all response windows, wherein the weight of the short-term associated response value in each response window is the change response value of that response window.

[0008] In one embodiment, the process of obtaining the short-term correlation response value is as follows: Calculate the metric distance between the three-dimensional spatial coordinates of any two adjacent moments; The short-term correlation response value is the absolute value of the temporal correlation coefficient between the difference value and the metric distance in each response window.

[0009] In one embodiment, the process of obtaining the change response value is as follows: Calculate the mean of all the difference values ​​in each response window, and calculate the arithmetic mean of all the difference values ​​within the preset time period; The change response value is positively correlated with the mean and negatively correlated with the arithmetic mean.

[0010] In one embodiment, the process of obtaining the effective period duration is as follows: Obtain the percentage of the effective correlation response value between a single type of environmental data and three-dimensional spatial coordinates relative to the sum of the effective correlation response values ​​between all types of environmental data and three-dimensional spatial coordinates; By combining the numerical proportions with the cycle duration of all environmental data within the preset time period, the effective cycle duration for predicting the location of the operational target is obtained.

[0011] In one embodiment, the effective period duration is the weighted sum of the period durations of all environmental data within the preset time period, wherein the weight of the period duration of a single environmental data within the preset time period is the percentage of the numerical value corresponding to that single environmental data.

[0012] In one embodiment, during the process of predicting the three-dimensional spatial coordinates of the target, the time span of the time series prediction model predicting the three-dimensional spatial coordinates of the target must be greater than or equal to the feature duration of the prediction compensation. The feature duration is calculated as follows: extracting the total time spent on data acquisition, transmission and preprocessing before prediction; extracting the total time from data reading to outputting the prediction result of the three-dimensional spatial coordinates of the target during the prediction process; and extracting the time spent after outputting the prediction result of the three-dimensional spatial coordinates of the target, from issuing the command to the aiming line to aim at the target and stabilize after the control turntable adjusts the aiming angle and position. The characteristic duration is the sum of the total time consumed, the total duration, and the time consumed.

[0013] Secondly, embodiments of this application also provide an optoelectronic aiming system based on prediction compensation, including a memory, a processor, and a computer program stored in the memory and running on the processor, wherein the processor executes the computer program to implement the steps of any of the above-described optoelectronic aiming methods based on prediction compensation.

[0014] This application has at least the following beneficial effects: This application fuses spectral image data and distance data using a feature fusion method, segments the target area, and extracts two-dimensional pixel coordinates. This accurately reflects the core position of the target and reduces positional shifts caused by target edge swaying. Furthermore, transforming the two-dimensional pixel coordinates into three-dimensional spatial coordinates facilitates subsequent analysis of the correlation between the target and environmental data. By analyzing changes in environmental data, the application effectively captures the periodic patterns of environmental data changes, achieving refined time segmentation of environmental dynamic characteristics. It further analyzes the short-term correlation between environmental data changes and target displacement, improving the ability to capture the dynamic mapping relationship between environmental disturbances and target movement. Analyzing the distribution of environmental data changes helps distinguish between truly significant environmental state changes and minor fluctuations, effectively assessing the confidence level of the impact of environmental changes on the target within each time window. Finally, calculating the effective correlation response value comprehensively reflects the significance and reliability of the impact of a single type of environmental data on the target position, thus providing a quantitative basis for determining the optimal prediction time scale. Furthermore, by comprehensively considering the period duration of different environmental data, the effective period duration for predicting the target position can be determined. This improves the accuracy of target position prediction under environmental influences, effectively compensating for target position deviations caused by environmental changes during operations, and enabling accurate prediction of the target's actual position at future moments. Based on the prediction results, the aiming line is corrected and aligned with the expected position of the target at future moments. This effectively reduces the impact of dynamic interference errors during actual operations, significantly reducing the target miss rate and the accidental damage rate from the surrounding environment. This achieves high-precision photoelectric aiming operations and improves the quality and efficiency of photoelectric aiming operations. Attached Figure Description

[0015] Figure 1 A flowchart illustrating the steps of a predictive compensation-based photoelectric aiming method provided in one embodiment of this application; Figure 2 This is a schematic diagram illustrating the process of obtaining the effective period duration; Figure 3 This is a schematic diagram of the prediction process for three-dimensional spatial coordinates. Detailed Implementation

[0016] The following description, in conjunction with the accompanying drawings, details the specific scheme of the electro-optical aiming method and system based on prediction compensation provided in this application.

[0017] Please see Figure 1 The diagram illustrates a flowchart of a predictive compensation-based photoelectric aiming method according to an embodiment of this application, which includes the following steps: Step 1: Synchronously and in real time collect spectral image data of the target, various environmental data in the working environment, and distance data between the working equipment and the target.

[0018] To improve the accuracy of subsequent photoelectric aiming operations, various data are collected from the equipment equipped with the photoelectric aiming system during actual operation. This allows for precise analysis of the impact of environmental factors on the accuracy of photoelectric aiming during the actual operation. The specific data collection process is as follows: During actual operations, a high-definition visible and near-infrared dual-band camera, a laser rangefinder, and various environmental sensors are used to simultaneously collect multiple types of data in real time. Specifically: the high-definition visible and near-infrared dual-band camera is used to collect spectral image data of the target in real time, including near-infrared and visible light image data; the laser rangefinder is used to collect distance data from the equipment to the target in real time; the wind speed sensor is used to collect wind speed data in the working environment in real time; the wind direction sensor is used to collect wind direction data in the working environment in real time; the light intensity sensor is used to collect light intensity data in the working environment in real time; the temperature sensor is used to collect temperature data in the working environment in real time; and the humidity sensor is used to collect humidity data in the working environment in real time.

[0019] In this embodiment, in order to ensure accurate capture of the operation target and analysis of the dynamic characteristics of the environment, spectral image data, distance data, wind speed data, wind direction data, light intensity data, temperature data and humidity data are all collected at a high frequency of 100Hz. The collection frequency is preset by the user and the implementer can set it according to the actual situation. This application does not impose any special restrictions.

[0020] All collected data is transmitted to the edge computing node of the operating equipment. At the same time, a data caching and synchronization calibration mechanism is adopted to eliminate the delay error of data collected by different sensors, ensuring that the multi-source data is consistent in time and space, and avoiding the problem of reduced accuracy of subsequent analysis and recognition due to data acquisition delay.

[0021] Step 2: For a single moment, a probability feature map is obtained by fusing spectral image data and distance data, and then the target area is segmented to extract the two-dimensional pixel coordinates of the target. The three-dimensional spatial coordinates of the target are then obtained through projection transformation.

[0022] To accurately extract the location of the target and improve the accuracy of subsequent prediction and compensation, the collected data is first preprocessed. Specifically, due to the influence of sensor noise, environmental electromagnetic interference, sudden changes in light reflection, and carrier vibration during the actual acquisition process, the quality of the acquired data is poor. Therefore, guided filtering algorithm is used to denoise the spectral image data to preserve the target edge features and suppress Gaussian and salt-and-pepper noise. Adaptive Kalman filtering is used to denoise the distance data and various environmental data to remove high-frequency random noise from the sensor, retain the true dynamic characteristics of the target displacement and the low-frequency change trend of the environmental state, thereby improving the quality of the acquired data. This improves the accuracy of subsequent target identification and the stability of the time series prediction model used for prediction and compensation.

[0023] Furthermore, to accurately extract the location of the operational target and improve the accuracy of prediction and compensation during the operation, a feature fusion method based on Bayesian estimation is used to fuse near-infrared image data, visible light image data, and distance data for a single moment. The fused result is a two-dimensional image with the same resolution as the original near-infrared and visible light image data. The pixel value of each pixel in the two-dimensional image represents the initial probability that the pixel belongs to the operational target, thus accurately reflecting the characteristics of the operational target and reducing the deviation of the operational target characteristics caused by the uncertainty of a single data point. The fused result is used as input to perform image segmentation using the U-Net semantic segmentation model, segmenting the image to obtain the operational target region, which is used to clarify the spatial distribution range of the operational target. Both the Bayesian estimation-based feature fusion method and the U-Net semantic segmentation model are well-known technologies and will not be described further in this application.

[0024] In this embodiment, the training set of the U-Net semantic segmentation model specifically consists of 5000 two-dimensional images obtained by a feature fusion method based on Bayesian estimation. Pixels in each two-dimensional image are manually labeled, with the label categories including target and background. The training loss function is the cross-entropy loss function, and the optimizer is Adam. The value of 5000 is merely one embodiment of this application; implementers can set its specific value according to actual circumstances, and this application does not impose any special limitations.

[0025] Furthermore, for a single moment, based on the segmented target area, the two-dimensional pixel coordinates of the target are extracted using the gray-scale centroid method. When extracting the two-dimensional pixel coordinates of the target using the gray-scale centroid method, the pixel value of each pixel within the target area is used as a weight, which can accurately reflect the core position of the target and reduce positional offset caused by target edge swaying. To analyze the correlation between the target position and environmental parameters, thereby obtaining a precise real-time target position, camera calibration and perspective projection transformation methods are used to transform the two-dimensional pixel coordinates into three-dimensional spatial coordinates, achieving precise spatial positioning of the target. The gray-scale centroid method, as well as the camera calibration and perspective projection transformation methods, are well-known technologies and will not be elaborated upon in this application.

[0026] Step 3: For a single photoelectric aiming process of the target, the preset time period before the single photoelectric aiming is divided into response windows for each type of environmental data; the effective correlation response value between the single type of environmental data and the three-dimensional spatial coordinates is obtained; the effective cycle time for predicting the position of the target is obtained; and the three-dimensional spatial coordinates of the target are predicted.

[0027] During photoelectric aiming, changes in environmental conditions can alter the motion state of the target, causing a deviation between the target's position obtained through sensor data acquisition and processing and its actual position. Therefore, during photoelectric aiming, it is necessary to analyze the dynamic characteristics of the target's position as it changes with the environment over a short period of time, predict and analyze the target's position, and compensate for the positional deviation caused by environmental changes during the operation.

[0028] Step 3.1: For a single photoelectric aiming process of the target, the cycle duration of a single environmental data is obtained by observing the changes in a single environmental data within a preset time period before the single photoelectric aiming, so as to divide the preset time period into response windows for each single environmental data.

[0029] For a single photoelectric aiming process of the target, the distance between the three-dimensional spatial coordinates of the target between any two adjacent moments within a preset time period before the single photoelectric aiming is recorded as the displacement change of the target between those two adjacent moments. The larger the calculated displacement change, the more drastic the change in the target's state within a short period. Since environmental changes are the main factor causing changes in the target's state during actual operations, the variation characteristics of the collected environmental data are analyzed. Taking the x-th type of environmental data as an example, the difference between any two adjacent moments of the x-th type of environmental data within the preset time period is recorded as the environmental state change of the x-th type of environmental data between those two adjacent moments. The larger the calculated environmental state change, the more drastic the environmental state change of the x-th type of environmental data within a short period, and the greater its potential impact on the target's state. By calculating the environmental state change and displacement change, subsequent correlation analysis can be performed to extract the data layer mapping features between multi-source disturbances and the target's position, thereby assisting subsequent time series prediction models in time series forecasting.

[0030] In this embodiment, the length of the preset time period is 5 seconds. The length of the preset time period is preset by a person, and the implementer can set it according to the actual situation. This application does not impose any special restrictions.

[0031] In this embodiment, the metric distance between three-dimensional spatial coordinates is specifically Euclidean distance.

[0032] In this embodiment, the difference between the xth type of environmental data is the absolute value of the difference. As for other implementations, while still being able to measure the degree of difference between the xth type of environmental data, the implementer may use other calculation methods, such as the square of the difference, the ratio, etc., and this application does not impose any special restrictions. It should be added that: for wind direction data, since the wind direction data is circular data from 0 degrees to 360 degrees, the formula for calculating the difference value C between the wind direction data is: , where min() represents the minimum value operation, and A and B both represent wind direction data.

[0033] Furthermore, considering that in actual operation, the photoelectric aiming system has a lag time in data reading and analysis, transmission and actuator response, and environmental changes may cause the target position to shift during this lag time, in order to accurately analyze the mutual influence characteristics between the target state and the environmental state in a short period of time, the correlation between the change in environmental state of the xth type of environmental data and the change in displacement of the target is analyzed within the preset time period, and the influence characteristics of the change in environmental state on the displacement of the target are judged based on the analysis results.

[0034] Based on the above analysis, by observing the changes in the x-th type of environmental data within the preset time period before a single photoelectric aiming, the period duration of the x-th type of environmental data is obtained, and the preset time period is divided into response windows for the x-th type of environmental data. The specific process is as follows: Autocorrelation analysis is performed on the changes in environmental state between any two adjacent moments within the preset time period using an autocorrelation function. The first peak point after removing the zero-delay point is extracted from the autocorrelation function curve, and the lag time corresponding to the first peak point is recorded as the period duration. The preset time period is then uniformly divided into response windows of equal length to the period duration. The autocorrelation function is a known technique and will not be described further in this application.

[0035] It should be added that if uniform division cannot be achieved, the window whose duration is not equal to the period duration after the final division will be retained as a response window.

[0036] It should be added that: if the absolute value of the amplitude of all peak points on the autocorrelation function curve after removing the zero delay point is less than the preset significance threshold, it indicates that the x-th environmental data does not show obvious periodic fluctuation characteristics within the preset time period. The period duration is assigned as the preset base duration. In this embodiment, the preset significance threshold and the preset base duration are 0.3 and 0.5s, respectively. The values ​​of the preset significance threshold and the preset base duration are calculated from experimental data.

[0037] Step 3.2: Obtain the effective correlation response value between a single environmental data and three-dimensional spatial coordinates by measuring the correlation between the changes of a single environmental data in each response window and the three-dimensional spatial coordinates, as well as the distribution of the changes of the single environmental data within the preset time period.

[0038] Furthermore, by analyzing the correlation between the x-th environmental data and the three-dimensional spatial coordinates in each response window, the short-time correlation response value between the x-th environmental data and the three-dimensional spatial coordinates in each response window is obtained, specifically as follows: The absolute value of the temporal correlation coefficient between the environmental state change and displacement change of the x-th environmental data in each response window is taken as the short-time correlation response value between the x-th environmental data and the three-dimensional spatial coordinates in each response window.

[0039] In this embodiment, the process of calculating the temporal correlation coefficient between the environmental state change and displacement change of the x-th environmental data in each response window is as follows: The environmental state change of the x-th environmental data in each response window is arranged chronologically between any two times to form a sequence of environmental state change of the x-th environmental data in each response window; the displacement change of the work target in each response window is arranged chronologically between any two times to form a sequence of displacement change of the work target in each response window; and the Pearson correlation coefficient between the environmental state change sequence and the displacement change sequence is calculated. The calculation of the Pearson correlation coefficient is a well-known technique and will not be elaborated upon in this application. As other implementation methods, based on the ability to measure the correlation between the environmental state change sequence and the displacement change sequence, implementers may use other existing feasible techniques, such as the Spearman correlation coefficient, etc. This application does not impose any special limitations.

[0040] It should be noted that the larger the calculated short-term correlation response value, the more significant the correlation between the change in the location of the task target and the change in the environmental data is in each response window of the x-th type of environmental data.

[0041] Furthermore, if the change of the xth environmental data within each response window is more significant compared to the overall change within the preset time period, it indicates a higher confidence level of the job target state changing with the environmental state within each response window, meaning a greater likelihood that drastic changes in the environmental state will cause a change in the job target state. Therefore, the change response value of the xth environmental data in each response window is obtained by analyzing the distribution of the change in the xth environmental data across all response windows within the preset time period, specifically as follows: Calculate the mean of the environmental state change of the x-th type of environmental data between any two times in each response window, and calculate the arithmetic mean of the environmental state change of the x-th type of environmental data between any two times in the preset time period. The change response value of the xth type of environmental data in each response window is positively correlated with the mean and negatively correlated with the arithmetic mean.

[0042] It should be noted that: positive correlation means that the variables change in the same direction, that is, when one variable increases, the other variable also increases, and when one variable decreases, the other variable also decreases; negative correlation means that the variables change in opposite directions, that is, when one variable increases, the other variable decreases, and when one variable decreases, the other variable increases.

[0043] In this embodiment, the ratio of the mean to the arithmetic mean is used as the change response value for each response window.

[0044] It should be noted that the larger the calculated change response value, the greater the possibility that the target state of the operation will change due to drastic changes in the environmental conditions within each response window.

[0045] It should be added that when calculating the change response value, if there is a case where the arithmetic mean is 0, the arithmetic mean is first mapped to a positive number before subsequent calculations are performed. There are many methods to map the data to a positive number, and the implementer can choose an existing feasible method according to the actual situation. In this embodiment, the purpose of mapping the data to a positive number is achieved by calculating the sum of the data and a preset value greater than 0. The value of the preset value greater than 0 is preset by the implementer and can be set by the implementer according to the actual situation. This application does not impose any special restrictions. In this embodiment, the value of the preset value greater than 0 is 0.01.

[0046] Furthermore, by using the short-term correlation response values ​​between the x-th environmental data and the three-dimensional spatial coordinates in each response window, and the change response values ​​of the x-th environmental data in each response window, the effective correlation response value between the x-th environmental data and the three-dimensional spatial coordinates is obtained, expressed as: In the formula, This represents the effective correlation response value between the x-th type of environmental data and the three-dimensional spatial coordinates; n represents the number of response windows for the x-th type of environmental data. This represents the short-time correlation response value between the x-th environmental data and the three-dimensional spatial coordinates in the i-th response window; This represents the response value of the x-th type of environmental data in the i-th response window.

[0047] It should be noted that the larger the calculated effective correlation response value, the more significant the impact of the change in the state of the xth type of environmental data on the state of the operational target.

[0048] Step 3.3: By combining the period duration of all environmental data with the effective correlation response value, the effective period duration for predicting the location of the operation target is obtained.

[0049] To determine the optimal historical data length for predicting the location of the operational target, we calculate and analyze the effective correlation response values ​​between different types of environmental data and three-dimensional spatial coordinates, and determine the effective period length accordingly, so as to accurately predict the location of the operational target at future moments.

[0050] Based on the above analysis, by combining the effective correlation response values ​​between all types of environmental data and three-dimensional spatial coordinates, and integrating the period duration of all types of environmental data, the effective period duration for predicting the location of the operational target is obtained, expressed as: In the formula, T represents the effective period for predicting the location of the operational target; m represents the number of types of environmental data. This represents the period duration of the xth type of environmental data; This represents the percentage of the effective correlation response value between the x-th type of environmental data and the three-dimensional spatial coordinates, relative to the sum of the effective correlation response values ​​between all types of environmental data and the three-dimensional spatial coordinates. A schematic diagram illustrating the process for obtaining the effective period duration is shown below. Figure 2 As shown.

[0051] It should be noted that the larger the proportion of the calculated x-th environmental data value, the greater the impact of the x-th environmental data on the target status within the cycle duration, and the higher the accuracy of target location prediction based on the cycle duration of the x-th environmental data. By comprehensively considering the cycle durations of different environmental data, the effective cycle duration for target location prediction can be determined, thereby improving the accuracy of target location prediction under environmental influences and achieving efficient photoelectric aiming operations.

[0052] Step 3.4: By extracting the three-dimensional spatial coordinates and all environmental data within the effective period before a single photoelectric aiming, the three-dimensional spatial coordinates of the target are predicted.

[0053] Furthermore, by extracting the three-dimensional spatial coordinates and all environmental data within the effective period prior to a single photoelectric aiming operation, a time series prediction model is used to predict the three-dimensional spatial coordinates of the target. A schematic diagram of the three-dimensional spatial coordinate prediction process is shown below. Figure 3 As shown.

[0054] In this embodiment, a Long Short-Term Memory (LSTM) network model is used to predict the three-dimensional spatial coordinates of the task target. The time step of the LSTM model is calculated by rounding up the product of the effective period duration and the acquisition frequency. The loss function used in the training process of the LSTM model is the weighted least mean square error function, and the optimizer is the Adam optimizer. To avoid the influence of differences in environmental data dimensions on the training and parameter optimization of the LSTM model used for prediction compensation, the collected environmental data are normalized to improve the convergence speed and computational accuracy of the LSTM model. Furthermore, the environmental data and the three-dimensional spatial coordinates of the task target are time-aligned to improve the LSTM model's adaptability to dynamic scenes and the specificity of error correction. In addition, to ensure the consistency of the input feature scale of the LSTM model, the three-dimensional spatial coordinates of the target are normalized, specifically the coordinate values ​​of the x-axis, y-axis, and z-axis. Since the coordinate values ​​in the predicted three-dimensional spatial coordinates of the target are also normalized after the above processing, it is necessary to perform reverse normalization on the predicted coordinate values ​​in the three-dimensional spatial coordinates of the target to restore them to the physical predicted values ​​of the three-dimensional spatial coordinates.

[0055] In this embodiment, the Z-Score normalization method is used to normalize the various environmental data collected. The Z-Score normalization method is a well-known technology and will not be described in detail in this application.

[0056] In this embodiment, the Min-Max normalization method is used to normalize the coordinate values ​​of the x-axis, y-axis, and z-axis respectively. Taking the normalization of the x-axis coordinate values ​​as an example, the maximum value refers to the maximum value among all three-dimensional spatial coordinate x-axis coordinate values ​​within the effective period, and the minimum value refers to the minimum value among all three-dimensional spatial coordinate x-axis coordinate values ​​within the effective period. The Min-Max normalization method is a well-known technique and will not be described in detail in this application.

[0057] To achieve accurate prediction and compensation, the time span for the time series prediction model to predict the 3D spatial coordinates of the target must be greater than or equal to the characteristic duration of the prediction and compensation. The characteristic duration is calculated as follows: the total time spent on data acquisition, transmission, and preprocessing before prediction; the total time spent from data reading to outputting the prediction result of the target's 3D spatial coordinates during prediction; and the time spent from issuing the command to adjust the aiming angle and position of the control turntable to aiming at the target and stabilizing it after outputting the prediction result. The characteristic duration is the sum of the total time spent, the total duration, and the time spent. In summary, using the time series prediction model based on data within the effective period, the 3D spatial coordinates of the target are predicted at a time after the last data acquisition moment within the effective period, with a time interval equal to the characteristic duration. These coordinates are used as the actual position coordinates of the target for accurate determination of the target's position during photoelectric aiming.

[0058] Step 4: Adjust the aiming line to perform electro-optical aiming.

[0059] Based on the characteristics of the impact of environmental interference on the target state during actual operation, the position of the target is predicted, optimized, and compensated to determine the actual position coordinates of the target at a future time. According to the prediction results of the target's three-dimensional spatial coordinates, the aiming line between the laser emission point and the actual position of the target is re-determined to achieve precise photoelectric aiming. The aiming line is the correction result after analyzing the impact of environmental interference. Specifically, based on the predicted actual position coordinates and the corrected aiming line, the turntable of the photoelectric aiming system is driven to align the aiming line with the expected position of the target at a future time in advance. That is, the aiming line is aligned with the predicted actual position coordinates of the target in advance, thereby reducing the impact of dynamic interference errors during actual operation and achieving high-precision photoelectric aiming.

[0060] Furthermore, during operation, the photoelectric aiming system dynamically adjusts the aiming angle and position via a turntable based on real-time updated predictive compensation results, continuously tracking the movement of the target. Even in scenarios with rapidly changing targets or fluctuating environmental parameters, it maintains stable alignment between the aiming line and the target, ensuring the continuity and reliability of the operation. When the aiming line is precisely aligned with the actual position coordinates of the target, the laser emission module triggers a firing command, achieving precise strike or detection of the target. This significantly reduces the target miss rate and the accidental damage rate to the surrounding environment, improving operational quality and efficiency, thus fully leveraging the advantages of photoelectric aiming technology in complex dynamic scenarios.

[0061] Based on the same inventive concept as the above methods, this application also provides an optoelectronic aiming system based on prediction compensation, including a memory, a processor, and a computer program stored in the memory and running on the processor. When the processor executes the computer program, it implements the steps of any one of the above optoelectronic aiming methods based on prediction compensation.

[0062] In summary, this application fuses spectral image data and distance data using a feature fusion method, segments the target area, and extracts two-dimensional pixel coordinates. This accurately reflects the core position of the target and reduces positional shifts caused by target edge swaying. Furthermore, transforming the two-dimensional pixel coordinates into three-dimensional spatial coordinates facilitates subsequent analysis of the correlation between the target and environmental data. By analyzing changes in environmental data, the periodic patterns of environmental data changes can be effectively captured, enabling refined time segmentation of dynamic environmental characteristics. Analyzing the short-term correlation between environmental data changes and target displacement enhances the ability to capture the dynamic mapping relationship between environmental disturbances and target movement. Analyzing the distribution of environmental data changes helps distinguish between truly significant environmental state changes and minor fluctuations, effectively assessing the confidence level of the impact of environmental changes on the target within each time window. Finally, calculating the effective correlation response value comprehensively reflects the significance and reliability of the impact of a single type of environmental data on the target position, thus providing a quantitative basis for determining the optimal prediction time scale. Furthermore, by comprehensively considering the period duration of different environmental data, the effective period duration for predicting the target position can be determined. This improves the accuracy of target position prediction under environmental influences, effectively compensating for target position deviations caused by environmental changes during operations, and enabling accurate prediction of the target's actual position at future moments. Based on the prediction results, the aiming line is corrected and aligned with the expected position of the target at future moments. This effectively reduces the impact of dynamic interference errors during actual operations, significantly reducing the target miss rate and the accidental damage rate from the surrounding environment. This achieves high-precision photoelectric aiming operations and improves the quality and efficiency of photoelectric aiming operations.

Claims

1. A photoelectric aiming method based on predictive compensation, characterized in that, The method includes the following steps: Synchronously and in real time, it collects spectral image data of the target, various environmental data in the working environment, and distance data between the working equipment and the target; For a single moment, a probabilistic feature map is obtained by fusing spectral image data and distance data, and then the target area is segmented to extract the two-dimensional pixel coordinates of the target. The three-dimensional spatial coordinates of the target are then obtained through projection transformation. For a single photoelectric aiming process of the target, the cycle duration of a single environmental data is obtained by observing the changes in a single environmental data within a preset time period before the single photoelectric aiming, so that the preset time period is divided into response windows for the single environmental data; by observing the correlation between the single environmental data and the three-dimensional spatial coordinates in each response window, and the distribution of the change in the single environmental data within the preset time period, the effective correlation response value between the single environmental data and the three-dimensional spatial coordinates is obtained. By integrating the effective correlation response value with the period duration of all environmental data, the effective period duration for predicting the position of the target is obtained; by extracting the three-dimensional spatial coordinates and all environmental data within the effective period duration before a single photoelectric aiming, the three-dimensional spatial coordinates of the target are predicted, and the aiming line is then corrected to perform the photoelectric aiming operation.

2. The photoelectric aiming method based on prediction compensation as described in claim 1, characterized in that, The environmental data includes: wind speed data, wind direction data, light intensity data, temperature data, and humidity data.

3. The photoelectric aiming method based on prediction compensation as described in claim 1, characterized in that, The process of dividing the response window is as follows: For a single type of environmental data, calculate the difference between any two adjacent time points. Use the autocorrelation function to perform autocorrelation analysis on the difference between any two adjacent time points within the preset time period. Extract the first peak point on the autocorrelation function curve after removing the zero delay point. Record the lag time corresponding to the first peak point as the period duration. The preset time period is evenly divided into response windows of equal length to the cycle duration.

4. The photoelectric aiming method based on prediction compensation as described in claim 3, characterized in that, The process for obtaining the effective correlation response value is as follows: By analyzing the correlation between changes in individual environmental data and three-dimensional spatial coordinates in each response window, short-term correlation response values ​​between individual environmental data and three-dimensional spatial coordinates are obtained. By analyzing the distribution of changes in a single type of environmental data across all response windows within the preset time period, the change response value of the single type of environmental data in each response window is obtained. The effective associated response value is the weighted sum of the short-term associated response values ​​in all response windows, wherein the weight of the short-term associated response value in each response window is the change response value of that response window.

5. The photoelectric aiming method based on prediction compensation as described in claim 4, characterized in that, The process for obtaining the short-time correlation response value is as follows: Calculate the metric distance between the three-dimensional spatial coordinates of any two adjacent moments; The short-term correlation response value is the absolute value of the temporal correlation coefficient between the difference value and the metric distance in each response window.

6. The photoelectric aiming method based on prediction compensation as described in claim 4, characterized in that, The process for obtaining the change response value is as follows: Calculate the mean of all the difference values ​​in each response window, and calculate the arithmetic mean of all the difference values ​​within the preset time period; The change response value is positively correlated with the mean and negatively correlated with the arithmetic mean.

7. The photoelectric aiming method based on prediction compensation as described in claim 1, characterized in that, The process for obtaining the effective period duration is as follows: Obtain the percentage of the effective correlation response value between a single type of environmental data and three-dimensional spatial coordinates relative to the sum of the effective correlation response values ​​between all types of environmental data and three-dimensional spatial coordinates; By combining the numerical proportions with the cycle duration of all environmental data within the preset time period, the effective cycle duration for predicting the location of the operational target is obtained.

8. The photoelectric aiming method based on prediction compensation as described in claim 7, characterized in that, The effective period duration is the weighted sum of the period durations of all environmental data within the preset time period, wherein the weight of the period duration of a single environmental data within the preset time period is the percentage of the value corresponding to that single environmental data.

9. The photoelectric aiming method based on prediction compensation as described in claim 1, characterized in that, In the process of predicting the three-dimensional spatial coordinates of the target, the time span of the time series prediction model in predicting the three-dimensional spatial coordinates of the target must be greater than or equal to the characteristic duration of the prediction compensation. The characteristic duration is calculated as follows: extract the total time spent on data acquisition, transmission and preprocessing before prediction; extract the total time spent from data reading to outputting the prediction result of the three-dimensional spatial coordinates of the target during the prediction process; after outputting the prediction result of the three-dimensional spatial coordinates of the target, extract the time spent from issuing the command to adjust the aiming angle and position of the control turntable to aim at the target and stabilize it. The characteristic duration is the sum of the total time consumed, the total duration, and the time consumed.

10. A predictive compensation-based photoelectric aiming system, comprising a memory, a processor, and a computer program stored in the memory and running on the processor, characterized in that, When the processor executes the computer program, it implements the steps of the electro-optical aiming method based on prediction compensation as described in any one of claims 1-9.

Images