A real-time target detection method and system based on motion trajectory prediction and calibration

Abstract

The application provides a real-time target detection method and system based on motion trajectory prediction and calibration. In the application, a multi-dimensional displacement vector is generated through motion measurement data, and a reference frame with the lowest blur in the image sequence is screened. A motion trajectory model is constructed by combining the displacement vector and the reference frame, the current frame is reversely compensated for motion blur, a local clear image is generated, and pixel displacement data is recorded. The clear image is decomposed in the frequency domain, and a blur intensity distribution map is generated in combination with the displacement data to adaptively enhance the high-frequency edges. At the same time, the target trajectory is predicted by fusing the displacement data, and a spatial offset weight matrix is generated to restore the geometric position of the target. Finally, the restored features and the enhanced edges are weighted and fused, the motion calibration coefficient map is used for trajectory consistency calibration, and the optimized feature map is output for target detection. Through dynamic trajectory modeling, blur compensation, edge enhancement and position restoration, the application significantly improves the accuracy of real-time target detection in a motion blur scene.

Description

Technical Field

[0001] This application relates to the field of real-time target detection technology, and in particular to a real-time target detection method and system based on motion trajectory prediction and calibration. Background Technology

[0002] In high-speed motion scenarios such as drone inspection, autonomous driving perception, and high-speed industrial visual inspection, there is often severe relative motion between the camera and the target, resulting in serious motion blur in the captured image sequence. This blur is particularly detrimental to small targets in the image, significantly reducing their edge sharpness and feature saliency, causing loss of target texture information and a decrease in signal-to-noise ratio. Therefore, there is an urgent need for a real-time, robust target detection method that can effectively overcome motion blur interference, especially significantly improving the detectability of small targets in blurred images and avoiding missed detection of critical small targets due to blur. This is crucial for ensuring system security and detection accuracy.

[0003] Currently, a mainstream targeted approach is a joint optimization method based on motion compensation and deep learning. This type of method typically utilizes motion information estimated from sensors or the image sequence itself to construct a motion blur degradation model. Then, before inputting it into the detection network, or embedding it into the network structure, inverse motion blur compensation is performed on blurred image frames to attempt to restore image sharpness. The compensated "sharpened" image or intermediate feature map is then input into a target detector based on a deep convolutional neural network for recognition and localization. Its core idea is to explicitly model and counteract the effects of motion blur, providing higher-quality input for subsequent detection.

[0004] Although methods based on joint optimization of motion compensation and deep learning aim to improve detection by modeling and offsetting blur, they still face significant challenges in small target detection scenarios with high-speed motion: the motion compensation process itself has inherent defects, relying on the estimation of complex high-speed motion information, which is often difficult to be accurate, especially when dealing with the local motion of small targets, resulting in a large error and causing the constructed motion blur degradation model to mismatch with reality; while attempting to restore sharpness, the compensation operation is prone to destroying the already weak high-frequency edges and texture details of small targets, and even introducing smoothing effects or ringing artifacts, which exacerbates the loss of key information; more importantly, this pixel-level rearrangement operation is extremely sensitive to the geometric position of small targets, and even slight compensation errors can cause significant shifts or distortions in the target's position on the feature map, severely damaging its spatial consistency and localization accuracy, making it difficult for the detector to reliably identify, and ultimately failing to effectively solve the problem of missed detection of small targets in high-speed blurred scenes. Summary of the Invention

[0005] This application provides a real-time target detection method and system based on motion trajectory prediction and calibration, which solves the problem that existing motion compensation-based deblurring methods may miss small targets in high-speed scenes due to the loss of high-frequency details and distortion of target geometric position caused by compensation errors.

[0006] Firstly, this application provides a real-time target detection method based on motion trajectory prediction and calibration, including:

[0007] Acquire timestamped image sequences and motion measurement data, generate multidimensional motion displacement vectors based on the motion measurement data, and dynamically select reference image frames with the lowest motion blur from the image sequences based on the multidimensional motion displacement vectors.

[0008] A motion trajectory model is constructed based on the multidimensional motion displacement vector and the reference image frame. Motion blur inverse compensation is performed on the current frame in the image sequence according to the motion trajectory model to generate a locally sharpened image, and the pixel displacement data generated during the compensation process is recorded.

[0009] The localized sharpened image is subjected to frequency domain decomposition processing, the pixel displacement data is retrieved in conjunction to generate a blur intensity distribution map, and the high-frequency edge components separated in the frequency domain are subjected to energy adaptive enhancement processing based on the blur intensity distribution map.

[0010] The target motion trajectory is predicted by fusing the multidimensional motion displacement vector and the pixel displacement data. A spatial offset weight matrix is ​​generated based on the target motion trajectory. The perceptual area of ​​the feature map is dynamically adjusted through the spatial offset weight matrix to restore the geometric position of the target.

[0011] The restored feature map is weighted and fused with the enhanced high-frequency edge components. A motion calibration coefficient map is generated by combining the target motion trajectory. The motion calibration coefficient map is then used to perform trajectory consistency calibration on the fused features to output an optimized feature map for target detection.

[0012] Optionally, the step of fusing the multidimensional motion displacement vector and the pixel displacement data to predict the target motion trajectory, generating a spatial offset weight matrix based on the target motion trajectory, and dynamically adjusting the receptive region of the feature map through the spatial offset weight matrix to achieve target geometric position restoration includes:

[0013] The multidimensional motion displacement vector and the pixel displacement data are superimposed on each other according to time markers to calculate and predict the continuous motion trajectory of the target from the reference image frame to the current frame as the target motion trajectory;

[0014] Based on the magnitude and direction of the change in the target position in the target motion trajectory, a weight value is assigned to each pixel position in the image plane to generate a two-dimensional spatial offset weight matrix.

[0015] The spatial offset weight matrix is ​​applied to each element of the feature map, and the range of the perceptual region for feature extraction is dynamically adjusted according to the assigned weight values, so that the target features are restored to their original geometric positions in the reference image frame.

[0016] Optionally, applying the spatial offset weight matrix to each element of the feature map and dynamically adjusting the perceptual region range of feature extraction according to the weight values, so that the target features are restored to their original geometric positions in the reference image frame, includes:

[0017] The size of the corresponding rectangular sensing region is determined based on the weight value of each element in the spatial offset weight matrix, and the rectangular sampling region is delineated with the element position in the feature map as the center point.

[0018] Within the rectangular sampling area, feature points along the offset direction are preferentially selected based on the position offset direction in the target motion trajectory, and the feature values ​​are sampled according to Gaussian distribution weights.

[0019] Bilinear interpolation is performed on the selected feature points to generate feature values ​​of the corresponding elements in the feature map at their original geometric positions. By performing the above operation on each element of the feature map, the target features are restored to their original geometric positions in the reference image frame.

[0020] Optionally, the step of weightedly fusing the restored feature map with the enhanced high-frequency edge components, generating a motion calibration coefficient map by combining the target motion trajectory, and using the motion calibration coefficient map to perform trajectory consistency calibration on the fused features to output an optimized feature map for target detection includes:

[0021] The restored feature map is assigned a first weight distribution, and the enhanced high-frequency edge components are assigned a second weight distribution. Based on the assigned weight distribution, position-by-position weighted superposition is performed to generate the initial fusion feature.

[0022] Based on the predicted motion direction of the target motion trajectory and the actual motion direction reflected by the initial fusion features, a direction consistency metric is calculated for each image location, and a motion calibration coefficient map is generated based on the direction consistency metric.

[0023] The motion calibration coefficient map is multiplied element-wise with the initial fusion feature to enhance the features in high-consistency regions and suppress the features in low-consistency regions, so as to output a calibrated optimized feature map for target detection.

[0024] Optionally, the step of constructing a motion trajectory model based on the multidimensional motion displacement vector and the reference image frame, performing motion blur inverse compensation on the current frame in the image sequence according to the motion trajectory model to generate a locally sharpened image, and recording the pixel displacement data generated during the compensation process includes:

[0025] Using the reference image frame as the reference time point, the multidimensional motion displacement vectors are arranged in chronological order, and the motion trajectory model from the reference image frame to each time point is constructed by accumulating the displacement components.

[0026] Based on the motion trajectory model, the displacement information of the current frame time point relative to the reference image frame in the image sequence is extracted, and the position offset of each pixel in the current frame relative to the pixel reference position in the reference image frame is derived.

[0027] Based on the position offset, the pixel position of the current frame is pushed to the corresponding position of the reference image frame in the opposite direction of motion, and the missing pixel positions are filled with neighboring values ​​during the pushing process to generate a locally sharpened image.

[0028] During the reverse shifting process, the actual displacement difference between the original position coordinates and the shifted position coordinates of each pixel is measured simultaneously to form pixel displacement data containing two-dimensional offset.

[0029] Optionally, the step of performing frequency domain decomposition processing on the locally sharpened image, simultaneously retrieving the pixel displacement data to generate a blur intensity distribution map, and performing energy adaptive enhancement processing on the high-frequency edge components separated in the frequency domain based on the blur intensity distribution map, includes:

[0030] A spatial-to-frequency domain transformation operation is performed on the locally sharpened image to separate the high-frequency edge components containing edge details and the low-frequency fundamental components of smooth regions.

[0031] The two-dimensional offset in the pixel displacement data is retrieved, and the blur intensity value is calculated based on the total displacement amplitude of each pixel position. The blur intensity values ​​of each pixel position are combined to form a blur intensity distribution map with the same size as the image.

[0032] In the frequency domain, the fuzzy intensity distribution map is aligned with the high-frequency edge component, and the energy intensity value of the high-frequency edge component is proportionally increased according to the fuzzy intensity value at the corresponding position of each frequency point.

[0033] Optionally, the step of acquiring timestamped image sequences and motion measurement data, generating multidimensional motion displacement vectors based on the motion measurement data, and dynamically selecting reference image frames with the lowest motion blur from the image sequence based on the multidimensional motion displacement vectors includes:

[0034] Acquire time-stamped image sequences and motion measurement data containing information on the device’s motion changes in multiple directions;

[0035] Based on the time interval between adjacent images in the image sequence, the motion measurement data is divided into multiple time intervals, and a multidimensional motion displacement vector is formed by combining the cumulative displacement components in each time interval.

[0036] For each image frame in the image sequence, a multidimensional motion displacement vector at the corresponding time point is extracted and the amplitude value of the multidimensional motion displacement vector is determined. The image frame with the smallest amplitude value is selected as the reference image frame by comparing the amplitude values ​​of all image frames.

[0037] Secondly, this application provides a real-time target detection system based on motion trajectory prediction and calibration, comprising:

[0038] The acquisition module is used to acquire image sequences with timestamps and motion measurement data, generate multidimensional motion displacement vectors based on the motion measurement data, and dynamically select reference image frames with the lowest motion blur from the image sequences based on the multidimensional motion displacement vectors.

[0039] The generation module is used to construct a motion trajectory model based on the multidimensional motion displacement vector and the reference image frame, perform motion blur inverse compensation operation on the current frame in the image sequence according to the motion trajectory model, generate a locally sharpened image, and record the pixel displacement data generated during the compensation process.

[0040] The processing module is used to perform frequency domain decomposition processing on the locally sharpened image, retrieve the pixel displacement data to generate a blur intensity distribution map, and perform energy adaptive enhancement processing on the high-frequency edge components separated in the frequency domain based on the blur intensity distribution map.

[0041] The adjustment module is used to fuse the multidimensional motion displacement vector and the pixel displacement data to predict the target motion trajectory, generate a spatial offset weight matrix based on the target motion trajectory, and dynamically adjust the perception area of ​​the feature map through the spatial offset weight matrix to restore the geometric position of the target.

[0042] The output module is used to perform weighted fusion of the restored feature map and the enhanced high-frequency edge components, generate a motion calibration coefficient map by combining the target motion trajectory, and perform trajectory consistency calibration on the fused features using the motion calibration coefficient map to output an optimized feature map for target detection.

[0043] Thirdly, this application provides a computing device, including a processing component and a storage component; the storage component stores one or more computer instructions; the one or more computer instructions are invoked and executed by the processing component to implement a real-time target detection method based on motion trajectory prediction and calibration as described in the first aspect above.

[0044] Fourthly, this application provides a computer storage medium storing a computer program, which, when executed by a computer, implements a real-time target detection method based on motion trajectory prediction and calibration as described in the first aspect.

[0045] In this application example, a timestamped image sequence and motion measurement data are acquired. A multidimensional motion displacement vector is generated based on the motion measurement data, and a reference image frame with the lowest motion blur is dynamically selected from the image sequence based on the multidimensional motion displacement vector. A motion trajectory model is constructed based on the multidimensional motion displacement vector and the reference image frame. A motion blur inverse compensation operation is performed on the current frame in the image sequence according to the motion trajectory model to generate a locally sharpened image, and the pixel displacement data generated during the compensation process is recorded. Frequency domain decomposition processing is performed on the locally sharpened image, and the pixel displacement data is retrieved in conjunction to generate a blur strength... The system generates a fuzzy intensity distribution map and performs energy adaptive enhancement processing on the high-frequency edge components separated in the frequency domain based on the fuzzy intensity distribution map. It then fuses the multi-dimensional motion displacement vector and the pixel displacement data to predict the target motion trajectory. Based on the target motion trajectory, it generates a spatial offset weight matrix and dynamically adjusts the receptive area of ​​the feature map to restore the target's geometric position. Finally, it performs weighted fusion of the restored feature map and the enhanced high-frequency edge components, combines the target motion trajectory to generate a motion calibration coefficient map, and uses the motion calibration coefficient map to perform trajectory consistency calibration on the fused features to output an optimized feature map for target detection.

[0046] The technical solution of this application has the following beneficial effects:

[0047] This application achieves precise motion blur inverse compensation by dynamically selecting low-blurry reference frames and constructing a motion trajectory model, generating a clear image that retains local details. It combines frequency domain decomposition and pixel displacement data to adaptively enhance high-frequency edges, effectively restoring the texture features of small targets lost due to blur. At the same time, it integrates multi-dimensional motion information to predict the target trajectory, uses a spatial offset weight matrix to correct geometric position offset, and finally outputs an optimized feature map after weighted fusion and trajectory consistency calibration. This significantly improves the detection accuracy and robustness of small targets in high-speed motion scenes, fundamentally solving the problem of small target feature annihilation and positioning distortion caused by motion blur.

[0048] Furthermore, by superimposing multidimensional motion displacement vectors and pixel displacement data with time stamps, the continuous motion trajectory of the target from the reference frame to the current frame is predicted. Based on the magnitude and direction of change at each position in this trajectory, a weight value is assigned to each pixel position in the image plane, generating a two-dimensional spatial offset weight matrix. Finally, this weight matrix is ​​applied to each element of the feature map, dynamically adjusting the perceptual area range for feature extraction, thereby restoring the target features to their original geometric position in the reference frame. This scheme, by predicting the continuous motion trajectory of the target and generating a spatial offset weight matrix, can accurately quantify the target position offset caused by high-speed motion. By dynamically guiding the feature extraction process with this matrix, the geometric position of the target features in the feature map is adaptively corrected, effectively overcoming the problem of target position distortion in the feature space caused by motion blur and compensation operations, and significantly improving the localization accuracy and spatial consistency of subsequent target detection.

[0049] These or other aspects of this application will become more apparent in the following description of the embodiments. Attached Figure Description

[0050] To more clearly illustrate the technical solutions in the embodiments of this application or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0051] Figure 1 A flowchart of a real-time target detection method based on motion trajectory prediction and calibration provided in this application is shown;

[0052] Figure 2 A schematic diagram of the structure of a real-time target detection system based on motion trajectory prediction and calibration provided in this application is shown;

[0053] Figure 3 A schematic diagram of the structure of a computing device provided in this application is shown. Detailed Implementation

[0054] To enable those skilled in the art to better understand the present application, the technical solutions in the embodiments of the present application will be clearly and completely described below with reference to the accompanying drawings.

[0055] In some of the processes described in the specification, claims, and accompanying drawings of this application, multiple operations appearing in a specific order are included. However, it should be clearly understood that these operations may not be executed in the order they appear herein, or may be executed in parallel. The operation numbers, such as 101, 102, etc., are merely used to distinguish different operations and do not themselves represent any execution order. Furthermore, these processes may include more or fewer operations, and these operations may be executed sequentially or in parallel. It should be noted that the descriptions such as "first," "second," etc., in this document are used to distinguish different messages, devices, modules, etc., and do not represent a chronological order, nor do they limit "first" and "second" to different types.

[0056] Research shows that existing schemes based on joint optimization of motion compensation and deep learning face severe challenges in real-time target detection in high-speed motion scenes: they rely on estimated motion information for deblurring compensation, which is difficult to accurately model under high-speed and complex motion, and is prone to blurring degradation and model mismatch; more importantly, while attempting to restore sharpness, the compensation operation is very likely to obliterate the already weak high-frequency edges and texture details of small targets, and introduce artifacts due to motion estimation errors or the inherent smoothing effect of the compensation itself; especially critically, such pixel-level operations are extremely sensitive to the geometric position of small targets, and slight compensation deviations can cause significant positional shifts or distortions of the target on the feature map, seriously damaging spatial consistency, and ultimately leading to feature loss and localization failure of small targets in high-speed blurred environments, resulting in serious missed detections.

[0057] To address the aforementioned problems, this invention proposes a real-time target detection method based on motion trajectory prediction and calibration. Its core lies in: dynamically selecting low-blurry reference frames and constructing an accurate motion trajectory model to achieve more reliable motion blur reverse compensation while preserving local details; linking the pixel displacement data generated by compensation with the frequency domain decomposition results to generate a blur intensity distribution map and adaptively enhance high-frequency edges, specifically restoring key textures of small targets; innovatively fusing multi-dimensional motion vectors and pixel displacement data to predict the continuous trajectory of the target, thereby generating a spatial offset weight matrix and dynamically adjusting the perceptual area of ​​the feature map to accurately restore the geometric position of the target; finally, outputting an optimized feature map through weighted fusion and trajectory consistency calibration. This method abandons the approach of simply relying on coarse motion compensation, improving compensation accuracy through trajectory modeling, protecting details through frequency domain enhancement, and ensuring spatial consistency through dynamic position correction, significantly improving the feature recognizability and positioning accuracy of small targets in high-speed motion blur scenes, fundamentally solving the problem of missed detection caused by this.

[0058] The technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, and not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0059] Figure 1 This application provides a flowchart of a real-time target detection method based on motion trajectory prediction and calibration, as shown in the following embodiments. Figure 1 As shown, the method includes:

[0060] 101. Obtain image sequences with timestamps and motion measurement data, generate multidimensional motion displacement vectors based on the motion measurement data, and dynamically select reference image frames with the lowest motion blur from the image sequences based on the multidimensional motion displacement vectors.

[0061] Optionally, step 101 may specifically include the following steps:

[0062] 1011. Acquire time-stamped image sequences and motion measurement data containing information on the device's motion changes in multiple directions;

[0063] 1012. Based on the time interval between adjacent images in the image sequence, the motion measurement data is divided into multiple time intervals, and a multidimensional motion displacement vector is formed by combining the cumulative displacement components in each time interval.

[0064] 1013. For each image frame in the image sequence, extract the multidimensional motion displacement vector at the corresponding time point and determine the amplitude value of the multidimensional motion displacement vector. By comparing the amplitude values ​​corresponding to all image frames, select the image frame with the smallest amplitude value as the reference image frame.

[0065] In the above scheme, the image sequence refers to a continuous visual scene record arranged in strict chronological order, containing precise time stamps for the capture time of each frame, used to capture the transient attitude and positional changes of targets in high-speed moving scenes. Motion measurement data refers to multidimensional motion information of the device collected by inertial sensors or pose estimation systems, including real-time temporal records of physical quantities such as linear acceleration and angular velocity, used to construct a motion physics model strictly synchronized with the image frames. The multidimensional motion displacement vector refers to a displacement quantization index generated by integrating the motion measurement data over a specific time interval, used to objectively quantify the physical motion intensity causing motion blur. The reference image frame refers to a baseline frame dynamically selected from the image sequence based on the criterion of minimizing the amplitude value of the multidimensional motion displacement vector, used to provide a high-fidelity spatial reference for subsequent motion compensation and target detection.

[0066] In this embodiment of the application, firstly, step 1011 involves the camera hardware and timing chip working together to capture continuous image frames of a dynamic scene at a fixed frame rate, and embedding a time stamp accurate to the millisecond level for each frame, such as frame... Marked as t=0ms, frame The sampling rate is t=33ms. Simultaneously, the inertial measurement unit (IMU) mounted on the same device acquires raw motion data of the device in three-dimensional space in real time at a higher sampling rate, including X / Y / Z axis acceleration and angular velocity. The camera and IMU are synchronized via a hardware clock signal to ensure strict alignment of the image timestamp with the time axis of the motion data. For example, when a drone is filming a high-voltage power line, the camera captures the 4th frame image at t=100ms. The inertial measurement unit collected three sets of acceleration data between t=99.5ms and t=100.5ms: , , The unit is m / s², and all data is stored as paired datasets by timestamp.

[0067] Next, in step 1012, based on the time-stamped image sequence and synchronous motion measurement data provided in step 1011, the time intervals of the motion data are divided according to the timestamp difference between adjacent image frames, ensuring that each interval strictly corresponds to the acquisition period between two frames. Then, numerical integration processing is performed on the raw motion data within each time interval, i.e., the three-axis acceleration acquired by the inertial measurement unit: the trapezoidal rule is used to perform a single integration of the acceleration data to calculate the instantaneous velocity, as shown in the following formula: ,in The sampling interval of the inertial measurement unit is used, and the displacement component is obtained by a second integration of the velocity. The calculation formula is as follows: Subsequently, the displacement calculation results along the X, Y, and Z axes within each interval were... Combined into a multidimensional motion displacement vector Finally, a displacement vector sequence corresponding one-to-one with the image frame intervals is generated, providing a quantitative basis for subsequent fuzzy evaluation. Taking the high-speed inspection drone's image of a power transmission line as an example, its camera captures the corresponding frame intervals of the image at 30fps. Simultaneously, the IMU acquires acceleration data at 100Hz. Processing frame t=0ms At t=33ms When processing motion data: the time interval is divided into 0-33ms, and three sets of inertial measurement unit samples are extracted within this time interval at t=10ms, t=20ms, and t=30ms; the X-axis acceleration sequence is processed. Perform integration: calculate speed by integrating once. Find the displacement using a second integral. Similarly, the displacements along the Y and Z axes are calculated as follows: m, m; Combined to generate displacement vector , characterization to The overall intensity of movement of equipment during a given time period.

[0068] Finally, based on the set of multidimensional motion displacement vectors generated in step 1012 that correspond one-to-one with the image frame intervals, step 1013 is used. For each displacement vector Calculate its amplitude value This value physically represents the overall motion intensity of the device within the k-th frame interval; subsequently, the system iterates through and compares the amplitude values ​​across all frame intervals. The global minimum amplitude value is located by sorting or minimum value search algorithm. Ultimately, The corresponding original image frame is as follows: The selected reference image frame is essentially a visual sample captured at the moment when the device moves the smoothest, with the slightest motion blur distortion, providing a high-fidelity spatial reference for subsequent processing.

[0069] In practical applications, such as intelligent monitoring on highways, cameras mounted on gantry frames capture traffic flow at 25fps intervals. First, obtain the time-stamped image sequence, frames. Timestamp t=0ms For t=40ms, Triaxial acceleration data from the onboard inertial measurement unit (IMU) was collected synchronously at t=80ms, with a sampling rate of 100Hz and a group size every 10ms; then processed. arrive Motion data for the interval: Extract 4 sets of X-axis accelerations for that time period. m / s², velocity calculated using trapezoidal integral Then calculate the displacement. m; similarly, calculate the Y-axis and Z-axis to get m、 m, generating vector ; then process arrive Vector obtained in the interval 40-80ms Finally, calculate the magnitude of each vector: m, m, because Select the exercise with the lowest intensity The reference image frame is used for subsequent vehicle detection.

[0070] The aforementioned overall scheme 101 simultaneously acquires timestamped image sequences and motion measurement data, generates a multi-dimensional motion displacement vector based on sensor data integration, and dynamically selects the reference frame with the least motion blur according to the vector amplitude. This process achieves objective quantification of device motion intensity, ensuring automatic selection of the clearest visual reference frame, providing a high-fidelity spatial reference for subsequent motion compensation and target detection, effectively improving the initial quality of image processing in high-speed scenes, overcoming the subjectivity and inefficiency of manual selection, and reducing the interference of motion blur on detection accuracy from the source.

[0071] 102. Construct a motion trajectory model based on the multidimensional motion displacement vector and the reference image frame, perform motion blur inverse compensation operation on the current frame in the image sequence according to the motion trajectory model, generate a locally sharpened image, and record the pixel displacement data generated during the compensation process;

[0072] Optionally, step 102 may specifically include the following steps:

[0073] 1021. Using the reference image frame as the reference time point, arrange the multidimensional motion displacement vectors in chronological order, and construct a motion trajectory model from the reference image frame to each time point by accumulating the displacement components;

[0074] 1022. Based on the motion trajectory model, extract the displacement information of the current frame time point in the image sequence relative to the reference image frame, and deduce the position offset of each pixel in the current frame relative to the pixel reference position in the reference image frame;

[0075] 1023. Based on the position offset, the pixel position of the current frame is moved in the opposite direction of motion to the corresponding position of the reference image frame, and the missing pixel positions are filled with neighboring values ​​during the moving process to generate a locally sharpened image.

[0076] 1024. During the reverse shifting process, the actual displacement difference between the original position coordinates and the shifted position coordinates of each pixel is measured simultaneously to form pixel displacement data containing two-dimensional directional offset.

[0077] In the above scheme, the motion trajectory model refers to the spatial position evolution path of the device constructed by accumulating multi-dimensional motion displacement vectors. Essentially, it reflects the physical motion trajectory of the image acquisition device in the time dimension and can be used to derive pixel-level spatial position offsets. Position offset refers to the two-dimensional coordinate deviation of a pixel in the current frame relative to its corresponding position in the reference frame. Essentially, it characterizes the pixel-level positional distortion of the target object caused by device motion and can be used to guide the reverse compensation operation for motion blur. The locally sharpened image refers to the visual optimization result generated by performing reverse motion pixel shifting on the current frame. It includes missing areas filled with neighbor values ​​and the restored intrinsic contour of the object, and can be used to improve the saliency of high-frequency edge features. Pixel displacement data refers to a structured mapping table recording the changes in pixel coordinates during the reverse compensation process. It includes the actual two-dimensional displacement difference between the original coordinates and the compensated coordinates of each pixel, reflecting the amount of correction of the pixel's spatial position by motion compensation, and can be used for subsequent frequency domain analysis and trajectory consistency calibration.

[0078] In this embodiment, firstly, step 1021 uses the timestamp of a reference image frame as the base time origin, and arranges the multidimensional motion displacement vectors obtained after the reference frame in the temporal order of the image sequence. For example, the reference frame... arrive vector , arrive vector The components in each motion direction are time-sequentially superimposed using a displacement accumulator algorithm: for the target frame... (k>ref), the coordinates of its trajectory points This involves accumulating the corresponding components of all displacement vectors from the reference frame to the target frame to generate a motion trajectory model describing the continuous spatial motion path of the device from the reference time to the target time. For example, in a UAV power line inspection scenario, the reference frame is selected as... , , ,calculate Frame trajectory points: .

[0079] Next, based on the motion trajectory model generated in step 1021, step 1022 is used. Extract its two-dimensional planar components First, it characterizes the cumulative physical displacement of the device in the horizontal and vertical directions of the image; second, it calls the camera calibration parameters and calculates the pixel offset through scalar mapping. , ,in This is the pixel / meter scaling factor in the X direction. This is the pixel / meter scaling factor in the Y direction; finally, the result is rounded to the nearest integer to generate the two-dimensional position offset of the current frame relative to the reference frame. .

[0080] Then, the position offset calculated in step 1023 is used as a basis for step 1022. The original coordinates of each pixel in the current frame Perform a coordinate transformation in the opposite direction of motion: calculate the target position. Next, the original pixel values ​​containing RGB color information are copied to the new target location; then, empty positions in the image plane are detected, i.e., coordinates to which no original pixel is mapped, and the nearest neighbor interpolation algorithm is used to fill them: searching the surrounding area with the empty position as the center. Calculate the effective pixels within the neighborhood. Disadvantages of empty space Euclidean distance The system selects the effective pixel value with the smallest distance to fill the gap; finally, it generates a locally sharpened image that eliminates the blurring and distortion caused by the device's motion path.

[0081] Finally, in step 1024, while generating a locally sharpened image by pixel shifting in step 1023, the coordinate transformation process of each pixel is traversed: the original position coordinates are recorded. Coordinates of the position after displacement Next, calculate the actual displacement difference in the two-dimensional direction: the horizontal offset is... The vertical offset is Finally, the unique coordinates of each pixel are identified. its displacement difference A key-value pair mapping relationship is constructed to form a structured pixel displacement dataset. This process is achieved through parallel stream processing technology to achieve efficient real-time recording and ensure that the data is strictly synchronized with the image generation.

[0082] In practical applications, in gantry monitoring systems, when t=40ms is selected... After using the reference image frame, process the frame at t=80ms. First, accumulate. to Displacement vector Generate three-dimensional trajectory points Next, extract the two-dimensional components. Combined with camera calibration parameters , Calculate pixel offset: , Output Then, a reverse shift is performed on the license plate character pixels (150, 200): new position After copying the pixel value, a gap was detected at the adjacent position (147, 202). The distance between this gap and the valid point (148, 201) was calculated. The distance between (147,200) Select the nearest pixel (148, 201) to fill; simultaneously record the pixel displacement data: from the original position (150, 200) to the new position (148, 201), and the difference. , Store as Finally, locally sharpened license plate images are generated and displacement datasets are output.

[0083] The overall scheme described above (102) constructs an accurate motion trajectory model by accumulating multi-dimensional motion displacement vectors, converting the physical displacement of the device into pixel-level positional offsets. Based on these offsets, it performs pixel shifting and neighbor filling in the opposite direction of motion on the current frame, generating a sharpened image with enhanced local details. Simultaneously, it records the pixel displacement difference data during the compensation process. This process achieves blur correction driven by the device's true motion path, significantly improving the contour sharpness and spatial consistency of key features of high-speed targets, laying a precise motion feature foundation for subsequent frequency domain processing, while avoiding the cumulative error problem of traditional optical flow estimation.

[0084] 103. Perform frequency domain decomposition processing on the locally sharpened image, retrieve the pixel displacement data to generate a blur intensity distribution map, and perform energy adaptive enhancement processing on the high-frequency edge components separated in the frequency domain based on the blur intensity distribution map.

[0085] Optionally, step 103 may specifically include the following steps:

[0086] 1031. Perform a spatial-to-frequency domain transformation operation on the locally sharpened image to separate the high-frequency edge components containing edge details and the low-frequency fundamental components of the smooth regions;

[0087] 1032. Retrieve the two-dimensional offset from the pixel displacement data, calculate the blur intensity value based on the total displacement amplitude of each pixel position, and combine the blur intensity values ​​of each pixel position to form a blur intensity distribution map with the same size as the image.

[0088] 1033. Align the fuzzy intensity distribution map with the high-frequency edge component in the frequency domain, and proportionally enhance the energy intensity value of the high-frequency edge component according to the fuzzy intensity value at the corresponding position of each frequency point.

[0089] In the above scheme, high-frequency edge components refer to the image detail signals obtained through frequency domain decomposition, containing amplitude and phase information of high-frequency regions in the spatial frequency spectrum. Essentially, they reflect rapidly changing visual features such as object contours and textures, and can be used to specifically enhance the saliency of details in motion-blurred regions. The blur intensity distribution map refers to a two-dimensional quantization matrix generated based on pixel displacement data, containing a normalized blur intensity value calculated for each pixel position based on the displacement amplitude. Essentially, it characterizes the degree of motion distortion differences in local image regions and can be used to guide adaptive adjustment of frequency domain energy. Energy adaptive enhancement processing refers to a gain control mechanism that links the blur intensity distribution map and high-frequency components. It includes a proportional coefficient that dynamically increases the amplitude of the high-frequency signal according to the local blur intensity value, essentially achieving directional compensation of edge energy in high-distortion regions, and can be used to restore the feature intensity of small targets attenuated by motion blur.

[0090] In this embodiment, a two-dimensional fast Fourier transform is first performed on the locally sharpened image in step 1031: the Cooley-Tukey algorithm is used to convert the spatial domain pixel matrix into a frequency domain complex matrix. Where W and H are the image width and height, respectively; the amplitude spectrum is calculated using the formula. Simultaneously, the phase spectrum is calculated using the formula. Finally, set the cutoff frequency. Components are separated by filtering based on frequency radius: for each frequency point If its frequency radius Then it is classified as a low-frequency basic component. If it corresponds to a smooth region in the image, it is otherwise classified as a high-frequency edge component. This corresponds to edge texture details. For example, when processing a 512×512 insulator image, the calculation... ; Calculate for frequency point (30,10) Therefore, its complex number F(30,10)=0.4+j0.3 is included in... ; and for frequency point (10,5) calculation Let its F(10,5)=1.2-j0.5 be included in This enables precise separation between the basic structure and detailed features.

[0091] Next, step 1032 retrieves the pixel displacement dataset recorded in step 1024, which contains the pixel displacement data for each pixel. Value; iterate through each position in the image. Calculate the total displacement amplitude of the pixel. This value physically represents the actual movement distance of a pixel during the compensation process; then, the entire image is scanned. The value is determined using a maximum value search algorithm to find the global maximum value. Then, normalization is performed on each pixel. The displacement amplitude is converted into a blur intensity value in the range [0,1], where 0 represents no blur and 1 represents the most severe blur. Finally, all values ​​are combined according to the original image spatial coordinates. This value generates a two-dimensional blur intensity distribution matrix that is exactly the same size as the input image. For example, for pixel A(100,50): displacement data , The total displacement amplitude of the pixel is calculated. Pixel B(150,80): Displacement data , The total displacement amplitude of the pixel is calculated. Full image scan ,calculate , Generate distribution map: store value 0.5 for position (100,50) and value 1.0 for position (150,80).

[0092] Finally, step 1033 establishes the coordinate mapping relationship between the spatial domain and the frequency domain: the fuzzy intensity distribution map is then mapped. Each spatial location Mapping to frequency domain coordinates through the correspondence of Fourier transform. This forms a frequency-domain aligned fuzzy intensity matrix. Secondly, traverse the high-frequency edge components. For each frequency point, extract the blur intensity value at its corresponding location. Calculate the adaptive gain coefficient ,in The preset enhancement factor is 1.2 (default value); subsequently, amplitude scaling is performed on the complex high-frequency signal: the real part is enhanced to... The imaginary part is enhanced to The phase remains unchanged, generating enhanced high-frequency components. Ultimately, low-frequency components are retained. Without modification, spatial adaptive adjustment of frequency domain energy is achieved. For example, when processing the frequency point (40,20) of the insulator image, its corresponding spatial position is determined to be (200,300). By referring to the fuzzy intensity distribution map, the fuzzy intensity value B=0.7 is obtained, and the gain G=1+1.2×0.7=1.84 is calculated. Based on this, the original high-frequency signal 0.5+j0.6 is scaled to 0.92+j1.104, achieving a precise 84% increase in crack edge energy.

[0093] In practical applications, after acquiring a locally sharpened insulator image of 512×512 pixels, frequency domain decomposition is performed to calculate the cutoff frequency. Perform a two-dimensional fast Fourier transform on the image to generate a frequency domain matrix. Regarding coordinates Calculate the frequency radius at the given frequency point. Since 44.72 > 25.6, it is classified as a high-frequency edge component. Record complex numbers Next, the spatial location corresponding to this frequency point is determined through coordinate mapping. Retrieve displacement data Calculate the displacement amplitude Pixels, maximum displacement obtained by scanning the entire image. Pixels, calculate normalized blur intensity Finally, take the preset coefficient. Calculate gain Amplitude scaling is applied to the high-frequency signal: the real part is enhanced by 0.920, and the imaginary part is enhanced by 1.104, resulting in the enhanced output signal. This increases the energy at the edge of the crack in the porcelain plate at that location by 84.0%.

[0094] The overall scheme described above (103) separates high-frequency components carrying edge details by performing frequency domain decomposition on locally sharpened images; it then generates a blur intensity distribution map by linking pixel displacement data to quantify the motion distortion degree of each region of the image; based on this distribution map, it performs energy adaptive enhancement on the high-frequency components, achieving directional enhancement of edge features in highly blurred regions and weak intervention to suppress noise in low-blurred regions. This method significantly improves the contour sharpness and texture saliency of small targets in motion-blurred scenes, while avoiding noise amplification in flat regions, providing a high signal-to-noise ratio visual feature foundation for subsequent detection.

[0095] 104. The multidimensional motion displacement vector and the pixel displacement data are fused to predict the target motion trajectory. A spatial offset weight matrix is ​​generated based on the target motion trajectory. The perceptual area of ​​the feature map is dynamically adjusted through the spatial offset weight matrix to restore the geometric position of the target.

[0096] Optionally, step 104 may specifically include the following steps:

[0097] 1041. The multidimensional motion displacement vector and the pixel displacement data are superimposed and calculated according to the time stamp to predict the continuous motion trajectory of the target from the reference image frame to the current frame as the target motion trajectory;

[0098] 1042. Based on the magnitude and direction of the change in the target position in the target motion trajectory, assign weight values ​​to each pixel position in the image plane to generate a two-dimensional spatial offset weight matrix;

[0099] 1043. Apply the spatial offset weight matrix to each element of the feature map, and dynamically adjust the perceptual region range of feature extraction according to the assigned weight values, so that the target features are restored to their original geometric positions in the reference image frame.

[0100] Step 1043 may specifically include the following process: determining the size of the corresponding rectangular sensing region based on the weight value of each element in the spatial offset weight matrix, and delineating a rectangular sampling region with the element position in the feature map as the center point; within the rectangular sampling region, prioritizing the selection of feature points in the offset direction based on the position offset direction in the target motion trajectory, and sampling the feature values ​​according to the Gaussian distribution weight; performing bilinear interpolation calculation on the selected feature points to generate the feature values ​​of the corresponding elements in the feature map at their original geometric positions, and by performing the above operation on each element of the feature map, restoring the target features to their original geometric positions in the reference image frame.

[0101] In the above scheme, the target motion trajectory refers to a continuous spatial path that fuses device displacement and pixel displacement data. It includes a superimposed sequence of displacement vectors from the reference frame to the current frame, essentially reflecting the absolute motion trend of the target in the physical coordinate system, and can be used to predict the precise positional evolution of high-speed targets. The spatial offset weight matrix refers to a two-dimensional coefficient distribution generated based on trajectory direction consistency, containing the product of the cosine of the direction angle at each pixel position and the normalized value of the displacement amplitude. It essentially quantifies the spatial correlation strength between feature points and the motion trajectory, and can be used to dynamically control the size of the sensing area and sampling priority. The rectangular sensing area refers to a sampling range dynamically defined around the feature points, which can be used to focus on key features in the target's motion direction. Gaussian distribution weighted sampling refers to a feature selection mechanism that assigns priority based on distance within the sensing area, including a weighting function, which can be used to increase the contribution of trajectory direction features.

[0102] In this embodiment of the application, the multidimensional motion displacement vector is first aligned by timestamp in step 1041. With pixel displacement data Pixel displacement is achieved through camera calibration coefficients The conversion to physical displacement is as follows: Secondly, perform vector superposition calculation: from the reference frame Begin by accumulating the device displacement and the transformed pixel displacement vector frame by frame to generate a sequence of absolute motion trajectory points of the target in physical space: The final output is from Continuous trajectory to the current frame .For example, Equipment displacement Pixel displacement , obtained through calculation ; Equipment displacement Pixel displacement , obtained through calculation ; represents the insulator target moving from reference frame (0,0) to (0.046m,0.011m).

[0103] Next, based on the target motion trajectory generated in step 1041, step 1042 calculates the total direction vector from the reference frame to the current frame. Determine the coordinates of the center point of the trajectory. ; Traverse each pixel position of the image Generate position vector Calculate the position vector With the total direction vector The cosine of the included angle; simultaneously calculate the normalized value of the trajectory amplitude. Then synthesize spatial weight values. Finally, combine all positions. Spatial weight value The value generates a two-dimensional spatial offset weight matrix of the same size as the input image, where the weight values ​​range from [0,1], and the high weight regions are concentrated in the direction of the motion trajectory.

[0104] Finally, step 1043 is performed based on each position in the spatial offset weight matrix. The weight value W is used to calculate the size of the rectangular sensing region. And round down, for example, correspond Pixel; based on the element position in the feature map Delineate the center A rectangular sampling area is defined; then, within this area, priority is given to filtering samples that align with the direction of the target's motion trajectory. Feature points with an included angle less than 30°; weighting the selected points according to a Gaussian distribution. Where d is the Euclidean distance from the feature point to the center. Finally, select the four closest feature points after weighting. Perform bilinear interpolation to generate feature points The target location is restored by iterating over all elements of the feature map using the feature values ​​at the original geometric location. For example, the feature point (100, 100) has a weight W = 0.6, and is randomly defined. Region; Filter Find the point within ±30° of the direction, and obtain the included angle (102, 101) of 26.6°; calculate... , , ; using weighted points Interpolating at four points will correct the offset license plate character features from (102, 101) back to the original position (100, 100).

[0105] In practical applications, in a vehicle detection scenario, the reference frame is selected as t=0ms. Process the frame at t=120ms. : Obtain the multidimensional motion displacement vector sequence of the device Corresponding to [0.1, 0.05] Corresponding to [0.2, -0.1], Corresponding to [0.15, 0.03] meters, and pixel displacement data [10, -5], [20, 10], [15, -2] pixels; take the camera calibration coefficients. pixels / meter Pixels / meter, convert pixel displacement to physical displacement [0.1, -0.05], [0.2, 0.1], [0.15, -0.02] meters; perform superposition calculation to obtain: rice, rice, Meters; then perform weight matrix generation: calculate the total direction vector. Meters, generate a position vector for the license plate feature point (400, 240). Calculate the position vector With direction vector The cosine of the included angle is 0.999, the largest in the entire image. Meters, A=1, weight W=1×0.999≈1; finally, take W=1 to calculate the perceptual area S=2×(1+8×1)=18 pixels, delineate an 18×18 area centered at (400,240), and filter. Points within ±30° of the direction are (410, 241). Calculate... , Gaussian distribution weighting The license plate features shifted to (410,241) are corrected back to their original position (400,240) using weighted bilinear interpolation.

[0106] The overall scheme described above (104) constructs the target's physical trajectory by fusing multi-dimensional motion displacement and pixel displacement data from the device, generating a spatial offset weight matrix to quantify the position correction intensity. Based on the weight values, it dynamically adjusts the perceptual area range extracted from features, performing Gaussian weighted sampling under trajectory direction constraints. Combined with bilinear interpolation, it achieves sub-pixel-level geometric position restoration. This method overcomes the limitations of traditional fixed sampling frames, accurately corrects feature offsets caused by motion blur in high-speed moving targets, significantly improves the spatial consistency of small target detection, and ensures that key components return to their original positions in the reference frame.

[0107] 105. The restored feature map is weighted and fused with the enhanced high-frequency edge components, and a motion calibration coefficient map is generated by combining the target motion trajectory. The motion calibration coefficient map is then used to perform trajectory consistency calibration on the fused features to output an optimized feature map for target detection.

[0108] Optionally, step 105 may specifically include the following steps:

[0109] 1051. Assign a first weight distribution to the restored feature map, assign a second weight distribution to the enhanced high-frequency edge components, and perform position-by-position weighted superposition based on the assigned weight distribution to generate the initial fusion feature;

[0110] 1052. Based on the predicted motion direction of the target motion trajectory and the actual motion direction reflected by the initial fusion features, calculate the orientation consistency metric for each image position, and generate a motion calibration coefficient map based on the orientation consistency metric;

[0111] 1053. Perform element-wise multiplication on the motion calibration coefficient map and the initial fusion feature to enhance the features in high consistency regions and suppress the features in low consistency regions, so as to output a calibrated optimized feature map for target detection.

[0112] In the above scheme, the first weight distribution refers to the spatial uniform coefficient matrix allocated to the geometric restoration feature map, which can be used to balance the fusion ratio of geometric features and edge features. The second weight distribution refers to the spatial uniform coefficient matrix allocated to enhance high-frequency edge components, which can be used to balance the fusion ratio of geometric stability and texture sharpness. The motion calibration coefficient map is a two-dimensional matrix generated based on the cosine of the angle between the predicted trajectory direction and the actual feature motion direction. It essentially quantifies the spatiotemporal matching degree between feature motion and physical trajectory, and can be used to distinguish between high-confidence regions and motion mismatch regions. Trajectory consistency calibration refers to the operation of multiplying the calibration coefficient map element-wise with the fused features. It includes feature enhancement mechanisms for high-consistency regions and feature suppression mechanisms for low-consistency regions, and can be used to improve the robustness against migration in high-speed target detection.

[0113] In this embodiment of the application, the geometric restoration feature map containing the target features after position correction is first loaded through step 1051. Enhanced image corresponding to enhanced high-frequency edge components Secondly, geometric reconstruction feature maps. Assign first weight To emphasize geometric positional stability, for Assigning a second weight To preserve edge detail saliency; finally, perform element-wise weighted fusion calculation: The process generates an initial fused feature map that simultaneously contains both location stability and edge saliency, and this process is achieved through parallel matrix operations for efficient processing.

[0114] Next, based on the generated initial fused feature map in step 1052, the predicted motion direction vector is extracted from the target motion trajectory. Secondly, the Lucas-Kanade optical flow estimation algorithm is executed on the fused feature map: the spatial gradient is calculated in 3×3 window units. , and time gradient The optical flow equation is constructed to solve for the actual motion direction field F; then the predicted direction at each position is calculated. The cosine of the angle between the feature map and the actual direction F is used to obtain the direction consistency metric C. Finally, the C value is mapped to a motion calibration coefficient map M of the same size as the feature map, where a C value closer to 1 indicates higher trajectory consistency. For example, predicting the direction of motion... Given a horizontal vector to the right of 5,0, the actual optical flow direction F is 4.8,0.3; calculating the dot product 5 multiplied by 4.8 plus 0 multiplied by 0.3 gives 24; The modulus length is 5, and the square root of the square of the modulus length F is 4.8 squared plus 0.3 squared is approximately 4.81; the directional consistency C is equal to 24 divided by 5 multiplied by 4.81, approximately 0.998; the generated calibration coefficient map M is stored at the corresponding position with 0.998, indicating that the deviation in the motion direction is only about 3 degrees, which belongs to the highly consistent region.

[0115] Finally, in step 1053, the motion calibration coefficient map and the initial fused feature map are spatially aligned to ensure accurate matching of coordinates at each location. An element-wise multiplication operation is performed, multiplying the fused feature value by the corresponding calibration coefficient value to obtain the optimized feature value. This operation essentially achieves trajectory consistency-driven feature modulation: when the calibration coefficient is close to 1, it indicates that the predicted motion direction is consistent with the actual height, and the feature value is fully preserved or even enhanced; when the calibration coefficient is close to 0, it indicates that the motion direction deviates significantly, and the feature value is significantly suppressed. Finally, the optimized feature map is output to the target detection network, making the target features in high-consistency areas more prominent and weakening the interference features in low-consistency areas. For example, at the license plate feature location, the initial fused feature value is 0.86, and the motion calibration coefficient is 0.998. Multiplying them yields an optimized feature value of 0.858, achieving a feature retention rate of 99.8%. At the background tree location, the initial feature value is 0.92, and the calibration coefficient is 0.4. The calculated optimized value is 0.92 multiplied by 0.4, which equals 0.368, indicating that the feature is suppressed by 60%, effectively reducing the risk of false detection.

[0116] In practical applications, when processing license plate detection in the current frame, feature fusion is first performed: the geometric restoration feature map is loaded, and the corrected license plate feature value of 0.7 is obtained at position (400, 240); the enhanced image corresponding to the enhanced high-frequency edge components is loaded, and the edge enhancement feature value of 1.1 is obtained at the same position; the first weight of 0.6 is assigned to the geometric features, and the second weight of 0.4 is assigned to the edge features; the weighted summation yields the initial fusion feature value of 0.6 × 0.7 + 0.4 × 1.1 = 0.86; then, calibration map generation is performed: the predicted motion direction [5, 0] is extracted from the trajectory in step 104; optical flow estimation is performed on the fusion features to obtain the actual motion direction [4.8, 0.3]; the dot product is calculated. ; Modulus , Consistency of direction The motion calibration coefficient map M(400,240)=0.998 is generated. Finally, the fused feature value 0.86 is multiplied by the calibration coefficient 0.998 to obtain the optimized feature value 0.86×0.998=0.858, which is output to the detection network. This makes the license plate feature retention rate at this position 99.8%, while the background tree position is suppressed by 60% due to inconsistent orientation C=0.4, feature value 0.92×0.4=0.368, which significantly improves the detection accuracy.

[0117] The overall scheme described above (105) generates initial features that balance positional stability and detail saliency by weighted fusion of geometric restoration features and edge enhancement images. Based on the consistency between the predicted motion direction and the actual feature optical flow direction, a motion calibration coefficient map is constructed. This coefficient map is then used to perform trajectory-driven modulation on the fused features, enhancing the target response in highly consistent regions and suppressing interfering features in motion-mismatched regions. This method significantly improves the robustness of high-speed moving target detection, ensures accurate identification of trajectory-consistent regions, reduces the risk of false background detection, and optimizes the quality of input features to the detection network.

[0118] Figure 2 This application provides a schematic diagram of the structure of a real-time target detection system based on motion trajectory prediction and calibration, as shown in the embodiment. Figure 2 As shown, the system includes:

[0119] The acquisition module 21 is used to acquire image sequences with timestamps and motion measurement data, generate multidimensional motion displacement vectors based on the motion measurement data, and dynamically select reference image frames with the lowest motion blur from the image sequences based on the multidimensional motion displacement vectors.

[0120] The generation module 22 is used to construct a motion trajectory model based on the multidimensional motion displacement vector and the reference image frame, perform motion blur inverse compensation operation on the current frame in the image sequence according to the motion trajectory model, generate a locally sharpened image, and record the pixel displacement data generated during the compensation process.

[0121] Processing module 23 is used to perform frequency domain decomposition processing on the locally sharpened image, retrieve the pixel displacement data to generate a blur intensity distribution map, and perform energy adaptive enhancement processing on the high-frequency edge components separated in the frequency domain based on the blur intensity distribution map.

[0122] Adjustment module 24 is used to fuse the multidimensional motion displacement vector and the pixel displacement data to predict the target motion trajectory, generate a spatial offset weight matrix according to the target motion trajectory, and dynamically adjust the perception area of ​​the feature map through the spatial offset weight matrix to restore the geometric position of the target.

[0123] The output module 25 is used to perform weighted fusion of the restored feature map and the enhanced high-frequency edge components, generate a motion calibration coefficient map by combining the target motion trajectory, and perform trajectory consistency calibration on the fused features using the motion calibration coefficient map to output an optimized feature map for target detection.

[0124] Figure 2 The aforementioned real-time target detection system based on motion trajectory prediction and calibration can perform... Figure 1 The implementation principle and technical effects of the real-time target detection method based on motion trajectory prediction and calibration described in the illustrated embodiment will not be repeated here. The specific methods by which each module and unit performs its operations in the real-time target detection system based on motion trajectory prediction and calibration described in the above embodiments have been described in detail in the embodiments related to this method, and will not be elaborated upon here.

[0125] In one possible design, Figure 2 The real-time target detection system based on motion trajectory prediction and calibration shown in the embodiment can be implemented as a computing device, such as... Figure 3 As shown, the computing device may include a storage component 31 and a processing component 32;

[0126] The storage component 31 stores one or more computer instructions, wherein the one or more computer instructions are invoked and executed by the processing component 32.

[0127] The processing component 32 is used for the above Figure 1 The embodiment describes a real-time target detection method based on motion trajectory prediction and calibration.

[0128] The processing component 32 may include one or more processors to execute computer instructions to complete all or part of the steps in the above-described method. Alternatively, the processing component may be implemented as one or more application-specific integrated circuits (ASICs), digital signal processors (DSPs), digital signal processing devices (DSPDs), programmable logic devices (PLDs), field-programmable gate arrays (FPGAs), controllers, microcontrollers, microprocessors, or other electronic components to perform the above-described method.

[0129] Storage component 31 is configured to store various types of data to support operations at the terminal. The storage component can be implemented by any type of volatile or non-volatile storage device or a combination thereof, such as static random access memory (SRAM), electrically erasable programmable read-only memory (EEPROM), erasable programmable read-only memory (EPROM), programmable read-only memory (PROM), read-only memory (ROM), magnetic storage, flash memory, magnetic disk, or optical disk.

[0130] Of course, computing devices may also include other components, such as input / output interfaces, display components, communication components, etc.

[0131] Input / output interfaces provide interfaces between processing components and peripheral interface modules, which can be output devices, input devices, etc.

[0132] The communication components are configured to facilitate wired or wireless communication between computing devices and other devices.

[0133] The computing device can be a physical device or an elastic computing host provided by a cloud computing platform. In this case, the computing device can refer to a cloud server, and the aforementioned processing components, storage components, etc., can be basic server resources rented or purchased from the cloud computing platform.

[0134] This application also provides a computer storage medium storing a computer program, which, when executed by a computer, can perform the above-described functions. Figure 1 The embodiment shown is a real-time target detection method based on motion trajectory prediction and calibration.

[0135] Those skilled in the art will clearly understand that, for the sake of convenience and brevity, the specific working processes of the systems, devices, and units described above can be referred to the corresponding processes in the foregoing method embodiments, and will not be repeated here.

[0136] The device embodiments described above are merely illustrative. The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the modules can be selected to achieve the purpose of this embodiment according to actual needs. Those skilled in the art can understand and implement this without any creative effort.

[0137] Through the above description of the embodiments, those skilled in the art can clearly understand that each embodiment can be implemented by means of software plus necessary general-purpose hardware platforms, and of course, it can also be implemented by hardware. Based on this understanding, the above technical solutions, in essence or the part that contributes to the prior art, can be embodied in the form of a software product. This computer software product can be stored in a computer-readable storage medium, such as ROM / RAM, magnetic disk, optical disk, etc., and includes several instructions to cause a computer device (which may be a personal computer, server, etc.) to execute the methods described in the various embodiments or some parts of the embodiments.

[0138] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of this application, and are not intended to limit them. Although this application has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features. Such modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of this application.

Claims

1. A real-time target detection method based on motion trajectory prediction and calibration, characterized in that, include: Acquire timestamped image sequences and motion measurement data, generate multidimensional motion displacement vectors based on the motion measurement data, and dynamically select reference image frames with the lowest motion blur from the image sequences based on the multidimensional motion displacement vectors. A motion trajectory model is constructed based on the multidimensional motion displacement vector and the reference image frame. Motion blur inverse compensation is performed on the current frame in the image sequence according to the motion trajectory model to generate a locally sharpened image, and the pixel displacement data generated during the compensation process is recorded. The localized sharpened image is subjected to frequency domain decomposition processing, the pixel displacement data is retrieved in conjunction to generate a blur intensity distribution map, and the high-frequency edge components separated in the frequency domain are subjected to energy adaptive enhancement processing based on the blur intensity distribution map. The target motion trajectory is predicted by fusing the multidimensional motion displacement vector and the pixel displacement data. A spatial offset weight matrix is ​​generated based on the target motion trajectory. The perceptual area of ​​the feature map is dynamically adjusted through the spatial offset weight matrix to restore the geometric position of the target. The restored feature map is weighted and fused with the enhanced high-frequency edge components, and a motion calibration coefficient map is generated by combining the target motion trajectory. The motion calibration coefficient map is then used to perform trajectory consistency calibration on the fused features to output an optimized feature map for target detection. The process involves constructing a motion trajectory model based on the multidimensional motion displacement vector and the reference image frame, performing motion blur inverse compensation on the current frame of the image sequence according to the motion trajectory model, generating a locally sharpened image, and recording the pixel displacement data generated during the compensation process, including: Using the reference image frame as the reference time point, the multidimensional motion displacement vectors are arranged in chronological order, and the motion trajectory model from the reference image frame to each time point is constructed by accumulating the displacement components. Based on the motion trajectory model, the displacement information of the current frame time point relative to the reference image frame in the image sequence is extracted, and the position offset of each pixel in the current frame relative to the pixel reference position in the reference image frame is derived. Based on the position offset, the pixel position of the current frame is pushed to the corresponding position of the reference image frame in the opposite direction of motion, and the missing pixel positions are filled with neighboring values ​​during the pushing process to generate a locally sharpened image. During the reverse shifting process, the actual displacement difference between the original position coordinates and the shifted position coordinates of each pixel is measured simultaneously to form pixel displacement data containing two-dimensional offset.

2. The method according to claim 1, characterized in that, The process of fusing the multidimensional motion displacement vector and the pixel displacement data to predict the target motion trajectory, generating a spatial offset weight matrix based on the target motion trajectory, and dynamically adjusting the receptive region of the feature map using the spatial offset weight matrix to restore the target's geometric position includes: The multidimensional motion displacement vector and the pixel displacement data are superimposed on each other according to time markers to calculate and predict the continuous motion trajectory of the target from the reference image frame to the current frame as the target motion trajectory; Based on the magnitude and direction of the change in the target position in the target motion trajectory, a weight value is assigned to each pixel position in the image plane to generate a two-dimensional spatial offset weight matrix. The spatial offset weight matrix is ​​applied to each element of the feature map, and the range of the perceptual region for feature extraction is dynamically adjusted according to the assigned weight values, so that the target features are restored to their original geometric positions in the reference image frame.

3. The method according to claim 2, characterized in that, The step of applying the spatial offset weight matrix to each element of the feature map and dynamically adjusting the perceptual region range of feature extraction according to the weight values, so that the target features are restored to their original geometric positions in the reference image frame, includes: The size of the corresponding rectangular sensing region is determined based on the weight value of each element in the spatial offset weight matrix, and the rectangular sampling region is delineated with the element position in the feature map as the center point. Within the rectangular sampling area, feature points along the offset direction are preferentially selected based on the position offset direction in the target motion trajectory, and the feature values ​​are sampled according to Gaussian distribution weights. Bilinear interpolation is performed on the selected feature points to generate feature values ​​of the corresponding elements in the feature map at their original geometric positions. By performing the above operation on each element of the feature map, the target features are restored to their original geometric positions in the reference image frame.

4. The method according to claim 1, characterized in that, The process of weightedly fusing the restored feature map with the enhanced high-frequency edge components, generating a motion calibration coefficient map based on the target motion trajectory, and using the motion calibration coefficient map to perform trajectory consistency calibration on the fused features to output an optimized feature map for target detection includes: The restored feature map is assigned a first weight distribution, and the enhanced high-frequency edge components are assigned a second weight distribution. Based on the assigned weight distribution, position-by-position weighted superposition is performed to generate the initial fusion feature. Based on the predicted motion direction of the target motion trajectory and the actual motion direction reflected by the initial fusion features, a direction consistency metric is calculated for each image location, and a motion calibration coefficient map is generated based on the direction consistency metric. The motion calibration coefficient map is multiplied element-wise with the initial fusion feature to enhance the features in high-consistency regions and suppress the features in low-consistency regions, so as to output a calibrated optimized feature map for target detection.

5. The method according to claim 1, characterized in that, The step of performing frequency domain decomposition processing on the locally sharpened image, simultaneously retrieving the pixel displacement data to generate a blur intensity distribution map, and performing energy adaptive enhancement processing on the high-frequency edge components separated in the frequency domain based on the blur intensity distribution map includes: A spatial-to-frequency domain transformation operation is performed on the locally sharpened image to separate the high-frequency edge components containing edge details and the low-frequency fundamental components of smooth regions. The two-dimensional offset in the pixel displacement data is retrieved, and the blur intensity value is calculated based on the total displacement amplitude of each pixel position. The blur intensity values ​​of each pixel position are combined to form a blur intensity distribution map with the same size as the image. In the frequency domain, the fuzzy intensity distribution map is aligned with the high-frequency edge component, and the energy intensity value of the high-frequency edge component is proportionally increased according to the fuzzy intensity value at the corresponding position of each frequency point.

6. The method according to claim 1, characterized in that, The process of acquiring timestamped image sequences and motion measurement data, generating multidimensional motion displacement vectors based on the motion measurement data, and dynamically selecting reference image frames with the lowest motion blur from the image sequences based on the multidimensional motion displacement vectors includes: Acquire time-stamped image sequences and motion measurement data containing information on the device’s motion changes in multiple directions; Based on the time interval between adjacent images in the image sequence, the motion measurement data is divided into multiple time intervals, and a multidimensional motion displacement vector is formed by combining the cumulative displacement components in each time interval. For each image frame in the image sequence, a multidimensional motion displacement vector at the corresponding time point is extracted and the amplitude value of the multidimensional motion displacement vector is determined. The image frame with the smallest amplitude value is selected as the reference image frame by comparing the amplitude values ​​of all image frames.

7. A real-time target detection system based on motion trajectory prediction and calibration, characterized in that, include: The acquisition module is used to acquire image sequences with timestamps and motion measurement data, generate multidimensional motion displacement vectors based on the motion measurement data, and dynamically select reference image frames with the lowest motion blur from the image sequences based on the multidimensional motion displacement vectors. The generation module is used to construct a motion trajectory model based on the multidimensional motion displacement vector and the reference image frame, perform motion blur inverse compensation operation on the current frame in the image sequence according to the motion trajectory model, generate a locally sharpened image, and record the pixel displacement data generated during the compensation process. The processing module is used to perform frequency domain decomposition processing on the locally sharpened image, retrieve the pixel displacement data to generate a blur intensity distribution map, and perform energy adaptive enhancement processing on the high-frequency edge components separated in the frequency domain based on the blur intensity distribution map. The adjustment module is used to fuse the multidimensional motion displacement vector and the pixel displacement data to predict the target motion trajectory, generate a spatial offset weight matrix based on the target motion trajectory, and dynamically adjust the perception area of ​​the feature map through the spatial offset weight matrix to restore the geometric position of the target. The output module is used to perform weighted fusion of the restored feature map and the enhanced high-frequency edge components, generate a motion calibration coefficient map by combining the target motion trajectory, and perform trajectory consistency calibration on the fused features using the motion calibration coefficient map to output an optimized feature map for target detection. The process involves constructing a motion trajectory model based on the multidimensional motion displacement vector and the reference image frame, performing motion blur inverse compensation on the current frame of the image sequence according to the motion trajectory model, generating a locally sharpened image, and recording the pixel displacement data generated during the compensation process, including: Using the reference image frame as the reference time point, the multidimensional motion displacement vectors are arranged in chronological order, and the motion trajectory model from the reference image frame to each time point is constructed by accumulating the displacement components. Based on the motion trajectory model, the displacement information of the current frame time point relative to the reference image frame in the image sequence is extracted, and the position offset of each pixel in the current frame relative to the pixel reference position in the reference image frame is derived. Based on the position offset, the pixel position of the current frame is pushed to the corresponding position of the reference image frame in the opposite direction of motion, and the missing pixel positions are filled with neighboring values ​​during the pushing process to generate a locally sharpened image. During the reverse shifting process, the actual displacement difference between the original position coordinates and the shifted position coordinates of each pixel is measured simultaneously to form pixel displacement data containing two-dimensional offset.

8. A computing device, characterized in that, It includes a processing component and a storage component; the storage component stores one or more computer instructions; the one or more computer instructions are invoked and executed by the processing component to implement a real-time target detection method based on motion trajectory prediction and calibration as described in any one of claims 1 to 6.

9. A computer storage medium, characterized in that, The device contains a computer program that, when executed by a computer, implements a real-time target detection method based on motion trajectory prediction and calibration as described in any one of claims 1 to 6.

Images