A motion blur compensation image enhancement method for high-speed cameras

Abstract

This invention discloses a motion blur compensation image enhancement method for high-speed cameras, relating to the field of computer vision technology. The method includes acquiring raw sequence data recorded by a high-speed imaging device, analyzing overlapping regions of adjacent frames in the sequence using a temporal convolutional network to determine the temporal continuity features corresponding to highly overlapping frames, obtaining a separated frame sequence, and using an optical flow estimation algorithm to track the object's movement trajectory based on the separated frame sequence. If the optical flow vector exceeds a preset threshold, it is judged as an interference factor in a dynamic scene, resulting in a motion-compensated frame sequence. This motion blur compensation image enhancement method for high-speed cameras not only improves the accuracy of image enhancement but also enables real-time and accurate extraction of key information frames in complex dynamic environments, optimizing the defect detection process. It has broad application prospects, particularly in high-speed motion analysis and industrial inspection, significantly improving detection efficiency and accuracy.

Description

Technical Field

[0001] This invention relates to the field of computer vision technology, and more specifically to an image enhancement method for motion blur compensation of high-speed cameras. Background Technology

[0002] In the fields of high-speed imaging and motion blur compensation, accurately recovering details of dynamic scenes from high-speed image sequences, especially in the presence of motion blur and lighting interference, has become a pressing problem. Existing image enhancement methods are often limited by how to efficiently extract realistic information from high-speed image sequences and effectively compensate for and remove interference such as motion blur, noise, and lighting jitter.

[0003] Traditional motion blur compensation techniques primarily rely on static image processing methods. However, for highly overlapping images captured by high-speed imaging devices, they often fail to accurately distinguish the temporal continuity between frames, resulting in an inability to accurately reproduce details in dynamic scenes. In high-speed photography, especially in industrial inspection and motion analysis applications, imaging devices frequently capture images that change rapidly within a short period. The overlapping areas between adjacent frames are often extremely subtle, making it difficult for traditional algorithms to effectively separate information from different points in time. Furthermore, imaging systems often cannot effectively detect and eliminate interference from light shading and dynamic disturbances. Existing methods, when faced with these complex dynamic disturbances, often rely on simple noise removal techniques, failing to effectively distinguish between normal dynamic changes and interference information, thus affecting the clarity and realism of the final image. Summary of the Invention

[0004] The purpose of this invention is to provide an image enhancement method for motion blur compensation of high-speed cameras, thereby solving the problems existing in the prior art.

[0005] To achieve the above objectives, the present invention provides the following technical solution: a motion blur compensation image enhancement method for high-speed cameras, comprising: S1. Obtain the raw sequence data recorded by the high-speed imaging device, analyze the overlapping area of ​​adjacent frames in the sequence through a temporal convolutional network, determine the temporal continuity features corresponding to highly overlapping frames, and obtain the separated frame sequence. S2. Based on the separated frame sequence, the optical flow estimation algorithm is used to track the object's movement trajectory. If the optical flow vector exceeds the preset threshold, it is judged as an interference factor in the dynamic scene, and the motion-compensated frame sequence is obtained. S3. Obtain the pixel offset value caused by light jitter through the frame sequence after motion compensation, and determine whether the offset value is within the preset range. If the offset value exceeds the range, apply differential filtering to determine the denoised frame sequence. S4. Extract the instantaneously changing edge details from the denoised frame sequence, and use a multi-scale decomposition method to separate low-frequency and high-frequency components to obtain an enhanced detail sequence; S5. Based on the enhanced detail sequence, determine the distribution of blurred areas in the image. If the proportion of blurred areas is higher than a preset threshold, adjust the pixel values ​​through backpropagation to obtain a sharpened sequence. S6. Obtain the time frames in the sharpened sequence, and perform interpolation fusion by combining the temporal continuity features to determine the real information frames required for defect detection. S7. Analyze surface defects using real information frames. If the boundaries are discontinuous, use deep learning to detect the defect boundaries, fill the discontinuous areas, and generate a detection sequence.

[0006] Preferably, S1 includes: Acquire the raw sequence data recorded by the high-speed imaging device and divide the raw sequence data into multiple groups of adjacent images; Adjacent frames are input into a temporal convolutional network to generate convolutional feature maps, and overlapping frames are determined based on the convolutional feature maps. Extract deep temporal dependencies of overlapping frames to construct temporal continuity features. If the inter-frame correlation degree corresponding to the temporal continuity features exceeds the preset standard, a redundant frame marker is generated. By removing frames with redundant frame markers, a separate frame sequence is obtained.

[0007] Preferably, S2 includes: Obtain the original separation frame sequence and generate dense optical flow field data from the original separation frame sequence; If the magnitude of the optical flow vector in the dense optical flow field data exceeds a preset threshold, it is identified as an interference factor. An interference region mask is constructed based on the interference factors, and the background optical flow vector set is extracted from the dense optical flow field data using the interference region mask. The original separated frame sequence is reversed geometrically based on the background optical flow vector set to obtain the motion-compensated frame sequence.

[0008] Preferably, S3 includes: Obtain the motion-compensated frame sequence; The brightness gradient change matrix is ​​extracted from the motion-compensated frame sequence, and the pixel offset value is separated from the brightness gradient change matrix. Determine whether the pixel offset value is within the preset offset threshold range; If the pixel offset value exceeds the offset threshold range, then the spatial neighborhood feature matrix is ​​extracted; The denoised frame sequence is determined by applying differential filtering based on the spatial neighborhood feature matrix and pixel offset values.

[0009] Preferably, S4 includes: Edge details are extracted, which are obtained by performing pixel interpolation on the denoised frame sequence; Low-frequency and high-frequency components are separated, and the low-frequency and high-frequency components are obtained by multi-scale decomposition of edge details; An enhanced high-frequency component is obtained, which is generated by contrast mapping when the high-frequency component meets a preset threshold. The enhanced detail sequence is obtained by weighted summation of the low-frequency components and the enhanced high-frequency components.

[0010] Preferably, S5 includes: The enhanced detail sequence is obtained, and the high-frequency components of the enhanced detail sequence are extracted using a high-pass filter operator to generate a gradient feature map; Identify sets of pixels whose grayscale change rate is lower than the sharpness standard based on gradient feature maps, and map these sets of pixels into a fuzzy region distribution matrix. Calculate the proportion of the fuzzy region distribution matrix. If the proportion is higher than a preset threshold, construct a loss function model. The backpropagation operation is performed based on the loss function model to obtain the corrected gradient vector. The corrected gradient vector is then used to adjust the pixel values ​​of the enhanced detail sequence to obtain the sharpened sequence.

[0011] Preferably, S6 includes: Obtain a sharpened video sequence containing the object to be detected, and extract the target frame and neighboring reference frames; An optical flow motion vector field is constructed based on the target frame and neighboring reference frames to generate motion-compensated aligned frames.

[0012] Preferably, S6 further includes: A spatiotemporal fusion weight matrix is ​​generated based on the grayscale difference between the target frame and the motion-compensated aligned frame, and the fused enhanced frame is obtained using the spatiotemporal fusion weight matrix. By eliminating abrupt noise based on the fused and enhanced frames, the true information frames used for defect detection are determined.

[0013] Preferably, S7 includes: Acquire real information frames for defect detection, and extract an initial defect mask containing abnormal textures based on the gray-level gradient distribution of the real information frames; If the initial defect mask has discontinuous boundaries, a local image patch is extracted as input boundary to complete the deep neural network and generate a predicted boundary path connecting the break points.

[0014] Preferably, S7 further includes: Based on the predicted boundary path, adaptive morphological structural elements are set, and a closing operation and texture synthesis filling are performed on the initial defect mask to obtain a morphologically complete corrected defect mask. The defect mask is mapped back to the original time coordinates and the frame data is reassembled to generate an accurate detection sequence with continuous boundary features.

[0015] As can be seen from the above technical solution, the present invention has the following beneficial effects: This motion blur compensation image enhancement method for high-speed cameras solves the problems of motion blur, noise, and light jitter in high-speed imaging by combining advanced techniques such as temporal convolutional networks, optical flow estimation, differential filtering, and multi-scale decomposition, significantly improving image clarity and realism. Compared with traditional methods, this invention can accurately extract the temporal continuity features in high-speed image sequences, effectively separate highly overlapping regions, compensate for blur and noise in dynamic scenes, and restore object details. By detecting defect boundaries through deep learning and combining adaptive morphological operations to repair discontinuous regions, the accuracy and reliability of surface defect detection are improved. This method not only improves the accuracy of image enhancement but also can extract real information frames at key moments in real time and accurately in complex dynamic environments, optimizing the defect detection process. It has broad application prospects, especially in high-speed motion analysis and industrial inspection, where it can significantly improve detection efficiency and accuracy. Attached Figure Description

[0016] Figure 1 This is a flowchart of the overall method of the present invention; Figure 2 This is a diagram illustrating the original sequence separation and processing of this invention; Figure 3 This is a diagram illustrating optical flow interference removal and motion compensation in this invention. Figure 4 This is the pixel offset detection and differential denoising diagram of the present invention; Figure 5 This is an image showing the edge detail enhancement and multi-scale decomposition of the present invention; Figure 6 This is a diagram illustrating the blurry region identification and sharpening process of this invention. Figure 7 This is a diagram of the spatiotemporal fusion real information frame generation of the present invention; Figure 8 This is a diagram showing the defect boundary detection, completion, and detection sequence reconstruction of the present invention. Detailed Implementation

[0017] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0018] like Figure 1As shown, the present invention provides a technical solution: a motion blur compensation image enhancement method for high-speed cameras, comprising: S1. Obtain the raw sequence data recorded by the high-speed imaging device, analyze the overlapping area of ​​adjacent frames in the sequence through a temporal convolutional network, determine the temporal continuity features corresponding to highly overlapping frames, and obtain the separated frame sequence. S2. Based on the separated frame sequence, the optical flow estimation algorithm is used to track the object's movement trajectory. If the optical flow vector exceeds the preset threshold, it is judged as an interference factor in the dynamic scene, and the motion-compensated frame sequence is obtained. S3. Obtain the pixel offset value caused by light jitter through the frame sequence after motion compensation, and determine whether the offset value is within the preset range. If the offset value exceeds the range, apply differential filtering to determine the denoised frame sequence. S4. Extract the instantaneously changing edge details from the denoised frame sequence, and use a multi-scale decomposition method to separate low-frequency and high-frequency components to obtain an enhanced detail sequence; S5. Based on the enhanced detail sequence, determine the distribution of blurred areas in the image. If the proportion of blurred areas is higher than a preset threshold, adjust the pixel values ​​through backpropagation to obtain a sharpened sequence. S6. Obtain the time frames in the sharpened sequence, and perform interpolation fusion by combining the temporal continuity features to determine the real information frames required for defect detection. S7. Analyze surface defects using real information frames. If the boundaries are discontinuous, use deep learning to detect the defect boundaries, fill the discontinuous areas, and generate a detection sequence.

[0019] In this embodiment, to address the issues of motion blur, lighting jitter, and noise superposition that high-speed cameras are prone to in dynamic scenes, image quality is improved through a combination of temporal modeling and spatial detail enhancement. First, in step S1, a temporal convolutional network is used to model consecutive frames. This involves extracting features along the temporal dimension using a one-dimensional or three-dimensional convolutional structure, analyzing overlapping regions between adjacent frames, and determining the temporal continuity features corresponding to highly overlapping scenes, thereby obtaining a separated frame sequence. Further, in step S2, based on the separated frame sequence, an optical flow estimation algorithm is used to track the trajectory of moving objects. This algorithm can be based on a pyramid structure or a deep learning model (such as PWC-Net or RAFT). When the optical flow vector exceeds a preset threshold, it is identified as an interference factor in the dynamic scene, and motion compensation processing is performed on the relevant regions to obtain an aligned frame sequence.

[0020] In some embodiments, after motion compensation is completed, in step S3, the pixel offset value caused by light jitter is obtained by analyzing the brightness changes between compensated frames. When the offset value exceeds a preset range, a differential filtering method is used to process it, so as to suppress random noise and determine the denoised frame sequence. In addition, in step S4, the instantaneously changing edge details are extracted from the denoised frame sequence, and a multi-scale decomposition method (such as wavelet transform or Laplacian pyramid) is used to separate low-frequency structural information and high-frequency detail information, thereby obtaining an enhanced detail sequence.

[0021] As a preferred embodiment, in step S5, blurred regions of the image are detected based on the enhanced detail sequence. The blurred regions can be determined based on gradient information or frequency domain energy distribution. When the proportion of blurred regions exceeds a preset threshold, pixel values ​​are adaptively adjusted through a backpropagation mechanism to obtain a sharpened sequence. Therefore, the method of this application can effectively restore details in blurred regions while preserving structural information.

[0022] Furthermore, in step S6, time frames from the sharpened sequence are acquired and interpolated and fused using the aforementioned temporal continuity features. Optical flow-based interpolation methods or deep learning frame interpolation models can be employed to determine the real information frames required for defect detection. In another specific implementation, this interpolation process can incorporate a time-weighted strategy to improve fusion accuracy. Finally, in step S7, surface defects are analyzed using the real information frames. When boundary discontinuities are detected, a deep learning model is used to detect the defect boundaries and fill in the discontinuous areas, thereby generating a detection sequence.

[0023] like Figure 2 As shown, S1 includes acquiring the raw sequence data recorded by the high-speed imaging device, dividing the raw sequence data into multiple groups of adjacent images; inputting the adjacent images into a temporal convolutional network to generate convolutional feature maps, determining overlapping images based on the convolutional feature maps; extracting deep temporal dependencies of the overlapping images to construct temporal continuity features, generating redundant frame markers if the inter-frame correlation corresponding to the temporal continuity features exceeds a preset standard; and removing images with redundant frame markers to obtain a separated frame sequence.

[0024] In this embodiment, the raw sequence data recorded by the high-speed imaging device is first acquired and then organized according to the acquisition time sequence to form a continuously arranged set of image frames. Before further processing, the raw sequence data undergoes image size standardization, grayscale range adjustment, and temporal sequence correction to avoid differences in data representation between consecutive frames that could affect subsequent overlap analysis results. Furthermore, the organized raw sequence data is divided into multiple groups of adjacent images. The group length of adjacent images is determined comprehensively based on the sampling frequency of the high-speed imaging device, the speed of the target's movement, the displacement variation between adjacent frames, and the accuracy requirements of subsequent temporal analysis. When the sampling frequency is high and the target's movement speed is fast, the number of frames in each group of adjacent images is relatively small to ensure a small time span between adjacent images within the same group; when the target's movement is slow or the scene changes are relatively stable, the number of frames in each group of adjacent images is increased accordingly to improve the stability of continuous feature extraction. Therefore, in this embodiment, by grouping the raw sequence data, a local temporal continuity relationship is established first, providing ordered input for subsequent convolutional feature extraction and overlapping image recognition.

[0025] Furthermore, each group of adjacent frames is input into a temporal convolutional network to generate convolutional feature maps corresponding to each group of frames. The temporal convolutional network extracts features layer by layer from consecutive frames along both temporal and spatial dimensions. It first extracts edge distribution, texture direction, brightness changes, and contour positions between adjacent frames, then aggregates these features layer by layer to form a deep feature response that reflects temporal relationships. The front layers of the network primarily highlight local structure and grayscale variation information, the middle layers extract continuous motion features across frames, and the back layers form convolutional feature maps with high discriminative power for overlapping frames. The receptive range of the convolutional kernel is determined based on the target detail scale and motion amplitude. When the target texture is fine and local changes are significant, the receptive range is smaller to highlight detail differences; when the target contour is large and the overall displacement is significant, the receptive range is correspondingly expanded to enhance the overall trend recognition ability. The number of network layers is determined based on the scene complexity and the depth of temporal relationships. When there is occlusion, reflection, or local deformation in the scene, the number of layers is increased accordingly to improve the deep feature extraction effect. It should be noted that processing the original image through a temporal convolutional network is not merely a static analysis of a single frame, but rather an extraction of the continuous change relationships between adjacent frames. This allows the convolutional feature map to reflect not only spatial structural information but also temporal evolution characteristics.

[0026] In some embodiments, the degree of overlap between adjacent frames is identified based on convolutional feature maps. Specifically, the convolutional feature maps of consecutive frames within the same group are first compared region by region, and the consistency of each region in terms of edge position, texture direction, contour shape, and local response intensity is analyzed. If a region's response position is close in two consecutive frames, its structural direction is clearly continuous, and its intensity change is small, then the region is determined to have an overlapping relationship; if a local region shows obvious displacement, structural breakage, or abrupt response change, then the degree of overlap in that region is determined to be low. After completing the region-level analysis, the comparison results of all regions are summarized to form a frame-level overlap judgment result. The region where the target subject is located, the region with concentrated edge information, and the region sensitive to structural changes have higher weights in the overall judgment, while the judgment weight of regions with flat backgrounds is relatively lower to prevent background similarity from interfering with the overlap judgment. Furthermore, the criteria for determining overlapping frames are based on the distribution of feature differences between adjacent frames in continuous scene samples. Specifically, this is achieved by comparing and statistically analyzing known continuous state frames with frames showing significant changes to determine the distinction boundary between the two types of samples. This is then corrected by considering cases of accidental deletion and false inclusion, ensuring that the final criterion can both identify truly continuous frames and exclude discontinuous changing frames. Therefore, in this embodiment, identifying the degree of overlap between adjacent frames using convolutional feature maps can accurately distinguish between repetitive information and valid change information.

[0027] Furthermore, after identifying overlapping frames, the deep temporal dependencies between them are extracted to construct temporal continuity features. These temporal continuity features do not merely represent the surface similarity between two adjacent frames, but rather the overall consistency across multiple consecutive frames in terms of edge migration, texture continuity, contour changes, and brightness transitions. Specifically, frames identified as having high overlap are first arranged in their original temporal order. Then, the direction of target boundary movement, the rhythm of local texture changes, the continuity of contour shapes, and the smooth transition of brightness distribution across multiple consecutive frames are analyzed. By jointly analyzing the trends of consecutive frames, the stable and consistent temporal correlations are extracted, while abrupt changes are excluded, thus forming the temporal continuity features. The number of frames participating in the continuity analysis is determined based on the target motion cycle, sampling time interval, and scene change rhythm. When the target motion state changes rapidly, the continuous analysis range is relatively narrowed to avoid crossing obvious motion phases; when the target motion state is stable, the continuous analysis range is correspondingly expanded to enhance the temporal trend recognition effect. As a preferred implementation, the construction process of temporal continuity features is also constrained by the requirement of regional continuity and consistency, that is, regional features that maintain stable positional and structural relationships in multiple consecutive frames are preferentially retained, thereby improving the ability of the constructed features to represent the real temporal state.

[0028] In some embodiments, a comprehensive analysis of temporal continuity features is performed to obtain the corresponding inter-frame correlation degree. Specifically, the changes in contour position, the continuity of the main texture direction, the changes in the brightness center, the continuity of edge response, and the preservation of overlapping areas are first statistically analyzed among multiple consecutive frames. Then, the analysis results of the above multiple aspects are uniformly converted into comparable correlation judgment results, and the influence of each content on temporal continuity is weighted and integrated to obtain the corresponding inter-frame correlation degree. Among them, the change in contour position is mainly used to reflect whether the target subject moves along a continuous trajectory; the change in the main texture direction is used to reflect whether the target surface features continue; the change in brightness center is used to eliminate local flicker interference; the continuity of edge response is used to identify the continuous relationship at the structural level; and the preservation of overlapping areas is used to confirm the spatial commonality between adjacent frames. The judgment criteria for inter-frame correlation degree are determined based on the sample statistical results of consecutive and non-consecutive frames, and adjusted in combination with the requirements of temporal integrity in subsequent processing. Specifically, the distribution differences in correlation between real continuous and non-continuous frames are first statistically analyzed. Then, adjustments are made based on redundancy compression requirements and key process retention requirements. When the processing target is biased towards compressing redundant data, the judgment standard is relatively higher; when the processing target is biased towards retaining more intermediate changes, the judgment standard is relatively lower. It should be noted that the judgment of inter-frame correlation in this application does not rely on a single feature, but is based on the joint analysis of multiple temporal information.

[0029] Furthermore, when the inter-frame correlation exceeds a preset standard, a redundant frame marker is generated. This preset standard is determined based on both key dynamic information retention requirements and redundant data compression requirements to avoid misclassifying frames carrying valid motion changes as redundant. Specifically, first, continuous frame segments with consistently high correlation are identified. Then, within these segments, the contribution of each frame to the overall motion expression is analyzed. If a frame only serves a repetitive function between its preceding and following frames, and its deletion does not disrupt the continuity of the target trajectory, edge transitions, or structural changes, then that frame is marked as redundant. Furthermore, to prevent the continuous deletion of too many frames from reducing the temporal resolution of subsequent sequences, the number of consecutive redundant frames is constrained. Only when the continuity between preceding and following frames remains intact after continuous deletion is the corresponding frame marked as redundant. Therefore, in this embodiment, the generation of redundant frame markers is not simply based on similarity, but rather a comprehensive judgment combining temporal continuity and subsequent fidelity requirements.

[0030] Subsequently, images with redundant frame markers are removed, while the original temporal order of unmarked images is preserved, resulting in a separated frame sequence. Specifically, all images are scanned frame by frame according to the original time sequence. Marked images are removed, while unmarked images are retained unchanged. During the removal process, the temporal span and structural continuity between adjacent retained images before and after removal are checked. If a clear continuity is maintained between images before and after a marked image is deleted, the removal is completed; if a significant jump, motion discontinuity, or abnormal edge continuity occurs after deletion, the image is restored and reintroduced into the retained sequence. After the removal is completed, the remaining images are rearranged to form a separated frame sequence that is temporally continuous, concise in content, and retains effective dynamic information. As can be seen from the above, in this embodiment, by grouping the original sequence data, extracting convolutional features, identifying overlap, analyzing temporal continuity, determining inter-frame correlation, and removing redundancy, effective compression of repetitive information in high-speed image sequences is achieved, while retaining the key temporal information required for subsequent motion compensation and image enhancement.

[0031] like Figure 3 As shown, S2 includes acquiring the original separated frame sequence, generating dense optical flow field data for the original separated frame sequence; if the optical flow vector magnitude in the dense optical flow field data exceeds a preset threshold, it is identified as an interference factor; constructing an interference region mask based on the interference factor, and extracting a background optical flow vector set from the dense optical flow field data using the interference region mask; performing inverse geometric correction on the original separated frame sequence based on the background optical flow vector set to obtain a motion-compensated frame sequence.

[0032] In this embodiment, the original separated frame sequence is first obtained, and the temporally adjacent preceding and following frames in the original separated frame sequence are treated as a group of processing units. The original separated frame sequence is a set of continuous image frames retained after removing redundant preceding frames; this sequence contains continuous change information during target motion and background displacement information. Further, dense optical flow field data is generated from the original separated frame sequence. Specifically, pixel-by-pixel matching is performed on the grayscale distribution, texture structure, and edge contours of the same and neighboring positions in the preceding and following frames to determine the direction and magnitude of positional change for each pixel from the preceding to the following frame. The positional change results of all pixels are then summarized to form dense optical flow field data covering the entire image. Each pixel position in the dense optical flow field data corresponds to an optical flow vector, which represents the displacement direction and magnitude of that pixel position between two adjacent frames.

[0033] In some embodiments, before generating dense optical flow field data, adjacent frames in the original separated frame sequence are first subjected to brightness consistency processing and noise suppression processing to eliminate the influence of local flicker and random noise on the pixel matching results. Brightness consistency processing is achieved by unifying the brightness distribution range of adjacent frames, and noise suppression processing is achieved by weakening isolated noise points and local abnormal responses, thereby maintaining consistency in the correspondence between adjacent frames. After preprocessing, all optical flow vectors in the dense optical flow field data are traversed, and the magnitude of each optical flow vector is calculated one by one. The magnitude of the optical flow vector is determined by the combined displacement of the corresponding pixel position in the horizontal and vertical directions, and the magnitude represents the actual motion intensity of the pixel position between two adjacent frames. Further, the magnitude of the optical flow vector is compared point-by-point with a preset threshold; when the magnitude of the optical flow vector exceeds the preset threshold, the corresponding pixel position is directly determined as the location of the interference factor.

[0034] The preset threshold is determined jointly based on the displacement statistics of the background area and the displacement statistics of the abnormal motion area. The specific determination process is as follows: First, select multiple sets of image data with stable background areas and clear local interference targets from continuously acquired sample images. Then, statistically analyze the optical flow modulus distribution at each pixel position within the background area and the optical flow modulus distribution at each pixel position within the interference target area. Subsequently, determine the modulus boundary position between the background area and the interference area, and use this boundary position as the preset threshold. If there is overall camera displacement in the scene, first, calculate the average displacement level of the overall background motion, and then superimpose the separation boundary of local abnormal displacement on this average displacement level to form the final threshold. Therefore, in this embodiment, the identification of interference factors is based on whether the optical flow modulus exceeds a clearly defined judgment standard, rather than on subjective judgment.

[0035] Furthermore, after identifying the interference factors, an interference region mask is constructed based on these factors. The specific process is as follows: First, all locations identified as interference factors are extracted. Then, continuously distributed abnormal locations are merged into the same region according to spatial adjacency, forming an initial interference region. Subsequently, the boundaries of the initial interference region are refined to ensure complete coverage of abnormal locations corresponding to the same interference target, thereby generating the interference region mask. Specifically, if multiple abnormal locations are spatially continuous and their corresponding optical flow directions are consistent, and their optical flow moduli are within the same variation range, these multiple abnormal locations are merged into the same interference region. If an abnormal location exists in isolation and there are no continuous abnormal responses in adjacent frames, this abnormal location is removed from the interference region to prevent isolated noise points from entering the mask area. The interference region mask is a region marking result consistent with the image size; the mask-covered area corresponds to the interference factor region, and the mask-uncovered area corresponds to the background region.

[0036] In another specific implementation, a background optical flow vector set is extracted from dense optical flow field data using an interference region mask. The specific process is as follows: First, the interference region mask is mapped onto the dense optical flow field data. Then, all optical flow vectors within the mask-covered area are deleted, retaining only those within the mask-uncovered area, forming an initial background optical flow vector set. Subsequently, the initial background optical flow vector set undergoes consistency screening, deleting anomalous vectors with abrupt direction changes, discrete magnitudes, and discontinuous spatial distributions. Optical flow vectors with concentrated directions, similar magnitudes, and continuous spatial distributions are retained, resulting in the background optical flow vector set. The criteria for determining abrupt direction changes are: the direction difference between a given optical flow vector and most optical flow vectors in its neighboring region exceeds a predetermined direction difference range; the criteria for discrete magnitudes are: the difference between the magnitude of a given optical flow vector and the average magnitude of the surrounding background exceeds a predetermined difference range; and the criteria for discontinuous spatial distribution are: there is no continuous displacement connection between the region containing the optical flow vector and adjacent background regions. The predetermined direction difference range and predetermined difference range are determined based on the continuous motion distribution results in the background region samples, with the stable motion range within the background region serving as the basis for judgment.

[0037] Furthermore, inverse geometric correction is performed on the original separated frame sequence based on the background optical flow vector set to obtain the motion-compensated frame sequence. The specific process is as follows: First, the background optical flow vector set is statistically analyzed to determine the principal displacement direction and magnitude of the background region between adjacent frames. Then, based on the principal displacement direction and magnitude, the positional offset relationship between the current frame and the reference frame is established. Subsequently, the pixel positions in the current frame are repositioned in the opposite direction to the positional offset relationship, so that the background region is realigned between the current frame and the reference frame. The inverse geometric correction is based on the overall motion result of the background region; therefore, this correction process directly eliminates image misalignment caused by camera shake, platform shift, and overall background movement. If there are local offset differences in different positions within the background region, after the overall repositioning is completed, regional position correction is performed on the local regions, so that each region completes its position adjustment according to the corresponding background optical flow result. After completing the repositioning of all pixel positions, the motion-compensated frame sequence is obtained.

[0038] As can be seen from the above, in this embodiment, the separation of the dynamic interference region and the background motion region is completed by acquiring the original separation frame sequence, generating dense optical flow field data, determining the optical flow vector magnitude, constructing the interference region mask, extracting the background optical flow vector, and performing inverse geometric correction processing, and the background motion offset compensation is also completed.

[0039] like Figure 4As shown, S3 includes acquiring the motion-compensated frame sequence; extracting the brightness gradient change matrix based on the motion-compensated frame sequence, and separating the pixel offset value based on the brightness gradient change matrix; determining whether the pixel offset value is within a preset offset threshold range; if the pixel offset value exceeds the offset threshold range, extracting the spatial neighborhood feature matrix; and applying differential filtering based on the spatial neighborhood feature matrix and the pixel offset value to determine the denoised frame sequence.

[0040] In this embodiment, a motion-compensated frame sequence is first acquired and used as input data for pixel offset analysis and noise suppression processing. The motion-compensated frame sequence is a set of continuous image frames after background motion correction. The background regions in this frame sequence have been spatially aligned, and the inter-frame positional deviations are mainly concentrated in areas of light jitter, local noise disturbances, and residual subtle displacements. Further, the motion-compensated frame sequence is scanned frame by frame, and a brightness variation distribution is established according to pixel position in each frame. Then, a brightness gradient variation matrix is ​​extracted based on the brightness differences between adjacent pixels. The brightness gradient variation matrix characterizes the intensity of brightness variation at each pixel position in the horizontal and vertical directions, reflecting edge transitions, texture undulations, and local brightness disturbances.

[0041] Furthermore, pixel offset values ​​are extracted based on the brightness gradient change matrix. The specific process is as follows: First, the brightness gradient change results at corresponding positions in the current frame are aligned and compared with those in the preceding and following frames. Then, the gradient peak position, gradient change direction, and gradient response intensity changes are extracted. The pixel offset value is determined based on the gradient center offset of the same structural region in consecutive frames. The pixel offset value characterizes the degree of positional drift of the same image structure between consecutive frames, and its determination is based on the positional change of the gradient response center in consecutive frames. If the gradient center position of the same edge or texture structure moves in consecutive frames, the amount of movement is determined as the pixel offset value at the corresponding position. The pixel offset value extraction process is based on the continuity of local structure. Flat areas without obvious gradient responses are first excluded, and then offset analysis is performed on areas with clear edge and texture responses, thereby ensuring that the extraction results are consistent with the actual structural changes.

[0042] In some embodiments, after obtaining the pixel offset value, it is determined whether the pixel offset value is within a preset offset threshold range. The offset threshold range is determined based on the distribution of normal inter-frame micro-displacements after motion compensation and the distribution of abnormal light jitter displacements. Specifically, the determination process is as follows: first, the pixel offset value distribution of stable regions is extracted from multiple sets of motion-compensated sample frame sequences; then, the pixel offset value distribution of regions affected by illumination disturbances and random noise is extracted; subsequently, the offset boundary range between stable regions and abnormally disturbed regions is determined, and this boundary range is used as the offset threshold range. If the pixel offset value falls within the offset threshold range, the corresponding position is determined to be within the normal fluctuation range allowed after compensation; if the pixel offset value exceeds the offset threshold range, the corresponding position is determined to have an abnormal offset, and subsequent differential filtering processing is initiated. Therefore, in this embodiment, by filtering pixel offset values ​​through the offset threshold range, the separation of normal micro-changes and abnormally disturbed changes is achieved.

[0043] Furthermore, when the pixel offset value exceeds the offset threshold range, a spatial neighborhood feature matrix is ​​extracted. This spatial neighborhood feature matrix characterizes the structural continuity, brightness distribution, and texture variation relationships of the surrounding area at the abnormal offset location. Specifically, the process involves first extracting a set of neighboring pixels within a preset range centered on the abnormal offset pixel location. Then, the brightness uniformity, gradient continuity, edge direction consistency, and texture distribution concentration within the neighboring pixel set are statistically analyzed to form the corresponding spatial neighborhood feature matrix. The preset range is determined based on image resolution, target edge width, and local noise diffusion range. When the image resolution is high, the neighborhood extraction range is determined based on the local structural dimensions; when the target edge is wide, the neighborhood extraction range covers the complete edge transition area; when the local noise diffusion range is large, the neighborhood extraction range covers the area where the abnormal response diffuses to the surrounding area. Through this processing, the spatial neighborhood feature matrix fully reflects the local structural state around the abnormal offset location.

[0044] In another specific implementation, differential filtering is applied based on the spatial neighborhood feature matrix and pixel offset values ​​to determine the denoised frame sequence. The specific process is as follows: First, the perturbation intensity at the abnormal offset position is determined based on the pixel offset value. Then, the continuity direction and brightness transition relationship of the surrounding structure at that position are determined based on the spatial neighborhood feature matrix. Subsequently, a differential comparison relationship is established between the abnormal offset position and its neighboring pixels. By comparing the differences in brightness, gradient, and structural response between the current pixel and its neighboring pixels, isolated abnormal responses and discontinuous perturbation responses are identified, and differential suppression processing is performed on these responses. The differential filtering process does not uniformly process all regions; instead, it applies directional filtering to positions where the pixel offset value exceeds the offset threshold range, while maintaining the original response for structural regions within the normal offset range. For real detail regions with continuous edges and clear textures, the spatial neighborhood feature matrix shows continuous direction and consistent gradient, thus preserving them during differential filtering. For abnormal responses caused by light jitter or local noise, the spatial neighborhood feature matrix shows directional breaks, brightness abrupt changes, or local isolation, thus suppressing them during differential filtering. After completing the differential filtering of all abnormal offset positions, the processed frames are recombined according to the original time order to obtain the denoised frame sequence.

[0045] like Figure 5 As shown, S4 includes extracting edge details, which are obtained by performing pixel interpolation on the denoised frame sequence; separating low-frequency components and high-frequency components, which are obtained by multi-scale decomposition of the edge details; obtaining enhanced high-frequency components, which are generated by contrast mapping when the high-frequency components meet a preset threshold; and performing a weighted summation of the low-frequency components and enhanced high-frequency components to obtain an enhanced detail sequence.

[0046] In this embodiment, a denoised frame sequence is first acquired, and each frame in the denoised frame sequence is used as the object for edge detail extraction. The denoised frame sequence is an image sequence processed by prior anomalous offset suppression and differential filtering. Random noise and local light disturbance responses in this sequence have been weakened, and the main information retained in the image is the target structure contour, texture changes, and edge transition information. Further, pixel difference calculations are performed on the denoised frame sequence to extract edge details. Specifically, for each pixel position in each frame image, the difference in brightness value between that pixel position and its neighboring pixel positions is calculated. Then, the brightness differences in each direction within the same region are summarized to form the edge response result for the corresponding position. The pixel difference calculation highlights areas with concentrated brightness changes and suppresses areas with gradual brightness changes. When there is a continuous and significant brightness difference between a position and its neighboring positions, that position is determined to belong to an edge detail region. When the brightness distribution of a region is gradual and the differences between neighboring positions are continuously small, that region is determined not to belong to an edge detail region. The selection range of the neighboring pixel positions is determined based on image resolution, edge width, and texture density. When the image resolution is high, the selection range of adjacent pixels covers the actual transition area of ​​the edge; when the edge width is large, the selection range of adjacent pixels covers the complete edge level; when the texture density is high, the selection range of adjacent pixels extends according to the texture distribution direction. Therefore, in this embodiment, edge details are extracted through pixel difference calculation, separating the structural boundaries and local details in the image from the denoised frame sequence, providing an input basis for subsequent frequency decomposition.

[0047] Furthermore, after extracting edge details, the edge details are decomposed at multiple scales to separate low-frequency and high-frequency components. Specifically, the edge detail image is decomposed layer by layer according to a scale hierarchy from coarse to fine. Slowly changing, continuously distributed structural information is assigned to the low-frequency components, while rapidly changing, edge-concentrated, and texture-prominent detail information is assigned to the high-frequency components. The low-frequency components represent the main outline, regional brightness transitions, and overall structural distribution in the image, while the high-frequency components represent fine edges, local textures, and sharp transitions. During multi-scale decomposition, the original edge detail image undergoes a first-level scale separation to extract the outermost structural and detail changes. The remaining structural parts are then subjected to the next level of scale separation until a preset decomposition level is reached. The decomposition level is determined based on the target structural size, the number of detail levels, and the required enhancement processing accuracy. When the target outline is large and the structural levels are clear, the decomposition level is set according to a hierarchical relationship from the whole to the part; when there are many detail levels, the decomposition level is further increased; when the enhancement processing accuracy requirements are clear, the decomposition level is determined by the number of layers that can completely separate the main structure and detail texture. It should be noted that this application divides edge details into low-frequency components and high-frequency components through multi-scale decomposition, so that subsequent enhancement processing can directly act on the detail information part, avoiding unnecessary disturbance to the main structure area.

[0048] In some embodiments, after separating the high-frequency components, a threshold judgment is performed on the high-frequency components, and contrast mapping processing is performed when the high-frequency components meet the preset threshold to obtain enhanced high-frequency components. The preset threshold is determined jointly based on the true detail response distribution and residual noise response distribution in the high-frequency components. Specifically, the determination process is as follows: first, high-frequency response results are extracted from multiple sets of denoised frame samples; then, the high-frequency intensity distribution of the true edge region and the high-frequency intensity distribution of the residual noise region are statistically analyzed respectively; subsequently, the boundary position between the two distributions is determined, and this boundary position is used as the processing threshold for the high-frequency components. When the response intensity of the high-frequency component reaches or exceeds the preset threshold, the corresponding position is determined to belong to the detail region that should be enhanced; when the response intensity of the high-frequency component is lower than the preset threshold, the corresponding position is determined not to enter the enhancement process to prevent residual weak noise from being amplified synchronously. Further, contrast mapping processing is performed on the high-frequency components that meet the threshold. Specifically, the response intensity of each position in the high-frequency component is first graded; then, according to the correspondence between response intensity and structural continuity, the high-frequency response amplitude is increased, so that positions with clear edges, concentrated textures, and clear detail transitions obtain higher contrast performance. The contrast mapping process does not uniformly amplify all high-frequency regions. Instead, it determines the enhancement amplitude based on the high-frequency response intensity, local structural continuity, and edge direction consistency. The more concentrated the high-frequency response intensity, the more complete the structural continuity, and the more consistent the edge direction, the higher the enhancement amplitude at that location. Locations with discrete high-frequency responses, interrupted structural continuity, or chaotic directional changes are not included in the enhancement mapping or have reduced enhancement amplitude. Through this process, an enhanced high-frequency component is obtained.

[0049] Furthermore, after obtaining the low-frequency components and enhanced high-frequency components, a weighted sum is performed on the low-frequency components and enhanced high-frequency components to obtain the enhanced detail sequence. The specific process is as follows: First, based on the roles of the low-frequency components and enhanced high-frequency components in image reconstruction, corresponding weights are assigned to each. Then, the low-frequency response value and enhanced high-frequency response value at the same location are superimposed according to the weights to form the enhanced image detail result. The weights of the low-frequency components are determined based on the requirement to preserve the main structure, while the weights of the high-frequency components are determined based on the requirement to enhance details. When the requirement for the integrity of the image's main outline is high, the weight of the low-frequency components is increased; when the requirement for local texture and edge sharpness is high, the weight of the enhanced high-frequency components is increased. The weight determination process is as follows: First, the structural fidelity results and detail enhancement results of sample images under different weight combinations are statistically analyzed. Then, the weight combination with complete main outline, sharp edges, continuous texture, and no over-enhanced artifacts is selected as the actual weights used. After the weighted summation is completed, the images of each frame are rearranged according to their original chronological order to form the enhanced detail sequence. As can be seen from the above, in this embodiment, by extracting edge details, multi-scale decomposition, high-frequency threshold screening, contrast mapping enhancement, and low-frequency and enhanced high-frequency reconstruction processing, the targeted enhancement of detail information in the denoised frame sequence is completed, so that the image can improve edge clarity and local texture recognition while maintaining the stability of the main structure.

[0050] like Figure 6 As shown, S5 includes acquiring an enhanced detail sequence, extracting high-frequency components of the enhanced detail sequence using a high-pass filter operator to generate a gradient feature map; identifying a set of pixels with a grayscale change rate lower than the sharpness standard based on the gradient feature map, and mapping the set of pixels to a blurry region distribution matrix; calculating the proportion of the blurry region distribution matrix, and if the proportion is higher than a preset threshold, constructing a loss function model; performing backpropagation based on the loss function model to obtain a corrected gradient vector, and using the corrected gradient vector to adjust the pixel values ​​of the enhanced detail sequence to obtain a sharpened sequence.

[0051] In this embodiment, an enhanced detail sequence is first obtained, and each frame in the enhanced detail sequence is used as the object for blurry region identification and sharpening processing. The enhanced detail sequence is an image sequence formed after edge detail extraction, multi-scale decomposition, high-frequency enhancement, and weighted reconstruction. The edge contours, local textures, and detail transition information in this sequence have been enhanced, making it suitable as the input basis for subsequent sharpness discrimination and pixel correction. Further, a high-pass filter operator is used to extract the high-frequency components of the enhanced detail sequence to generate a gradient feature map. Specifically, a high-pass filter is first performed on each frame in the enhanced detail sequence to extract information about regions with concentrated brightness changes, obvious edge transitions, and prominent texture responses. Then, the high-frequency response intensity and direction distribution corresponding to each pixel position are summarized to form a gradient feature map. The selection of the high-pass filter operator is based on the accuracy of edge response extraction, texture change separation, and local detail localization, ensuring a clear distinction between sharp edge regions and blurry transition regions in the gradient feature map. Therefore, in this embodiment, by extracting high-frequency components through a high-pass filter operator, the initial separation of sharp and blurry structures in the enhanced detail sequence is achieved.

[0052] Furthermore, based on the gradient feature map, a set of pixels with a grayscale change rate lower than the sharpness standard is identified, and this set of pixels is mapped to a fuzzy region distribution matrix. The specific process is as follows: First, each pixel position in the gradient feature map is analyzed point-by-point, and the grayscale change amplitude, edge transition steepness, and gradient response concentration of the corresponding position within a local range are statistically analyzed. Then, the results are compared item by item with the sharpness standard. When the grayscale change rate of a certain pixel position is lower than the sharpness standard, that pixel position is directly classified into the fuzzy pixel set. The sharpness standard is determined based on the edge grayscale change range, texture transition intensity, and local structural response level in the sharp image samples. Specifically, the determination process involves: first, extracting the gradient change results of the target edge region and texture region from multiple sets of sharp image samples; then, statistically analyzing the grayscale change rate distribution of the sharp region under different structural types; and finally, selecting the boundary result that can characterize the minimum change requirement for sharp edges as the sharpness standard. If different types of target edges exist in the scene to be processed, the grayscale change patterns of each type of edge are statistically analyzed, and the lowest sharpness requirement is used as the unified standard. After pixel identification, all blurred pixels are mapped according to their original spatial locations to generate a blurred region distribution matrix with the same size as the image. Blurred locations in the distribution matrix are marked as blurred regions, and unmarked locations are identified as unblurred regions. It should be noted that this application identifies blurred pixel sets through gradient feature maps, not based on a single brightness value, but on the relationship between the local grayscale change rate and the structural clarity standard. Therefore, the blurred region identification results have a clear structural basis.

[0053] In some embodiments, after obtaining the blurred region distribution matrix, the proportion of the blurred region distribution matrix is ​​calculated, and it is determined whether the proportion is higher than a preset threshold. The specific process is as follows: first, the number of pixels marked as blurred regions in the blurred region distribution matrix is ​​counted; then, the total number of pixels in the current image frame is counted; subsequently, the ratio of the number of blurred region pixels to the total number of pixels is calculated to obtain the proportion of the blurred region distribution matrix. The proportion represents the actual range occupied by the blurred region in the current image frame. The preset threshold is determined based on the triggering conditions of subsequent sharpening processing. Specifically, the determination process is as follows: first, the defect identification results, edge recognition results, and detail preservation results of multiple sets of sample images under different blurred areas are statistically analyzed; then, it is determined that when the blurred region reaches a certain coverage range, it will have a significant impact on the subsequent detection results, and the proportion corresponding to this coverage range is used as the preset threshold. When the proportion is higher than the preset threshold, it is determined that the current image frame as a whole has a blurred effect that requires sharpening correction; when the proportion is not higher than the preset threshold, it is determined that the current image frame maintains its existing enhancement results. Therefore, in this embodiment, the determination of whether to proceed with subsequent sharpening processing is based on the proportion of blurred regions, ensuring that the pixel correction process is based on the overall blur level meeting the processing conditions.

[0054] Furthermore, when the proportion exceeds a preset threshold, a loss function model is constructed. The specific process is as follows: First, based on the gradient feature distribution, edge continuity distribution, texture preservation distribution, and blurred region distribution results of the current image frame in the enhanced detail sequence, a pixel correction target is established. Then, around the pixel correction target, clear edge recovery constraints, texture continuity preservation constraints, structural integrity preservation constraints, and local over-enhancement suppression constraints are set to form a loss function model. Specifically, the clear edge recovery constraint is used to make the edge gradient within the blurred region approach the clear standard; the texture continuity preservation constraint is used to ensure that the corrected texture distribution is consistent with adjacent non-blurred regions; the structural integrity preservation constraint is used to prevent the correction process from destroying the main outline and region boundaries; and the local over-enhancement suppression constraint is used to prevent individual pixel positions from being over-corrected, resulting in abnormal sharpening. The weights of each constraint term in the loss function model are jointly determined based on the edge recovery priority, texture preservation priority, and structural stability priority. When the target boundary is clear and the defect contour recognition requirements are strict, the weight of the edge recovery constraint is increased; when the target surface texture continuity requirements are strict, the weight of the texture continuity preservation constraint is increased; and when the image main structure stability requirements are strict, the weight of the structural integrity preservation constraint is increased. The aforementioned weights were determined by comparing the edge sharpness, texture distortion, and structural deviation of the sample images under different correction results, and the weight combination that simultaneously met the requirements of sharpness improvement, texture continuity, and structural stability was selected as the actual application result.

[0055] Furthermore, backpropagation is performed based on the loss function model to obtain a corrected gradient vector. This corrected gradient vector is then used to adjust the pixel values ​​of the enhanced detail sequence, resulting in a sharpened sequence. Specifically, the enhancement result of the current image frame is first input into the loss function model to calculate the deviation between the current image state and the target sharp state. This deviation is then propagated layer by layer along the pixel response direction to determine the influence of each pixel position on the overall deviation, forming a corrected gradient vector. The corrected gradient vector represents the adjustment direction and magnitude of each pixel position in the sharpening direction. Subsequently, pixel values ​​in blurred regions are corrected point-by-point according to the corrected gradient vector. Positions with slow grayscale transitions are adjusted towards edges with clear transitions, while positions with blurred textures are adjusted towards texture continuity and clear structure. During pixel value adjustment, the center of the blurred region is corrected first, followed by continuous correction of adjacent transition positions to ensure consistent structural continuity within the local area. For non-blurred regions and regions that have already reached the sharpness standard, the original enhancement result remains unchanged. After correcting the pixel values ​​of all blurred regions, the images are rearranged in their original chronological order to form a sharpened sequence.

[0056] like Figure 7 As shown, S6 includes acquiring a sharpened video sequence containing the object to be detected, extracting the target frame and neighboring reference frames; constructing an optical flow motion vector field based on the target frame and neighboring reference frames, and generating a motion-compensated aligned frame; generating a spatiotemporal fusion weight matrix based on the grayscale difference between the target frame and the motion-compensated aligned frame, and obtaining a fused enhanced frame using the spatiotemporal fusion weight matrix; and removing abrupt noise points based on the fused enhanced frame to determine the real information frame used for defect detection.

[0057] In this embodiment, a sharpened video sequence containing the object to be detected is first acquired, and the target frame and neighboring reference frames are extracted from the sharpened video sequence. The sharpened video sequence is a continuous image sequence obtained after prior blurred region identification, pixel value correction, and sharpening processing; the main outline, local edges, and texture information in this sequence have been enhanced. The target frame is the frame currently being processed for real information extraction, and the neighboring reference frames are adjacent frames located before and after the target frame and having a continuous temporal relationship with it. Furthermore, the extraction of the target frame and neighboring reference frames is performed according to the original temporal order, and a preset number of reference frames are selected forward and backward from the target frame to form a local frame group for temporal fusion. The number of neighboring reference frames is determined based on the continuity of target motion, video frame rate, and the stability of defect details. When the target motion is continuous and the video frame rate is high, the number of neighboring reference frames is determined according to the principle of temporal adjacency; when the defect boundary changes are small and the texture continuity is stable, the coverage of the neighboring reference frames is further expanded to ensure complete extraction of temporal information. Therefore, in this embodiment, by extracting the target frame and the neighboring reference frame, a temporal correlation basis is established between the current frame to be detected and the adjacent time frame, providing input conditions for subsequent motion compensation and spatiotemporal fusion.

[0058] Furthermore, an optical flow motion vector field is constructed based on the target frame and neighboring reference frames to generate a motion-compensated aligned frame. The specific process is as follows: First, point-by-point matching is performed on the corresponding pixel positions, local edge distributions, texture structures, and contour changes between the target frame and each neighboring reference frame. The direction and amount of position change of each pixel in the reference frame relative to the target frame are extracted, and the displacement results of all pixel positions are summarized to form an optical flow motion vector field between the target frame and the neighboring reference frames. The optical flow motion vector field characterizes the motion change relationship corresponding to the mapping of each position in the neighboring reference frames to the position in the target frame. Further, position callback is performed on each pixel position in the neighboring reference frames based on the optical flow motion vector field, aligning the structural contours, edge positions, and texture distributions in the neighboring reference frames to the target frame, generating a motion-compensated aligned frame. This motion-compensated aligned frame is a reference image that, after displacement correction, maintains spatial consistency with the target frame. The construction of the optical flow motion vector field is based on the local motion consistency and structural continuity between consecutive frames. Therefore, when generating motion-compensated aligned frames, position callbacks are first performed on regions with clear contours and continuous textures, and then continuous corrections are performed on edge transition regions to ensure that the alignment result is structurally consistent with the target frame. It should be noted that this application achieves motion-compensated alignment from the neighboring reference frame to the target frame through the optical flow motion vector field, so that effective detail information acquired at different times is mapped to a unified spatial position, thereby eliminating the influence of temporal displacement on the subsequent fusion results.

[0059] In some embodiments, a spatiotemporal fusion weight matrix is ​​generated based on the grayscale difference between the target frame and the motion-compensated aligned frame. The specific process is as follows: First, the grayscale values ​​of the target frame and the motion-compensated aligned frame at the same position are compared pixel-by-pixel to obtain the grayscale difference distribution at the corresponding position. Then, based on the magnitude of the grayscale difference, the continuity of local structure, and temporal consistency, corresponding fusion weights are assigned to each pixel position to form a spatiotemporal fusion weight matrix. The grayscale difference characterizes the degree of information consistency between the target frame and the motion-compensated aligned frame at the same spatial position. The smaller the grayscale difference, the higher the content consistency at that position across different time frames, and the higher the fusion weight at that position. Conversely, the larger the grayscale difference, the more local changes, residual misalignment, or abnormal responses exist at that position, and the lower the fusion weight at that position. The weights at each position in the spatiotemporal fusion weight matrix are simultaneously constrained by temporal proximity and spatial structural consistency. Neighboring reference frames that are closer to the target frame in time have their corresponding weights increased preferentially; positions with clear edge continuity and complete texture continuity in space have their corresponding weights further increased. The weight allocation criteria are jointly determined based on the consistent grayscale distribution of stable regions in continuous samples, the edge preservation requirement of defective regions, and the exclusion requirement of abrupt change regions. This ensures that the pixel positions participating in the fusion maintain both temporal continuity and spatial structural consistency. Therefore, in this embodiment, by constructing a spatiotemporal fusion weight matrix using grayscale differences, quantitative allocation of information from multiple frames at a unified spatial location is achieved, allowing reliable temporal information to participate in enhancement while restricting abnormal temporal information from participating in the fusion.

[0060] Furthermore, a fused enhanced frame is obtained using a spatiotemporal fusion weight matrix. The specific process is as follows: first, the pixel values ​​at the same positions in the target frame and each motion-compensated aligned frame are weighted and superimposed according to the spatiotemporal fusion weight matrix; then, the fusion results at each position are normalized to form the fused enhanced frame. The fused enhanced frame is the enhanced image result obtained after spatial alignment and weighted fusion of multi-frame temporal information in the target frame coordinate system. For positions in the target frame with insufficient local details but complete corresponding details in neighboring reference frames, the spatiotemporal fusion weight matrix incorporates effective information from the neighboring reference frames into the fusion result; for positions in the target frame that are already clear but have local perturbations in the neighboring reference frames, the spatiotemporal fusion weight matrix retains the original information of the target frame as the main content. Further, after weighted superposition, a local consistency check is performed on each pixel position in the fused enhanced frame to confirm whether edge continuity, texture continuity, and grayscale transitions remain stable, ensuring that the fusion result is structurally consistent with the main information of the target frame. Through the above processing, the fused enhanced frame simultaneously retains the main structural information of the target frame and the supplementary detail information from the neighboring reference frames.

[0061] Furthermore, based on the removal of abrupt noise from the fused enhanced frame, the true information frame for defect detection is determined. The specific process is as follows: First, the fused enhanced frame is scanned region by region to identify abnormal pixel sets characterized by abrupt grayscale changes, spatial isolation, texture breaks, and unconnected edges. Then, based on the distribution characteristics of these abnormal pixel sets within a local area, isolated responses that do not conform to continuous structural relationships are identified as abrupt noise. The criteria for identifying abrupt noise include sudden grayscale changes, lack of continuous transition with adjacent pixels, absence of structural continuity within the neighborhood, and lack of stable repetition in adjacent time frames. After identifying abrupt noise, the corresponding pixel positions are removed, and consistency compensation is applied to the removal positions based on the grayscale distribution and edge direction of the surrounding continuous structural regions to ensure the processed image maintains local structural continuity. Subsequently, the fused enhanced frame after abrupt noise removal is determined as the true information frame. The true information frame is a detection input frame that simultaneously meets the requirements of temporal consistency, spatial alignment, detail enhancement, and abrupt noise removal. As can be seen from the above, in this embodiment, the determination of real information frames is completed by extracting target frames, selecting neighboring reference frames, constructing optical flow motion vector fields, aligning motion compensation, allocating spatiotemporal fusion weights, enhancing multi-frame fusion, and removing abrupt noise. This enables subsequent defect detection to be based on images with stable structures, complete details, and controlled noise.

[0062] like Figure 8 As shown, S7 includes acquiring real information frames for defect detection; extracting an initial defect mask containing abnormal textures based on the gray-level gradient distribution of the real information frames; if the initial defect mask has discontinuous boundaries, extracting local image patches and inputting the boundary to complete the deep neural network to generate predicted boundary paths connecting the break points; setting adaptive morphological structural elements based on the predicted boundary paths, performing closing operations and texture synthesis filling on the initial defect mask to obtain a corrected defect mask with a complete shape; mapping the corrected defect mask back to the original time coordinates and reconstructing the frame data to generate an accurate detection sequence with continuous boundary features.

[0063] In this embodiment, a real information frame for defect detection is first acquired and used as the input for defect region identification and boundary correction. The real information frame is the detection input frame obtained after pre-sequence temporal fusion, motion compensation, and abrupt noise removal. The target region in this frame has a clear structure, continuous edges, and stable local texture, meeting the image quality requirements for defect identification. Further, an initial defect mask containing abnormal textures is extracted based on the gray-level gradient distribution of the real information frame. Specifically, the gray-level change intensity and direction are calculated for each pixel position in the real information frame. Then, the gradient concentration, direction abruptness, and texture distribution continuity within the local area are statistically analyzed to identify abnormal regions that do not conform to the normal surface texture distribution. These abnormal regions exhibit abnormal gray-level transitions, abrupt texture direction changes, local gradient aggregation, or disordered edge distribution. After identifying abnormal regions, the corresponding pixel positions are marked as candidate defect regions, and all candidate defect regions are aggregated according to their spatial location to generate an initial defect mask. The initial defect mask has the same spatial dimensions as the real information frame; the mask-covered area corresponds to the abnormal texture region, and the mask-uncovered area corresponds to the normal texture region. The criteria for determining abnormal textures are based on the texture gradient distribution, edge continuity distribution, and surface transition patterns in normal samples, with the gradient stability range of the normal region serving as the basis for identification. Therefore, in this embodiment, the initial defect mask is extracted using the grayscale gradient distribution of the real information frame, achieving a clear separation between abnormal texture regions and normal surface regions.

[0064] Furthermore, after obtaining the initial defect mask, it is determined whether the initial defect mask has boundary discontinuities. The specific process is as follows: first, a segment-by-segment scan is performed along the edge contour of the initial defect mask; then, the spatial connectivity, directional continuity, and contour continuity of the boundary lines are statistically analyzed. When there are interruptions, gaps, local breaks, or missing contour connections in the boundary lines, it is directly determined that the initial defect mask has boundary discontinuities. The criteria for determining boundary discontinuities include the loss of connectivity between boundary curves in adjacent areas, consistent directional extension at both ends of the break and lack of boundary response in the middle area, sudden interruption of the boundary width, and incomplete or missing contour shapes. The criteria for determining boundary discontinuities are determined based on the range of boundary length variation, directional extension, and break spacing distribution in continuous defect edge samples. When the break spacing is within the connectable range of the defect boundary and there is a consistent edge extension direction at both ends of the break, the area is determined to be a region to be filled. It should be noted that this application does not judge boundary discontinuities based on a single pixel interruption result, but rather on the overall boundary direction, local connectivity, and spatial distribution of the break segments. Therefore, the boundary discontinuity recognition result has a clear contour basis.

[0065] In some embodiments, if the initial defect mask has discontinuous boundaries, local image patches are extracted and input into a boundary completion deep neural network to generate predicted boundary paths connecting the break points. Specifically, the process involves: first, taking the break point as the center, synchronously extracting corresponding local image patches from the real information frame and the initial defect mask according to a preset spatial range. Each local image patch simultaneously contains known edge information on both sides of the break boundary, surrounding texture distribution information, and local structural information of the defect region. The extraction range of the local image patch is determined based on the break spacing, the degree of boundary curvature, and the influence range of local textures. When the break spacing is large, the extraction range extends outwards from both ends of the break to completely cover the boundary extension; when the boundary curvature is significant, the extraction range covers the complete turning area to maintain the continuity of the contour; when local texture interference is strong, the extraction range covers the surrounding stable texture area to enhance boundary prediction constraints. After extracting the local image patches, they are input into the boundary completion deep neural network. The boundary completion deep neural network generates predicted boundary paths connecting the break points based on the direction information, spacing information, curvature change information, and local texture continuity of the known boundary segments. The predicted boundary paths characterize the connection positions, extension directions, and contour morphology between fracture boundaries. The boundary completion deep neural network is trained based on complete defect boundary samples, local fracture boundary samples, and corresponding real connection path samples, ensuring that the network output meets the requirements of consistent boundary direction, accurate connection positions, and continuous contour morphology. Therefore, in this embodiment, the predicted boundary paths generated by the boundary completion deep neural network achieve targeted recovery of the connection relationships between fracture boundaries.

[0066] Furthermore, adaptive morphological structural elements are set based on the predicted boundary path, and closing operations and texture synthesis filling are performed on the initial defect mask to obtain a morphologically complete corrected defect mask. The specific process is as follows: First, adaptive morphological structural elements are set based on the extension direction, boundary width, and curvature change of the predicted boundary path. The size of the adaptive morphological structural element is determined based on the fracture spacing and boundary width, the direction is determined based on the main extension direction of the predicted boundary path, and the shape is determined based on the local boundary curvature change. When the fracture spacing is narrow, the structural element size is set according to the local boundary width; when the fracture spacing is wide, the structural element size covers the fracture connection range; when the boundary extends linearly, the structural element is set according to a strip shape; when the boundary extends curvedly, the structural element is set according to an arc-shaped extension trend. After setting the adaptive morphological structural elements, a closing operation is performed on the initial defect mask to fill the boundary fracture area and restore the overall connectivity of the defect area. After the closing operation is completed, texture synthesis filling is performed based on the predicted boundary path and the surrounding normal texture distribution. Specifically, the process involves first extracting the texture distribution features adjacent to the defect boundary around the fractured area. Then, texture filling is performed within the fractured area according to the region defined by the predicted boundary path, ensuring that the filled area maintains consistency with adjacent defect areas in terms of boundary connection, texture direction, and grayscale transition. The constraints of texture synthesis filling are determined based on the texture direction distribution, brightness variation patterns, and edge transition relationships of neighboring areas, thereby ensuring that the corrected mask has complete boundaries, closed regional shapes, and coherent local structures. After the above processing, a corrected defect mask with a complete shape is obtained.

[0067] Furthermore, the defect correction mask is mapped back to its original time coordinates and the frame data is reassembled to generate an accurate detection sequence with continuous boundary features. The specific process is as follows: First, based on the temporal position of the real information frame within the entire video sequence, the defect correction mask corresponding to the current frame is restored to its corresponding frame position in the original temporal order. Then, according to the temporal correlation between adjacent frames, the defect correction masks corresponding to consecutive moments are arranged frame by frame and temporally reassembled. If the same defect region in adjacent frames has continuous spatial displacement, the corresponding position mapping of the defect correction mask is performed based on the previous motion compensation result, ensuring that the same defect maintains a temporally consistent boundary tracking relationship between adjacent frames. After mapping, the defect correction mask corresponding to each moment is combined with the original detection frame data to generate an accurate detection sequence with continuous boundary features. In this accurate detection sequence, the defect boundaries maintain a continuous extension relationship in the temporal dimension, a complete closure relationship in the spatial dimension, and an abnormal feature expression consistent with the original defect region in terms of texture distribution. As can be seen from the above, in this embodiment, the defect region boundary correction and detection sequence reconstruction are completed by acquiring real information frames, extracting initial defect masks, identifying boundary discontinuities, cropping local image blocks, generating predicted boundary paths, setting adaptive morphological structural elements, performing closing operations and texture synthesis filling, as well as mapping the original time coordinates and reassembling frame data, thereby obtaining an accurate detection sequence with continuous boundary features.

[0068] Although embodiments of the invention have been shown and described, it will be understood by those skilled in the art that various changes, modifications, substitutions and alterations can be made to these embodiments without departing from the principles and spirit of the invention, the scope of which is defined by the appended claims and their equivalents.

Claims

1. A motion blur compensation image enhancement method for high-speed cameras, characterized in that, include: S1. Obtain the raw sequence data recorded by the high-speed imaging device, analyze the overlapping area of ​​adjacent frames in the sequence through a temporal convolutional network, determine the temporal continuity features corresponding to highly overlapping frames, and obtain the separated frame sequence. S2. Based on the separated frame sequence, the optical flow estimation algorithm is used to track the object's movement trajectory. If the optical flow vector exceeds the preset threshold, it is judged as an interference factor in the dynamic scene, and the motion-compensated frame sequence is obtained. S3. Obtain the pixel offset value caused by light jitter through the frame sequence after motion compensation, and determine whether the offset value is within the preset range. If the offset value exceeds the range, apply differential filtering to determine the denoised frame sequence. S4. Extract the instantaneously changing edge details from the denoised frame sequence, and use a multi-scale decomposition method to separate low-frequency and high-frequency components to obtain an enhanced detail sequence; S5. Based on the enhanced detail sequence, determine the distribution of blurred areas in the image. If the proportion of blurred areas is higher than a preset threshold, adjust the pixel values ​​through backpropagation to obtain a sharpened sequence. S6. Obtain the time frames in the sharpened sequence, and perform interpolation fusion by combining the temporal continuity features to determine the real information frames required for defect detection. S7. Analyze surface defects using real information frames. If the boundaries are discontinuous, use deep learning to detect the defect boundaries, fill the discontinuous areas, and generate a detection sequence.

2. The image enhancement method for motion blur compensation of high-speed cameras according to claim 1, characterized in that: S1 includes: Acquire the raw sequence data recorded by the high-speed imaging device and divide the raw sequence data into multiple groups of adjacent images; Adjacent frames are input into a temporal convolutional network to generate convolutional feature maps, and overlapping frames are determined based on the convolutional feature maps. Extract deep temporal dependencies of overlapping frames to construct temporal continuity features. If the inter-frame correlation degree corresponding to the temporal continuity features exceeds the preset standard, a redundant frame marker is generated. By removing frames with redundant frame markers, a separate frame sequence is obtained.

3. The image enhancement method for motion blur compensation of high-speed cameras according to claim 1, characterized in that: S2 includes: Obtain the original separation frame sequence and generate dense optical flow field data from the original separation frame sequence; If the magnitude of the optical flow vector in the dense optical flow field data exceeds a preset threshold, it is identified as an interference factor. An interference region mask is constructed based on the interference factors, and the background optical flow vector set is extracted from the dense optical flow field data using the interference region mask. The original separated frame sequence is reversed geometrically based on the background optical flow vector set to obtain the motion-compensated frame sequence.

4. The image enhancement method for motion blur compensation of high-speed cameras according to claim 1, characterized in that: S3 includes: Obtain the motion-compensated frame sequence; The brightness gradient change matrix is ​​extracted from the motion-compensated frame sequence, and the pixel offset value is separated from the brightness gradient change matrix. Determine whether the pixel offset value is within the preset offset threshold range; If the pixel offset value exceeds the offset threshold range, then the spatial neighborhood feature matrix is ​​extracted; The denoised frame sequence is determined by applying differential filtering based on the spatial neighborhood feature matrix and pixel offset values.

5. The image enhancement method for motion blur compensation of high-speed cameras according to claim 1, characterized in that: S4 includes: Edge details are extracted, which are obtained by performing pixel interpolation on the denoised frame sequence; Low-frequency and high-frequency components are separated, and the low-frequency and high-frequency components are obtained by multi-scale decomposition of edge details; An enhanced high-frequency component is obtained, which is generated by contrast mapping when the high-frequency component meets a preset threshold. The enhanced detail sequence is obtained by weighted summation of the low-frequency components and the enhanced high-frequency components.

6. The image enhancement method for motion blur compensation of high-speed cameras according to claim 1, characterized in that: S5 includes: The enhanced detail sequence is obtained, and the high-frequency components of the enhanced detail sequence are extracted using a high-pass filter operator to generate a gradient feature map; Identify sets of pixels whose grayscale change rate is lower than the sharpness standard based on gradient feature maps, and map these sets of pixels into a fuzzy region distribution matrix. Calculate the proportion of the fuzzy region distribution matrix. If the proportion is higher than a preset threshold, construct a loss function model. The backpropagation operation is performed based on the loss function model to obtain the corrected gradient vector. The corrected gradient vector is then used to adjust the pixel values ​​of the enhanced detail sequence to obtain the sharpened sequence.

7. The image enhancement method for motion blur compensation of high-speed cameras according to claim 1, characterized in that: S6 includes: Obtain a sharpened video sequence containing the object to be detected, and extract the target frame and neighboring reference frames; An optical flow motion vector field is constructed based on the target frame and neighboring reference frames to generate motion-compensated aligned frames.

8. The image enhancement method for motion blur compensation of high-speed cameras according to claim 7, characterized in that: S6 further includes: A spatiotemporal fusion weight matrix is ​​generated based on the grayscale difference between the target frame and the motion-compensated aligned frame, and the fused enhanced frame is obtained using the spatiotemporal fusion weight matrix. By eliminating abrupt noise based on the fused and enhanced frames, the true information frames used for defect detection are determined.

9. The image enhancement method for motion blur compensation of high-speed cameras according to claim 1, characterized in that: S7 includes: Acquire real information frames for defect detection, and extract an initial defect mask containing abnormal textures based on the gray-level gradient distribution of the real information frames; If the initial defect mask has discontinuous boundaries, a local image patch is extracted as input boundary to complete the deep neural network and generate a predicted boundary path connecting the break points.

10. The image enhancement method for motion blur compensation of high-speed cameras according to claim 9, characterized in that: The S7 also includes: Based on the predicted boundary path, adaptive morphological structural elements are set, and a closing operation and texture synthesis filling are performed on the initial defect mask to obtain a morphologically complete corrected defect mask. The defect mask is mapped back to the original time coordinates and the frame data is reassembled to generate an accurate detection sequence with continuous boundary features.

Images