A machine vision-based vehicle-mounted AI camera target detection method

Abstract

The application discloses a kind of vehicle-mounted AI camera target detection methods based on machine vision, comprising the following steps: step one: generating binary fuzzy area mask;Step two: get complete complete image;Step three: generate fusion feature map;Step four: the fusion feature map is input into improved InternImage network, introduce physical field coupling encoder to carry out field propagation calculation, and binary fuzzy area mask is mapped as energy source map and participates in field propagation calculation, and output enhanced feature map;Step five: generate preliminary target set;Step six: the original image and last frame image are input into RAFT light flow algorithm, generate dense light flow chart, and the position of candidate target area is corrected, and get compensation target set;Step seven: generate credible target set;Step eight: output final detection result.The application combines improved InternImage network and RAFT light flow algorithm, and realizes accurate target detection.

Description

Technical Field

[0001] This invention relates to the field of computer vision technology, and in particular to a target detection method for vehicle-mounted AI cameras based on machine vision. Background Technology

[0002] With the development of intelligent driving technology and the widespread application of vehicle-mounted visual perception systems, target detection methods based on forward-looking cameras have become an important component for realizing environmental perception, path planning, and driving decision-making. Existing vehicle-mounted target detection methods typically rely on deep convolutional neural networks to process single-frame images to extract salient target regions and output corresponding location information and confidence scores. However, in real-world road scenarios, images captured by vehicle-mounted cameras are often blurred or lack detail due to factors such as vehicle shaking, dynamic occlusion, focal length changes, and rain / fog interference, which affects the input quality and result stability of the detection network. Existing technologies generally suffer from the following problems:

[0003] The processing of blurred regions in images relies on end-to-end unified network modeling, which lacks a targeted compensation mechanism, resulting in unclear target boundaries and fragmented structures, affecting the accuracy of target localization. In the process of processing multiple consecutive images, most existing methods ignore the impact of cross-frame target displacement on confidence evaluation, making it difficult to distinguish between stable targets and abnormal moving regions, resulting in discontinuous detection results in the time dimension. In the target screening and redundancy integration stages, simple non-maximum suppression strategies are usually used to process overlapping regions, failing to dynamically adjust in conjunction with the changes in confidence, resulting in the wrong deletion of some high-confidence targets.

[0004] Therefore, how to provide a machine vision-based target detection method for in-vehicle AI cameras is a problem that urgently needs to be solved by those skilled in the art. Summary of the Invention

[0005] One objective of this invention is to propose a target detection method for vehicle-mounted AI cameras based on machine vision. This invention combines an improved InternImage network with the RAFT optical flow algorithm, introduces a physical field coupled encoder for field propagation calculation, and uses dense optical flow maps to perform temporal displacement compensation for candidate target regions. This enables stable and accurate target detection in complex dynamic scenes, effectively improving the spatial consistency, temporal continuity, and overall reliability of target detection results.

[0006] A target detection method for an in-vehicle AI camera based on machine vision according to an embodiment of the present invention includes the following steps:

[0007] Step 1: Acquire the original image of the current frame captured by the vehicle camera, and acquire the corresponding previous frame image, perform grayscale change analysis, and generate a binary blurred region mask;

[0008] Step 2: Based on the binary blurred region mask, extract the corresponding image block to be completed from the original image, input the image block to be completed into the diffusion inversion model for reverse diffusion processing to generate the completed image block, and fill the completed image block back into the corresponding position of the original image to obtain the complete image;

[0009] Step 3: Extract features from the original image and the complete completed image respectively, and then stitch them together to generate a fused feature map;

[0010] Step 4: Input the fused feature map into the improved InternImage network. The improved InternImage network introduces a physical field coupled encoder to perform field propagation calculation, and maps the binary blurred region mask into an energy source map to participate in the field propagation calculation, and outputs an enhanced feature map.

[0011] Step 5: Perform multi-scale feature parsing based on the enhanced feature map to generate a preliminary target set including candidate target regions and corresponding initial confidence levels;

[0012] Step 6: Input the original image and the previous frame image into the RAFT optical flow algorithm to generate a dense optical flow map, and correct the position of the candidate target regions in the preliminary target set according to the dense optical flow map to obtain the compensated target set;

[0013] Step 7: Evaluate the credibility of candidate target regions in the compensation target set and update the corresponding initial confidence to generate a credible target set;

[0014] Step 8: Filter and integrate the set of trusted targets, and output the final detection result including target location and target confidence level.

[0015] Optionally, step one specifically includes:

[0016] Acquire the current frame's raw image and the corresponding previous frame image captured by the vehicle-mounted camera;

[0017] The original image of the current frame is divided into multiple non-overlapping image sub-regions, each of which contains the same number of pixels.

[0018] For each image sub-region, the adjacent pixels within the image sub-region are traversed sequentially, the difference in gray values ​​between adjacent pixels is calculated and accumulated to obtain a first value;

[0019] For the same sub-region of an image, the difference between the maximum and minimum gray values ​​is calculated to obtain the second value;

[0020] The image sub-region in the current frame's original image is compared pixel by pixel with the corresponding image sub-region in the previous frame's image, and the average value of the grayscale difference of the corresponding pixels is calculated to obtain the third value.

[0021] The first average value is obtained by averaging the first values ​​of all image sub-regions in the original image of the current frame; the second average value is obtained by averaging the second values ​​of all image sub-regions; and the third average value is obtained by averaging the third values ​​of all image sub-regions.

[0022] When the first value of the image sub-region is less than the first average value and the second value is less than the second average value, or when the third value of the image sub-region is greater than the third average value, the blur region mask value corresponding to the image sub-region is set to 1; otherwise, the blur region mask value corresponding to the image sub-region is set to 0.

[0023] The blurry region mask values ​​corresponding to all image sub-regions are combined to form a binary blurry region mask corresponding to the spatial position of the original image in the current frame.

[0024] Optionally, step two specifically involves:

[0025] Based on the binary blurred region mask, image sub-regions with a blurred region mask value of 1 are located in the original image, and image sub-regions with common pixel boundaries at the boundary positions are merged according to the pixel spatial connectivity relationship to form multiple independent regions to be completed.

[0026] Based on the spatial position of each region to be completed in the original image, multiple image blocks corresponding to each region to be completed are cropped from the original image, and the position coordinates of each image block to be completed in the original image are recorded.

[0027] Each image block to be completed is input into the diffusion inversion model, which includes a forward diffusion module, a feature encoding module, and a backward diffusion module.

[0028] The forward diffusion module performs a multi-step noise superposition operation on the image block to be completed. In each diffusion step, random perturbations are superimposed on the pixel values ​​of the image block to be completed to obtain the intermediate image state of the corresponding diffusion step. After completing all diffusion steps, the noise representation of the image block to be completed is generated.

[0029] The feature encoding module performs convolutional feature extraction and hierarchical feature mapping on the image block to be completed, generating a structural feature representation corresponding to the image block to be completed.

[0030] The back diffusion module takes the noise representation as the initial input and combines it with the structural feature representation. It performs multi-step denoising processing according to the diffusion step sequence opposite to that of the forward diffusion module. In each back diffusion step, the intermediate image state is updated according to the intermediate image state and structural feature representation corresponding to the current diffusion step. After completing all back diffusion steps, the module outputs the completed image block corresponding to each image block to be completed.

[0031] Based on the position coordinates of each image block to be completed in the original image, the corresponding completed image blocks are backfilled into the positions in the original image corresponding to each region to be completed. The pixel values ​​in the completed image blocks are then used to replace the pixel values ​​at the corresponding positions in the original image to obtain a complete completed image.

[0032] Optionally, step three specifically includes:

[0033] The original image is processed by three layers of spatial convolution. In the first layer of spatial convolution, a 3×3 pixel neighborhood around each pixel is selected for convolution calculation, and a downsampling operation with a stride of 2 is performed to obtain the first layer of the original image feature map.

[0034] In the second layer of spatial convolution processing, the first layer of original image feature map is used as input, a 3×3 pixel neighborhood is used for convolution calculation, and a downsampling operation with a stride of 2 is performed to obtain the second layer of original image feature map.

[0035] In the third-layer spatial convolution processing, the original image feature map of the second layer is used as input, and convolution calculation is performed using a 3×3 pixel neighborhood to obtain the original image feature map of the third layer.

[0036] The complete image is processed by three spatial convolution layers in sequence, with the same number of spatial convolution layers, pixel neighborhood range and downsampling order as the original image, to obtain the first complete image feature map, the second complete image feature map and the third complete image feature map respectively.

[0037] For original image feature maps and completed image feature maps at the same level, the height and width are adjusted by bilinear interpolation to ensure that the original image feature maps and completed image feature maps at the corresponding level have the same spatial size.

[0038] The original image feature map and the completed image feature map, which are at the same level and have the same spatial size, are stitched together in the channel dimension to obtain the first stitched feature map, the second stitched feature map and the third stitched feature map in sequence.

[0039] The first, second, and third stitched feature maps are adjusted to the same spatial size through interpolation and resampling, and then stitched together in the channel dimension to obtain a multi-layer stitched feature map.

[0040] Channel mapping processing is performed on the multi-layer stitched feature map. The channel mapping processing is a linear transformation based on the 1×1 pixel neighborhood. The multi-layer stitched feature map is mapped in the channel dimension to generate a fused feature map.

[0041] Optionally, step four specifically includes:

[0042] The fused feature map is input into the improved InternImage network, which uses the backbone structure of InternImage as its basic framework. The backbone structure is composed of multiple feature processing layers connected sequentially.

[0043] Each feature processing layer includes a convolutional computation unit and a feature integration unit connected in sequence. The convolutional computation unit performs convolutional computation on the input feature map, with the convolutional kernel size set to 3×3 and the stride set to 1, and outputs an intermediate feature map. The feature integration unit concatenates the intermediate feature map with the input feature map of the current feature processing layer in the channel dimension to obtain a combined feature map, and performs convolutional computation on the combined feature map to obtain the output feature map of the current feature processing layer.

[0044] Within the backbone structure of the improved InternImage network, a physical field coupling encoder is introduced between two adjacent feature processing layers. The output feature map of the previous feature processing layer is input into the physical field coupling encoder before entering the next feature processing layer. After being processed by the physical field coupling encoder, it becomes the input feature map of the next feature processing layer.

[0045] The feature map input to the physical field coupled encoder is modeled as a two-dimensional signal field in the spatial dimension, wherein each spatial position in the feature map corresponds to a field point in the two-dimensional signal field, and the channel vector of the feature map at the corresponding spatial position is used as the initial state vector of the field point.

[0046] The binary blurred region mask is mapped to an energy source map with the same spatial size as the two-dimensional signal field. The spatial position of the blurred region mask with a value of 1 corresponds to an energy value of 1 in the energy source map, and the spatial position of the blurred region mask with a value of 0 corresponds to an energy value of 0 in the energy source map.

[0047] Three field propagation calculations are performed in the physical field coupled encoder, and each field propagation calculation performs an update operation on each field point of the two-dimensional signal field.

[0048] After completing the three field propagation calculations, the updated two-dimensional signal field is arranged in spatial order and restored to the form of a feature map, which is then used as the input feature map for the next feature processing layer.

[0049] After being processed sequentially by the multi-level feature processing layer and the physical field coupled encoder in the improved InternImage network, an enhanced feature map is output.

[0050] Optionally, each field propagation calculation performs an update operation on each field point of the two-dimensional signal field, specifically as follows:

[0051] Select four adjacent field points (up, down, left, and right) centered on the current field point, read the state vectors of the four adjacent field points respectively, and add them together to obtain the neighborhood summation vector. Divide the neighborhood summation vector by 4 to obtain the neighborhood mean vector.

[0052] Subtracting the state vector of the current field point from the neighborhood mean vector yields the diffusion difference vector.

[0053] Multiplying the diffusion difference vector by a set diffusion coefficient yields the diffusion increment vector;

[0054] Read the energy value of the current field point at the corresponding spatial location in the energy source map, where the energy value is the numerical value of the energy source map at the corresponding spatial location.

[0055] Expand the energy value into an energy vector with the same dimension as the current field point state vector;

[0056] The diffusion increment vector is added element-wise to the energy vector to obtain the update increment vector;

[0057] The state vector of the current field point is added element by element to the update increment vector to obtain the updated state vector, and the updated state vector replaces the state vector of the current field point.

[0058] Optionally, step five specifically includes:

[0059] Three sets of convolution calculations with different kernel sizes were performed on the enhanced feature map at the same spatial resolution. The kernel sizes were set to 3×3, 5×5 and 7×7 respectively, and the stride was set to 1 for each, resulting in three scale feature maps corresponding to different receptive ranges.

[0060] For each scale feature map, a sliding window is traversed in the spatial dimension according to a set step size. At each sliding window position, the scale feature map region covered by the current sliding window is defined as a candidate target region.

[0061] For each candidate target region, the feature values ​​at all spatial locations within the candidate target region are read pixel by pixel, and the feature values ​​are added element by element to obtain the region feature accumulation vector.

[0062] The elements in the region feature accumulation vector are summed to obtain the total region response value. The total region response value is then divided by the number of pixels in the candidate target region to obtain the average response value of the corresponding candidate target region, which is used as the initial confidence level of the candidate target region.

[0063] All candidate target regions and their corresponding initial confidence scores obtained from the feature maps at three scales are aggregated to generate a preliminary target set.

[0064] Optionally, step six specifically includes:

[0065] The original image of the current frame and the corresponding image of the previous frame are input into the RAFT optical flow algorithm. Convolutional feature extraction processing is performed on the original image and the image of the previous frame respectively to obtain the feature map of the current frame and the feature map of the previous frame.

[0066] For each spatial coordinate position in the current frame feature map, select all spatial coordinate positions in the previous frame feature map in sequence, and perform a dot product operation between the feature vector at the corresponding spatial coordinate position in the current frame feature map and the feature vector at each spatial coordinate position in the previous frame feature map.

[0067] The scalar value obtained from each dot product operation is stored at a four-dimensional coordinate position determined by the spatial coordinate position in the current frame feature map and the corresponding spatial coordinate position in the previous frame feature map, thus generating a four-dimensional correlation volume.

[0068] In the RAFT optical flow algorithm, the optical flow estimation result is initialized. The optical flow estimation result is a two-dimensional vector field with the same spatial size as the original image of the current frame. During initialization, the two-dimensional displacement vector at each pixel position is set to zero vector. The two-dimensional displacement vector includes the displacement component in the horizontal direction and the displacement component in the vertical direction.

[0069] Based on the four-dimensional correlation volume and optical flow estimation results, multiple iterative updates are performed. In each iteration, for each spatial location in the feature map of the current frame, the two-dimensional displacement vector corresponding to the spatial location is read from the optical flow estimation results.

[0070] Using the spatial position in the current frame feature map as the reference starting position, the reference starting position is translated in the spatial coordinate system along the horizontal and vertical directions according to the two-dimensional displacement vector, which is used as the predicted corresponding position in the previous frame feature map.

[0071] Centered on the predicted location, a defined neighborhood consisting of multiple adjacent spatial locations is selected in the four-dimensional correlation volume, and multiple scalar values ​​corresponding to the neighborhood are read and combined to generate a local correlation feature vector.

[0072] Perform a linear mapping operation on the local correlation feature vector to obtain a two-dimensional optical flow update vector corresponding to the current spatial position;

[0073] The updated two-dimensional optical flow vector is obtained by adding the two-dimensional displacement vector at the corresponding spatial position in the optical flow estimation result element by element.

[0074] Based on the updated two-dimensional displacement vectors at all spatial locations, an updated optical flow estimation result is generated, and the updated optical flow estimation result is used as the input for the next iteration;

[0075] After all iterations are completed, the final optical flow estimation result is used as a dense optical flow map, where each pixel position in the dense optical flow map corresponds to a two-dimensional optical flow vector.

[0076] For each candidate target region in the preliminary target set, all pixel positions within the candidate target region are traversed, and the two-dimensional optical flow vector at the corresponding pixel position in the dense optical flow map is read.

[0077] The average displacement of the candidate target region in the horizontal and vertical directions is obtained by summing all two-dimensional optical flow vectors in the candidate target region and dividing them by the number of pixels in the candidate target region, thus generating the position-corrected candidate target region.

[0078] All candidate target regions after position correction are aggregated to generate a compensation target set.

[0079] Optionally, step seven specifically includes:

[0080] For each candidate target region in the compensation target set, read the initial confidence level of the candidate target region in the preliminary target set;

[0081] Simultaneously, the average horizontal displacement and the average vertical displacement of the candidate target region in the compensation target set are read.

[0082] The absolute values ​​of the average displacement in the horizontal direction and the average displacement in the vertical direction are taken respectively, and the two absolute values ​​are added together to obtain the displacement amplitude value of the candidate target area.

[0083] Based on the relationship between the displacement amplitude value and the set displacement reference value, the confidence of the candidate target region is evaluated. When the displacement amplitude value is greater than the set displacement reference value, the initial confidence of the candidate target region is multiplied by a set reduction factor to obtain the updated confidence.

[0084] When the displacement amplitude value is less than or equal to the set displacement reference value, the initial confidence of the candidate target region is multiplied by the set amplification factor to obtain the updated confidence.

[0085] The candidate target regions that have completed credibility assessment and confidence update, along with their corresponding updated confidence levels, are collectively collected to generate a set of credible targets.

[0086] Optionally, step eight specifically includes:

[0087] All candidate target regions in the set of trusted targets are filtered according to their corresponding updated confidence levels, and candidate target regions whose updated confidence levels are less than a set confidence threshold are deleted.

[0088] The candidate target regions retained after screening are integrated. When two or more candidate target regions overlap in spatial location, each candidate target region is compared as a whole. Only the candidate target region with the highest confidence after the update is retained, and the other candidate target regions that overlap with it are deleted.

[0089] All candidate target regions obtained after screening and integration are taken as the final target regions, and the target location and target confidence corresponding to each final target region are output to generate the final detection result. The target location is the spatial coordinate of the final target region in the original image of the current frame, and the target confidence is the updated confidence corresponding to the final target region.

[0090] The beneficial effects of this invention are:

[0091] This invention addresses key issues in target detection using automotive AI cameras by improving the structural design that combines the InternImage network with a physical field coupled encoder. The improved InternImage network, based on its existing multi-level feature processing layers, introduces a physical field coupled encoder between feature layers. By constructing a two-dimensional signal field and energy source map and performing multiple field propagation calculations, it enhances feature continuity and structural consistency within blurred regions, effectively improving the recognition of moving target regions. In the target detection stage, a multi-scale sliding window is used to extract feature responses from the enhanced feature map, and a preliminary target set is generated by combining the mean of regional responses. A four-dimensional correlation volume is constructed using the RAFT optical flow algorithm, and the optical flow estimation results are iteratively updated to accurately correct the cross-frame position of the target region, resulting in a compensated target set. Furthermore, through dynamic confidence evaluation, displacement reference values ​​and scaling factors are set to update the confidence of candidate target regions and generate a reliable target set. Finally, multi-target integration is achieved through spatial overlap processing and the maximum confidence criterion, outputting a final detection result with high positioning accuracy and reliability. This invention exhibits strong dynamic robustness and detection accuracy in practical automotive applications. Attached Figure Description

[0092] The accompanying drawings are provided to further illustrate the invention and form part of the specification. They are used in conjunction with embodiments of the invention to explain the invention and do not constitute a limitation thereof. In the drawings:

[0093] Figure 1 This is an overall flowchart of a machine vision-based target detection method for vehicle-mounted AI cameras proposed in this invention.

[0094] Figure 2 This is a schematic diagram of the improved InternImage network structure of a machine vision-based vehicle AI camera target detection method proposed in this invention.

[0095] Figure 3 This is a schematic diagram illustrating the process of generating dense optical flow graphs based on the RAFT optical flow algorithm in a machine vision-based vehicle AI camera target detection method proposed in this invention. Detailed Implementation

[0096] The present invention will now be described in further detail with reference to the accompanying drawings. These drawings are simplified schematic diagrams, illustrating only the basic structure of the invention, and therefore only show the components relevant to the invention.

[0097] refer to Figures 1-3 A target detection method for vehicle-mounted AI cameras based on machine vision includes the following steps:

[0098] Step 1: Acquire the original image of the current frame captured by the vehicle camera, and acquire the corresponding previous frame image, perform grayscale change analysis, and generate a binary blurred region mask;

[0099] Step 2: Based on the binary blurred region mask, extract the corresponding image block to be completed from the original image, input the image block to be completed into the diffusion inversion model for reverse diffusion processing, generate the completed image block, and fill the completed image block back into the corresponding position of the original image to obtain the complete image;

[0100] Step 3: Extract features from the original image and the complete completed image respectively, and then stitch them together to generate a fused feature map;

[0101] Step 4: Input the fused feature map into the improved InternImage network. The improved InternImage network introduces a physical field coupled encoder to perform field propagation calculation, and maps the binary blurred region mask as an energy source map to participate in the field propagation calculation, outputting an enhanced feature map;

[0102] Step 5: Perform multi-scale feature parsing based on the enhanced feature map to generate a preliminary target set including candidate target regions and their corresponding initial confidence levels;

[0103] Step 6: Input the original image and the previous frame image into the RAFT optical flow algorithm to generate a dense optical flow map, and correct the position of the candidate target regions in the preliminary target set based on the dense optical flow map to obtain the compensated target set;

[0104] Step 7: Evaluate the credibility of candidate target regions in the compensation target set and update the corresponding initial confidence to generate a credible target set;

[0105] Step 8: Filter and integrate the set of trustworthy targets, and output the final detection results including target location and target confidence level.

[0106] In this embodiment, step one specifically includes:

[0107] Acquire the current frame's raw image and the corresponding previous frame image captured by the vehicle-mounted camera;

[0108] The original image of the current frame is divided into multiple non-overlapping image sub-regions, each of which contains the same number of pixels.

[0109] For each image sub-region, the adjacent pixels within the sub-region are traversed sequentially, the difference in grayscale values ​​between adjacent pixels is calculated and accumulated to obtain the first value;

[0110] For the same sub-region of an image, the difference between the maximum and minimum gray values ​​is calculated to obtain the second value;

[0111] The image sub-region in the current frame's original image is compared pixel by pixel with the corresponding image sub-region in the previous frame's image, and the average value of the grayscale difference of the corresponding pixels is calculated to obtain the third value.

[0112] The first average value is obtained by averaging the first values ​​of all image sub-regions in the original image of the current frame; the second average value is obtained by averaging the second values ​​of all image sub-regions; and the third average value is obtained by averaging the third values ​​of all image sub-regions.

[0113] When the first value of the image sub-region is less than the first average value and the second value is less than the second average value, or when the third value of the image sub-region is greater than the third average value, the blur region mask value corresponding to the image sub-region is set to 1; otherwise, the blur region mask value corresponding to the image sub-region is set to 0.

[0114] Combine the blur region mask values ​​corresponding to all image sub-regions to form a binary blur region mask corresponding to the spatial position of the original image in the current frame;

[0115] In this invention, by analyzing the grayscale changes of adjacent pixels and the grayscale distribution within each sub-region of the current frame's original image, and combining this with the grayscale change analysis of corresponding regions between the current frame's original image and the previous frame's image, a comprehensive judgment is made on the spatial detail changes and temporal changes of each sub-region, thereby determining the mask value for the blurred region. The first value reflects the cumulative grayscale changes of adjacent pixels within the sub-region. When the first value is lower than the overall average level, it indicates that the overall grayscale change amplitude in that region is relatively small, and the pixel distribution tends to be flat, typically corresponding to areas that are out of focus, fogged, or have details obscured. The second value represents the distribution range of grayscale values ​​within the sub-region. When the second value is lower than the overall average level, it indicates that the brightness levels in that region are concentrated, lacking variation in brightness, commonly seen in areas with low contrast or decreased visibility. Using both the first and second values ​​as judgment conditions simultaneously can avoid misjudgments caused solely by insufficient local brightness changes.

[0116] The third value originates from the grayscale change between the corresponding regions in the current frame and the previous frame. When the third value is higher than the overall average level, it indicates that the region has significant changes in the time dimension, usually caused by rapid motion, dynamic occlusion, or image jitter. Such regions also have high uncertainty within a single frame. Therefore, this invention uniformly marks image sub-regions that meet any of the above conditions as blurred regions, and sets the corresponding blurred region mask value to 1 to ensure that subsequent processing can focus on these regions, thereby improving the stability and reliability of overall target detection.

[0117] In this embodiment, step two specifically includes:

[0118] Based on the binary blurred region mask, the image sub-region with a blurred region mask value of 1 is located in the original image, and the image sub-regions with common pixel boundaries at the boundary positions are merged according to the pixel spatial connectivity relationship to form multiple independent regions to be completed.

[0119] Based on the spatial position of each region to be completed in the original image, multiple image blocks corresponding to each region to be completed are cropped from the original image, and the position coordinates of each image block to be completed in the original image are recorded.

[0120] Each image block to be completed is input into the diffusion inversion model, which includes a forward diffusion module, a feature encoding module, and a backward diffusion module.

[0121] The forward diffusion module performs a multi-step noise superposition operation on the image block to be completed. In each diffusion step, random perturbations are superimposed on the pixel values ​​of the image block to be completed to obtain the intermediate image state of the corresponding diffusion step. After completing all diffusion steps, a noise representation of the image block to be completed is generated.

[0122] The feature encoding module performs convolutional feature extraction and hierarchical feature mapping on the image block to be completed, generating a structural feature representation corresponding to the image block to be completed.

[0123] In this invention, the feature encoding module first performs convolutional feature extraction on the image block to be incomplete. Specifically, a two-dimensional convolution operator with a kernel size of 3×3 and a stride of 1 is used to perform layer-by-layer convolution operations on the image block to be incomplete. After each convolutional layer, a batch normalization operation is connected and a ReLU nonlinear activation function is applied to extract local edges, textures, and structural information in the image block. The number of convolutional channels is set to 32, 64, and 128 respectively, thereby gradually enhancing the feature representation capability while maintaining spatial resolution. Subsequently, hierarchical feature mapping processing is performed on the convolutional output features. The feature map is scaled by setting a downsampling convolution with a stride of 2 to form a multi-level feature representation. The features obtained at different scales are concatenated in sequence to obtain a structural feature representation that simultaneously contains local detail information and overall structural information.

[0124] The back diffusion module takes noise representation as initial input and combines it with structural feature representation. It performs multi-step denoising processing according to the diffusion step sequence opposite to that of the forward diffusion module. In each back diffusion step, the intermediate image state is updated according to the intermediate image state and structural feature representation corresponding to the current diffusion step. After completing all back diffusion steps, the module outputs the completed image block corresponding to each image block to be completed.

[0125] In this invention, the backdiffusion module uses the noise representation obtained after forward diffusion as its initial input. The total number of steps in the backdiffusion process is set to 50, and the steps are executed sequentially from step 50 to step 1. In each backdiffusion step, the backdiffusion module first receives the intermediate image state corresponding to the current diffusion step and receives the structural feature representation output by the feature encoding module. The structural feature representation is adjusted to have the same spatial resolution as the current intermediate image state through bilinear interpolation. Subsequently, the structural feature representation and the current intermediate image state are concatenated in the channel dimension to form a joint feature tensor, and the number of channels of the joint feature tensor is set to 256. The joint feature tensor is then... The noise correction is fed into the denoising sub-network, which consists of three consecutive two-dimensional convolutional layers. Each convolutional layer has a kernel size of 3×3 and a stride of 1. The number of output channels is set to 128, 64, and the same number of channels as the image, respectively, thereby generating the noise correction amount corresponding to the current diffusion step. The noise correction amount is subtracted pixel by pixel from the current intermediate image state to obtain the intermediate image state for the next diffusion step. By repeating the above noise correction and state update process in each backdiffusion step, the backdiffusion module gradually reduces the noise components in the intermediate image state and completes the gradual reconstruction of the image block to be completed under the continuous constraint of structural feature representation. After completing all backdiffusion steps, the final completed image block is output.

[0126] Based on the position coordinates of each image block to be completed in the original image, the corresponding completed image blocks are backfilled into the positions in the original image corresponding to each region to be completed. The pixel values ​​in the original image are replaced with the pixel values ​​in the completed image blocks to obtain the complete image.

[0127] In this invention, blurred regions in the original image are accurately located using a binary blurred region mask, and spatially connected blurred regions are merged into multiple regions to be completed. Diffusion inversion processing is performed only on these regions to be completed, which effectively avoids unnecessary generation operations on clear regions, thereby maintaining the integrity of reliable information in the original image. Furthermore, the regions to be completed are cropped into multiple image blocks to be completed and diffusion inversion is performed on each block separately. This allows the diffusion inversion model to perform noise superposition and denoising updates within a smaller spatial range, which helps reduce computational complexity and improve generation stability. By introducing structural feature representations extracted by the feature encoding module during the backdiffusion process, the intermediate image state is updated step by step, ensuring that the completed result is consistent with the original image context in terms of spatial structure and texture distribution. This results in a completed image with good structural continuity and fewer artifacts, providing more reliable input data for target detection.

[0128] In this embodiment, step three specifically includes:

[0129] The original image is processed by three layers of spatial convolution. In the first layer of spatial convolution, a 3×3 pixel neighborhood around each pixel is selected for convolution calculation, and a downsampling operation with a stride of 2 is performed to obtain the first layer of the original image feature map.

[0130] In the second layer of spatial convolution processing, the original image feature map of the first layer is used as input, and a 3×3 pixel neighborhood is used for convolution calculation. A downsampling operation with a stride of 2 is performed to obtain the original image feature map of the second layer.

[0131] In the third-layer spatial convolution processing, the original image feature map of the second layer is used as input, and convolution calculation is performed using a 3×3 pixel neighborhood to obtain the original image feature map of the third layer.

[0132] The complete image is processed by three spatial convolution layers in sequence, with the same number of spatial convolution layers, pixel neighborhood range and downsampling order as the original image, to obtain the first, second and third complete image feature maps respectively.

[0133] For original image feature maps and completed image feature maps at the same level, the height and width are adjusted by bilinear interpolation to ensure that the original image feature maps and completed image feature maps at the corresponding level have the same spatial size.

[0134] The original image feature map and the completed image feature map, which are at the same level and have the same spatial size, are stitched together in the channel dimension to obtain the first stitched feature map, the second stitched feature map and the third stitched feature map in sequence.

[0135] The first, second, and third stitched feature maps are adjusted to the same spatial size through interpolation and resampling, and then stitched together in the channel dimension to obtain a multi-layer stitched feature map.

[0136] Channel mapping processing is performed on the multi-layer stitched feature map. Channel mapping processing is a linear transformation based on the 1×1 pixel neighborhood. The multi-layer stitched feature map is mapped in the channel dimension to generate a fused feature map.

[0137] In this invention, three-layer spatial convolution processing is performed on both the original image and the fully completed image. Downsampling is introduced in the first two convolution layers, ensuring that the resulting multi-layer original image feature maps and multi-layer completed image feature maps simultaneously contain image information at different spatial resolutions. This preserves local detail while introducing structural information within a larger receptive field. By aligning the spatial dimensions of the original image feature maps and the completed image feature maps within the same layer and stitching them along the channel dimension, reliable information preserved in the original image and restored regional information in the completed image can be jointly expressed at the corresponding scale, avoiding reliance on a single image. The source causes information loss; furthermore, the first, second, and third stitched feature maps obtained from different levels are uniformly adjusted to the same spatial size before channel stitching, so that multi-scale information is represented in the same feature space, which is beneficial to simultaneously retain fine-grained texture information and high-level structural information; finally, channel mapping is performed on the multi-layer stitched feature maps by linear transformation based on 1×1 pixel neighborhood, and the channel information is integrated without changing the spatial resolution, resulting in a fused feature map with consistent structure and complete scale information, providing a more stable and information-rich feature input for subsequent target detection processing.

[0138] In this embodiment, step four specifically includes:

[0139] The fused feature map is input into the improved InternImage network. The improved InternImage network uses the backbone structure of InternImage as its basic framework. The backbone structure is composed of multiple feature processing layers connected sequentially.

[0140] Each feature processing layer includes a convolutional computation unit and a feature integration unit connected in sequence. The convolutional computation unit performs convolutional computation on the input feature map, with the kernel size set to 3×3 and the stride set to 1, and outputs an intermediate feature map. The feature integration unit concatenates the intermediate feature map with the input feature map of the current feature processing layer in the channel dimension to obtain a combined feature map, and performs convolutional computation on the combined feature map to obtain the output feature map of the current feature processing layer.

[0141] Within the improved InternImage network backbone structure, a physical field coupling encoder is introduced between adjacent feature processing layers. The output feature map of the previous feature processing layer is input into the physical field coupling encoder before entering the next feature processing layer. After being processed by the physical field coupling encoder, it becomes the input feature map of the next feature processing layer.

[0142] The feature map input to the physical field coupled encoder is modeled as a two-dimensional signal field in the spatial dimension, where each spatial position in the feature map corresponds to a field point in the two-dimensional signal field, and the channel vector of the feature map at the corresponding spatial position is used as the initial state vector of the field point.

[0143] In this invention, the channel vector originates from the feature map input to the physical field coupled encoder. This feature map is output from the previous feature processing layer in the improved InternImage network. It maintains a regular two-dimensional grid structure in the spatial dimension and contains multiple feature channels in the channel dimension. For any spatial location in the feature map, the values ​​of all channels at that spatial location are read sequentially along the channel dimension and combined according to the channel index order to form a set of numerical sequences. This numerical sequence constitutes the channel vector corresponding to the spatial location. The dimension of the channel vector is equal to the number of channels in the feature map, and each element in the vector corresponds to the feature response value of different channels at that spatial location. In this way, each spatial location in the feature map corresponds to a unique channel vector, thereby realizing the mapping of the two-dimensional feature map into a two-dimensional signal field structure composed of multiple field points, wherein the initial state vector of each field point is determined by the channel vector at the corresponding spatial location.

[0144] The binary blurred region mask is mapped to an energy source map with the same spatial size as the two-dimensional signal field. The spatial position of the blurred region mask with a value of 1 corresponds to the energy value of 1 in the energy source map, and the spatial position of the blurred region mask with a value of 0 corresponds to the energy value of 0 in the energy source map.

[0145] Three field propagation calculations are performed in the physical field coupled encoder, and each field propagation calculation performs an update operation on each field point of the two-dimensional signal field.

[0146] After completing the three field propagation calculations, the updated two-dimensional signal field is arranged in spatial order and restored to the form of a feature map, which is then used as the input feature map for the next feature processing layer.

[0147] After sequential processing by the multi-level feature processing layer and the physical field coupled encoder in the improved InternImage network, an enhanced feature map is output.

[0148] In this invention, the improved InternImage network inherits the basic framework of the existing InternImage network in its structural design. Both adopt a backbone structure composed of multiple feature processing layers connected sequentially. By performing convolution calculations and feature integration operations in each feature processing layer, the input feature map is transformed and represented step by step. The improved InternImage network maintains consistency with the existing InternImage network in the basic composition of the feature processing layers, the hierarchical transfer method of the feature flow, and the spatial organization of the feature map, thereby ensuring the stability and compatibility of the overall network structure.

[0149] Compared to the existing InternImage network, the improved InternImage network of this invention introduces a physical field coupled encoder as a new structural unit within the backbone structure. This physical field coupled encoder is set between adjacent feature processing layers, so that the feature map output by the previous feature processing layer undergoes a two-dimensional signal field modeling process before entering the next feature processing layer. In this process, the feature map is mapped into a two-dimensional signal field composed of multiple field points, and the state vector of each field point is updated through multiple field propagation calculations. At the same time, an energy source map obtained by mapping a binary blurred region mask is introduced as an external constraint to participate in the field propagation process, thereby introducing information related to the spatial distribution of the blurred region of the image at the feature level.

[0150] Through the above structural improvements, the InternImage network, while maintaining the original network hierarchy and convolutional feature processing method, enables the correlation between different spatial locations in the feature map to be explicitly modeled through field propagation calculation. The resulting enhanced feature map is more stable in terms of spatial continuity and regional consistency, and can more fully reflect the feature change relationship between fuzzy and non-fuzzy regions, thus providing a more consistent and structured feature representation basis for target region parsing.

[0151] In this embodiment, each field propagation calculation performs an update operation on each field point of the two-dimensional signal field, specifically as follows:

[0152] Select four adjacent field points (up, down, left, and right) centered on the current field point, read the state vectors of the four adjacent field points respectively and add them together to obtain the neighborhood sum vector. Divide the neighborhood sum vector by 4 to obtain the neighborhood mean vector.

[0153] Subtracting the state vector of the current field point from the neighborhood mean vector yields the diffusion difference vector.

[0154] Multiplying the diffusion difference vector by a set diffusion coefficient yields the diffusion increment vector;

[0155] Read the energy value of the current field point at the corresponding spatial location in the energy source map. The energy value is the value of the energy source map at the corresponding spatial location.

[0156] Expand the energy value into an energy vector with the same dimension as the current field point state vector;

[0157] The diffusion increment vector is added element-wise to the energy vector to obtain the update increment vector;

[0158] Add the current state vector to the update increment vector element by element to obtain the updated state vector, and replace the current state vector with the updated state vector.

[0159] In this invention, the state vectors of each field point in the two-dimensional signal field are updated using a neighborhood-based method during field propagation calculation. By summing and averaging the state vectors of the four neighboring field points around the current field point, the state update process is influenced by the changes in the features of adjacent spatial locations, thus establishing a stable spatial continuity at the feature level. By using the difference between the neighborhood mean vector and the current field point state vector as a diffusion difference and introducing a fixed diffusion coefficient of 0.5 for weighted updates, the change amplitude of the state vector is kept within a controlled range, avoiding violent oscillations in the feature update process. Furthermore, by incorporating the energy vector obtained from the energy source map into the state update, the spatial location corresponding to the fuzzy region continuously receives additional updates during field propagation, thus explicitly introducing the spatial location information of the fuzzy region into the feature update process. Through the above update operations, the state vectors obtained after multiple field propagation calculations exhibit higher consistency and continuity in the local region, providing a smoother and more structurally clear feature representation foundation for the feature processing layer.

[0160] In this embodiment, step five specifically includes:

[0161] Three sets of convolution calculations with different kernel sizes were performed on the enhanced feature maps at the same spatial resolution. The kernel sizes were set to 3×3, 5×5 and 7×7 respectively, and the stride was set to 1 for each, resulting in three scale feature maps corresponding to different receptive ranges.

[0162] For each scale feature map, a sliding window is traversed in the spatial dimension according to a set step size. The width and height of the sliding window are both set to 8 pixels, and the sliding step size is set to 4 pixels. At each sliding window position, the scale feature map region covered by the current sliding window is defined as a candidate target region. The candidate target region is uniquely determined in the scale feature map by the coordinates of the upper left corner and the lower right corner.

[0163] For each candidate target region, the feature values ​​at all spatial locations within the candidate target region are read pixel by pixel, and the feature values ​​are added element by element to obtain the region feature accumulation vector.

[0164] The summation of each element in the region feature accumulation vector is used to obtain the total region response value. The total region response value is then divided by the number of pixels in the candidate target region to obtain the average response value of the corresponding candidate target region, which is used as the initial confidence level of the candidate target region.

[0165] All candidate target regions and their corresponding initial confidence scores obtained from the feature maps at three scales are collected together to generate a preliminary target set.

[0166] In this invention, multiple scale feature maps are formed by applying convolution calculations with different kernel sizes to the enhanced feature maps at the same spatial resolution. This allows the feature parsing process to cover different receptive ranges while maintaining consistent spatial coordinates. Since feature maps of different scales have different feature response distributions, this method can simultaneously reflect local detail changes and feature aggregation over a larger range, thereby avoiding the problem of insufficient description of the target structure under a single receptive range.

[0167] Candidate target regions are generated by traversing a sliding window with a fixed size and step size on the feature map at each scale. This ensures that the generation process of candidate target regions has clear spatial rules and stable analytical granularity. By accumulating the feature values ​​of candidate target regions pixel by pixel and calculating the average response value, the calculation result of the initial confidence directly reflects the overall feature response level of the region, reducing the impact of abnormal responses of individual pixels on the region evaluation result. Furthermore, the candidate target regions and their initial confidence obtained at different scales are uniformly collected to form a preliminary target set with a clear structure, providing a consistent input basis for target position correction and confidence update.

[0168] In this embodiment, step six specifically includes:

[0169] The original image of the current frame and the corresponding image of the previous frame are input into the RAFT optical flow algorithm. Convolutional feature extraction processing is performed on the original image and the image of the previous frame respectively to obtain the feature map of the current frame and the feature map of the previous frame.

[0170] For each spatial coordinate position in the current frame feature map, select all spatial coordinate positions in the previous frame feature map in sequence, and perform a dot product operation between the feature vector at the corresponding spatial coordinate position in the current frame feature map and the feature vector at each spatial coordinate position in the previous frame feature map.

[0171] The scalar value obtained from each dot product operation is stored at a four-dimensional coordinate position determined by the spatial coordinate position in the current frame feature map and the corresponding spatial coordinate position in the previous frame feature map, thus generating a four-dimensional correlation volume.

[0172] In the RAFT optical flow algorithm, the optical flow estimation result is initialized. The optical flow estimation result is a two-dimensional vector field with the same spatial size as the original image of the current frame. During initialization, the two-dimensional displacement vector at each pixel position is set to zero vector. The two-dimensional displacement vector includes the displacement component in the horizontal direction and the displacement component in the vertical direction of the spatial position.

[0173] Based on the four-dimensional correlation volume and optical flow estimation results, multiple iterative updates are performed. In each iteration, for each spatial location in the feature map of the current frame, the two-dimensional displacement vector corresponding to the spatial location is read from the optical flow estimation results.

[0174] Using the spatial position in the current frame feature map as the reference starting position, the reference starting position is translated in the spatial coordinate system along the horizontal and vertical directions according to the two-dimensional displacement vector, which is used as the predicted corresponding position in the previous frame feature map.

[0175] Centered on the predicted location, a defined neighborhood consisting of multiple adjacent spatial locations is selected in the four-dimensional correlation volume, and multiple scalar values ​​corresponding to the neighborhood are read and combined to generate a local correlation feature vector.

[0176] Perform a linear mapping operation on the local correlated feature vectors to obtain a two-dimensional optical flow update vector corresponding to the current spatial position;

[0177] The updated two-dimensional optical flow vector is obtained by adding the two-dimensional optical flow update vector to the two-dimensional displacement vector at the corresponding spatial position in the optical flow estimation result element by element.

[0178] Based on the updated two-dimensional displacement vectors at all spatial locations, an updated optical flow estimation result is generated, and the updated optical flow estimation result is used as the input for the next iteration.

[0179] After all iterations are completed, the final optical flow estimation result is used as a dense optical flow map, where each pixel position in the dense optical flow map corresponds to a two-dimensional optical flow vector.

[0180] For each candidate target region in the initial target set, traverse all pixel positions within the candidate target region and read the two-dimensional optical flow vector at the corresponding pixel position in the dense optical flow map;

[0181] The summation of all two-dimensional optical flow vectors in the candidate target region is performed in the horizontal and vertical directions respectively, and then divided by the number of pixels in the candidate target region to obtain the average displacement of the candidate target region in the horizontal and vertical directions. Based on the average displacement, the coordinates of the upper left corner and the lower right corner of the candidate target region are translated and updated to generate the position-corrected candidate target region.

[0182] All candidate target regions after position correction are aggregated to generate a compensation target set;

[0183] In this invention, a dense motion estimation method based on the RAFT optical flow algorithm is adopted to model the pixel-level displacement relationship between consecutive frames. By performing convolutional feature extraction processing on the original image of the current frame and the image of the previous frame respectively, feature maps of the current frame and the previous frame with consistent spatial resolution are generated, and a four-dimensional correlation volume is constructed on this basis. The four-dimensional correlation volume is obtained by performing a dot product operation on each spatial position in the feature map of the current frame and all spatial positions in the feature map of the previous frame, which completely preserves the feature matching relationship in the global spatial range across frames, thereby avoiding the matching instability problem caused by relying solely on local neighborhood search.

[0184] In the optical flow estimation process, the optical flow estimation result is initialized as a two-dimensional displacement vector field with the same spatial size as the original image of the current frame. The initial displacement vector at each pixel position is set as a zero vector and is gradually updated through multiple iterations. In each iteration, the two-dimensional displacement vector provided by the current optical flow estimation result is used as a displacement reference. A fixed range of neighborhood is selected in the four-dimensional correlation volume around the predicted corresponding position for sampling. The neighborhood range is set as a square region with a side length of 9 pixels centered on the predicted corresponding position, thus forming a local correlation feature vector. By performing a linear mapping operation on the local correlation feature vector, the two-dimensional optical flow update vector at the corresponding spatial position is obtained and added element-wise with the original displacement vector to achieve gradual refinement of the optical flow estimation result.

[0185] The resulting dense optical flow map is used to correct the overall displacement of candidate target regions in the preliminary target set. By averaging the pixel-level optical flow vectors within the candidate target regions, stable compensation for the target region position is achieved, effectively reducing the target positioning deviation caused by motion blur or inter-frame displacement, and improving the spatial consistency and temporal stability of the target detection results.

[0186] In this embodiment, step seven specifically includes:

[0187] For each candidate target region in the compensation target set, read the initial confidence level of the candidate target region in the preliminary target set;

[0188] Simultaneously read the average horizontal displacement and the average vertical displacement of the candidate target region in the compensation target set;

[0189] The absolute values ​​of the average displacement in the horizontal direction and the average displacement in the vertical direction are taken respectively, and the two absolute values ​​are added together to obtain the displacement amplitude value of the candidate target area.

[0190] Based on the relationship between the displacement amplitude value and the set displacement reference value, the confidence of the candidate target region is evaluated. When the displacement amplitude value is greater than the set displacement reference value, the initial confidence of the candidate target region is multiplied by the set reduction factor to obtain the updated confidence.

[0191] When the displacement amplitude value is less than or equal to the set displacement reference value, the initial confidence of the candidate target region is multiplied by the set amplification factor to obtain the updated confidence.

[0192] The candidate target regions that have completed credibility assessment and confidence update, along with their corresponding updated confidence levels, are uniformly collected to generate a set of credible targets.

[0193] In this invention, the confidence assessment is based on the displacement stability of the target region in the time dimension. Specifically, the displacement reference value is set to 4 pixels to distinguish between stable and abnormal motion states of the target. When the displacement amplitude of the candidate target region exceeds 4 pixels, it indicates that the target has undergone a large positional shift between adjacent frames, and the reliability of its detection result decreases. At this time, the initial confidence is updated by multiplying it by a reduction factor of 0.7 to suppress the influence of unstable targets. When the displacement amplitude is less than or equal to 4 pixels, it indicates that the target maintains a relatively stable spatial positional relationship in continuous time frames. At this time, the initial confidence is updated by multiplying it by a magnification factor of 1.2 to enhance the confidence of stable targets. Through the above confidence assessment and update operations based on displacement amplitude, the final generated set of reliable targets can effectively reduce unreliable detection results introduced by motion blur while maintaining detection sensitivity, thereby improving the stability and consistency of the overall target detection results in continuous vehicle scenes.

[0194] In this embodiment, step eight specifically includes:

[0195] All candidate target regions in the trusted target set are filtered according to their corresponding updated confidence levels, and candidate target regions with updated confidence levels less than the set confidence threshold are deleted. Specifically, the updated confidence levels of all candidate target regions in the trusted target set are statistically analyzed, and the mean and standard deviation of all updated confidence levels are calculated. The result of subtracting the standard deviation from the mean is used as the confidence threshold.

[0196] The candidate target regions retained after screening are integrated. When two or more candidate target regions overlap in spatial location, each candidate target region is compared as a whole. Only the candidate target region with the highest confidence after the update is retained, and the other candidate target regions that overlap with it are deleted.

[0197] All candidate target regions obtained after screening and integration are taken as the final target regions. The target location and target confidence of each final target region are output to generate the final detection result. The target location is the spatial coordinate information of the final target region in the original image of the current frame. The spatial coordinate information includes the coordinates of the upper left corner and the lower right corner of the final target region. The upper left corner coordinate consists of horizontal and vertical coordinate values, and the lower right corner coordinate consists of horizontal and vertical coordinate values. The target confidence is the updated confidence of the final target region.

[0198] Example 1: To verify the feasibility of this invention in practice, it was applied to the target detection module of an in-vehicle AI camera in a certain intelligent driving assistance system. This module detects and tracks vehicles, pedestrians, and non-motorized vehicles in the traffic scene ahead, improving the accuracy and stability of target recognition in dynamic traffic environments. Traditional target detection methods are prone to missed or false detections under complex weather conditions (such as rain, fog, or backlighting) or obstructed / blurred conditions. Especially when the camera or target is moving at high speed, the lack of an effective temporal information compensation mechanism makes it impossible to accurately capture the true position of the target, leading to delayed or unstable decision output from the system.

[0199] In this embodiment, the video captured by the vehicle-mounted AI camera has a frame rate of 30 frames per second and a resolution of 1280×720. First, a binary blurred region mask is constructed, and reverse diffusion processing is performed based on the diffusion inversion model to obtain a complete image. Then, features are extracted from the original image and the complete image respectively, and they are stitched together to generate a fused feature map. The fused feature map is input into the improved InternImage network. The backbone of the network adopts a combination of multi-level feature processing layers and physical field coupled encoders. Each feature processing layer includes a convolutional computation unit and a feature integration unit, and a physical field coupled encoder is introduced between adjacent layers to model the two-dimensional signal field of the feature map and its propagation process, thereby enhancing the spatial consistency and boundary integrity of the features.

[0200] Multi-scale feature parsing was performed based on enhanced feature maps, and candidate target regions and initial confidence values ​​were constructed using sliding windows of different scales. The sliding window size was set to 8×8 pixels with a step size of 4 pixels. Parallel extraction was performed on the feature maps corresponding to the receptive fields of 3×3, 5×5, and 7×7, resulting in a total of 2134 candidate target regions. The initial confidence of each candidate target region was obtained by calculating the average response value of the region.

[0201] During target tracking and compensation, the current frame image and the previous frame image are jointly input into the RAFT optical flow algorithm to generate a dense optical flow map. The feature map is extracted through convolution to construct a four-dimensional correlation volume, and then iteratively updated 10 times. In each round, local correlation feature vectors are sampled based on the current optical flow estimation result, and a linear mapping is performed to generate a two-dimensional optical flow update vector, thereby correcting the spatial position of the target in the current frame. In this embodiment, the optical flow estimation accuracy reaches 93.2%, and the average optical flow estimation error is 1.41 pixels.

[0202] Based on the average displacement amplitude, the confidence level of each candidate target region is evaluated, and the initial confidence level is updated to generate a set of reliable targets. Finally, the set of reliable targets is filtered and integrated to output the final detection result, which includes the target location and the target confidence level.

[0203] To verify the effectiveness of the present invention, it was compared with the existing InternImage network and the traditional YOLOv5 network. The test dataset contained 12,000 frames of images, covering urban roads, highways, tunnels, bridges and complex weather scenes.

[0204] Table 1. Comparison of target detection performance of three networks

[0205] Network Model Accuracy (%) Recall rate (%) F1 score (%) YOLOv5 network 81.3 78.4 79.8 Existing InternImage network 84.6 81.1 82.8 Improve the InternImage network 88.1 84.3 86.2

[0206] As can be seen from the data in Table 1 above, in the target detection task, the improved InternImage network outperforms both the YOLOv5 network and the existing InternImage network in all three core metrics: accuracy, recall, and mean precision. Accuracy reflects the proportion of candidate regions that are actually targets; the improved InternImage network achieves 88.1% accuracy, a 6.8 percentage point improvement over YOLOv5's 81.3% and also better than the existing InternImage network's 84.6%, indicating its stronger ability to reduce false detections. Recall represents the proportion of all real targets correctly detected; the improved InternImage network achieves a higher recall rate. The accuracy rate was 84.3%, which is 5.9 percentage points higher than YOLOv5 and 3.2 percentage points higher than the existing InternImage, indicating a lower false negative rate. The F1 score is calculated based on the harmonic mean formula of accuracy and recall, which can effectively measure the comprehensive ability of the detection model to maintain accuracy and recall. The improved InternImage network achieved 86.2%, which is better than YOLOv5's 82.8% and the existing InternImage's 79.8%, indicating that the present invention has significant advantages in comprehensive detection capabilities. Especially when dealing with complex dynamic scenes and blurred target regions, it can improve the target detection accuracy while ensuring a high target recall rate, thereby improving the reliability and stability of the overall detection system.

[0207] In this embodiment, by introducing an improved InternImage network, the semantic representation capability of the target region is effectively enhanced, and the target recognition accuracy in complex scenes is improved. On this basis, the RAFT optical flow algorithm is further combined to perform high-precision modeling and position correction of the displacement changes of the candidate target region between consecutive time frames, which significantly reduces the target offset problem caused by rapid movement or image blur. At the same time, the confidence of the candidate target region is dynamically adjusted, which effectively improves the stability and credibility of the final detection result and ensures the robustness and real-time performance of the detection system in the actual vehicle environment.

[0208] The above description is only a preferred embodiment of the present invention, but the scope of protection of the present invention is not limited thereto. Any equivalent substitutions or modifications made by those skilled in the art within the scope of the technology disclosed in the present invention, based on the technical solution and inventive concept of the present invention, should be covered within the scope of protection of the present invention.

Claims

1. A target detection method based on machine vision for vehicle-mounted AI cameras, characterized in that, Includes the following steps: Step 1: Acquire the original image of the current frame captured by the vehicle camera, and acquire the corresponding previous frame image, perform grayscale change analysis, and generate a binary blurred region mask; Step 2: Based on the binary blurred region mask, extract the corresponding image block to be completed from the original image, input the image block to be completed into the diffusion inversion model for reverse diffusion processing to generate the completed image block, and fill the completed image block back into the corresponding position of the original image to obtain the complete image; Step 3: Extract features from the original image and the complete completed image respectively, and then stitch them together to generate a fused feature map; Step 4: Input the fused feature map into the improved InternImage network. The improved InternImage network introduces a physical field coupled encoder to perform field propagation calculation, and maps the binary blurred region mask into an energy source map to participate in the field propagation calculation, and outputs an enhanced feature map. Step 5: Perform multi-scale feature parsing based on the enhanced feature map to generate a preliminary target set including candidate target regions and corresponding initial confidence levels; Step 6: Input the original image and the previous frame image into the RAFT optical flow algorithm to generate a dense optical flow map, and correct the position of the candidate target regions in the preliminary target set according to the dense optical flow map to obtain the compensated target set; Step 7: Evaluate the credibility of candidate target regions in the compensation target set and update the corresponding initial confidence to generate a credible target set; Step 8: Filter and integrate the set of trusted targets, and output the final detection result including target location and target confidence level; Step four specifically involves: The fused feature map is input into the improved InternImage network, which uses the backbone structure of InternImage as its basic framework. The backbone structure is composed of multiple feature processing layers connected sequentially. Each feature processing layer includes a convolutional computation unit and a feature integration unit connected in sequence. The convolutional computation unit performs convolutional computation on the input feature map, with the convolutional kernel size set to 3×3 and the stride set to 1, and outputs an intermediate feature map. The feature integration unit concatenates the intermediate feature map with the input feature map of the current feature processing layer in the channel dimension to obtain a combined feature map, and performs convolutional computation on the combined feature map to obtain the output feature map of the current feature processing layer. Within the backbone structure of the improved InternImage network, a physical field coupling encoder is introduced between two adjacent feature processing layers. The output feature map of the previous feature processing layer is input into the physical field coupling encoder before entering the next feature processing layer. After being processed by the physical field coupling encoder, it becomes the input feature map of the next feature processing layer. The feature map input to the physical field coupled encoder is modeled as a two-dimensional signal field in the spatial dimension, wherein each spatial position in the feature map corresponds to a field point in the two-dimensional signal field, and the channel vector of the feature map at the corresponding spatial position is used as the initial state vector of the field point. The binary blurred region mask is mapped to an energy source map with the same spatial size as the two-dimensional signal field. The spatial position of the blurred region mask with a value of 1 corresponds to an energy value of 1 in the energy source map, and the spatial position of the blurred region mask with a value of 0 corresponds to an energy value of 0 in the energy source map. Three field propagation calculations are performed in the physical field coupled encoder, and each field propagation calculation performs an update operation on each field point of the two-dimensional signal field. After completing the three field propagation calculations, the updated two-dimensional signal field is arranged in spatial order and restored to the form of a feature map, which is then used as the input feature map for the next feature processing layer. After being processed sequentially by the multi-level feature processing layer and the physical field coupled encoder in the improved InternImage network, an enhanced feature map is output.

2. The target detection method for an in-vehicle AI camera based on machine vision according to claim 1, characterized in that, Step one specifically involves: Acquire the current frame's raw image and the corresponding previous frame image captured by the vehicle-mounted camera; The original image of the current frame is divided into multiple non-overlapping image sub-regions, each of which contains the same number of pixels. For each image sub-region, the adjacent pixels within the image sub-region are traversed sequentially, the difference in gray values ​​between adjacent pixels is calculated and accumulated to obtain a first value; For the same sub-region of an image, the difference between the maximum and minimum gray values ​​is calculated to obtain the second value; The image sub-region in the current frame's original image is compared pixel by pixel with the corresponding image sub-region in the previous frame's image, and the average value of the grayscale difference of the corresponding pixels is calculated to obtain the third value. The first average value is obtained by averaging the first values ​​of all image sub-regions in the original image of the current frame; the second average value is obtained by averaging the second values ​​of all image sub-regions; and the third average value is obtained by averaging the third values ​​of all image sub-regions. When the first value of the image sub-region is less than the first average value and the second value is less than the second average value, or when the third value of the image sub-region is greater than the third average value, the blur region mask value corresponding to the image sub-region is set to 1; otherwise, the blur region mask value corresponding to the image sub-region is set to 0. The blurry region mask values ​​corresponding to all image sub-regions are combined to form a binary blurry region mask corresponding to the spatial position of the original image in the current frame.

3. The target detection method for an in-vehicle AI camera based on machine vision according to claim 1, characterized in that, Step two specifically involves: Based on the binary blurred region mask, image sub-regions with a blurred region mask value of 1 are located in the original image, and image sub-regions with common pixel boundaries at the boundary positions are merged according to the pixel spatial connectivity relationship to form multiple independent regions to be completed. Based on the spatial position of each region to be completed in the original image, multiple image blocks corresponding to each region to be completed are cropped from the original image, and the position coordinates of each image block to be completed in the original image are recorded. Each image block to be completed is input into the diffusion inversion model, which includes a forward diffusion module, a feature encoding module, and a backward diffusion module. The forward diffusion module performs a multi-step noise superposition operation on the image block to be completed. In each diffusion step, random perturbations are superimposed on the pixel values ​​of the image block to be completed to obtain the intermediate image state of the corresponding diffusion step. After completing all diffusion steps, a noise representation of the image block to be completed is generated. The feature encoding module performs convolutional feature extraction and hierarchical feature mapping on the image block to be completed, generating a structural feature representation corresponding to the image block to be completed. The reverse diffusion module takes the noise representation as the initial input and combines it with the structural feature representation. It performs multi-step denoising processing according to the diffusion step sequence opposite to that of the forward diffusion module. In each reverse diffusion step, the intermediate image state is updated according to the intermediate image state and structural feature representation corresponding to the current diffusion step. After completing all reverse diffusion steps, the module outputs the completed image block corresponding to each image block to be completed. Based on the position coordinates of each image block to be completed in the original image, the corresponding completed image blocks are backfilled into the positions in the original image corresponding to each region to be completed. The pixel values ​​in the completed image blocks are then used to replace the pixel values ​​at the corresponding positions in the original image to obtain a complete completed image.

4. The target detection method for an in-vehicle AI camera based on machine vision according to claim 1, characterized in that, Step three specifically involves: The original image is processed by three layers of spatial convolution. In the first layer of spatial convolution, a 3×3 pixel neighborhood around each pixel is selected for convolution calculation, and a downsampling operation with a stride of 2 is performed to obtain the first layer of the original image feature map. In the second layer of spatial convolution processing, the first layer of original image feature map is used as input, a 3×3 pixel neighborhood is used for convolution calculation, and a downsampling operation with a stride of 2 is performed to obtain the second layer of original image feature map. In the third-layer spatial convolution processing, the original image feature map of the second layer is used as input, and convolution calculation is performed using a 3×3 pixel neighborhood to obtain the original image feature map of the third layer. The complete image is processed by three spatial convolution layers in sequence, with the same number of spatial convolution layers, pixel neighborhood range and downsampling order as the original image, to obtain the first complete image feature map, the second complete image feature map and the third complete image feature map respectively. For original image feature maps and completed image feature maps at the same level, the height and width are adjusted by bilinear interpolation to ensure that the original image feature maps and completed image feature maps at the corresponding level have the same spatial size. The original image feature map and the completed image feature map, which are at the same level and have the same spatial size, are stitched together in the channel dimension to obtain the first stitched feature map, the second stitched feature map and the third stitched feature map in sequence. The first, second, and third stitched feature maps are adjusted to the same spatial size through interpolation and resampling, and then stitched together in the channel dimension to obtain a multi-layer stitched feature map. Channel mapping processing is performed on the multi-layer stitched feature map. The channel mapping processing is a linear transformation based on the 1×1 pixel neighborhood. The multi-layer stitched feature map is mapped in the channel dimension to generate a fused feature map.

5. The target detection method for an in-vehicle AI camera based on machine vision according to claim 1, characterized in that, Each field propagation calculation involves an update operation on each field point of the two-dimensional signal field, specifically: Select four adjacent field points (up, down, left, and right) centered on the current field point, read the state vectors of the four adjacent field points respectively, and add them together to obtain the neighborhood summation vector. Divide the neighborhood summation vector by 4 to obtain the neighborhood mean vector. Subtracting the state vector of the current field point from the neighborhood mean vector yields the diffusion difference vector. Multiplying the diffusion difference vector by a set diffusion coefficient yields the diffusion increment vector; Read the energy value of the current field point at the corresponding spatial location in the energy source map, where the energy value is the numerical value of the energy source map at the corresponding spatial location. Expand the energy value into an energy vector with the same dimension as the current field point state vector; The diffusion increment vector is added element-wise to the energy vector to obtain the update increment vector; The state vector of the current field point is added element by element to the update increment vector to obtain the updated state vector, and the updated state vector replaces the state vector of the current field point.

6. The target detection method for an in-vehicle AI camera based on machine vision according to claim 1, characterized in that, Step five specifically involves: Three sets of convolution calculations with different kernel sizes were performed on the enhanced feature map at the same spatial resolution. The kernel sizes were set to 3×3, 5×5 and 7×7 respectively, and the stride was set to 1 for each, resulting in three scale feature maps corresponding to different receptive ranges. For each scale feature map, a sliding window is traversed in the spatial dimension according to a set step size. At each sliding window position, the scale feature map region covered by the current sliding window is defined as a candidate target region. For each candidate target region, the feature values ​​at all spatial locations within the candidate target region are read pixel by pixel, and the feature values ​​are added element by element to obtain the region feature accumulation vector. The elements in the region feature accumulation vector are summed to obtain the total region response value. The total region response value is then divided by the number of pixels in the candidate target region to obtain the average response value of the corresponding candidate target region, which is used as the initial confidence level of the candidate target region. All candidate target regions and their corresponding initial confidence scores obtained from the feature maps at three scales are aggregated to generate a preliminary target set.

7. The target detection method for an in-vehicle AI camera based on machine vision according to claim 1, characterized in that, Step six specifically involves: The original image of the current frame and the corresponding image of the previous frame are input into the RAFT optical flow algorithm. Convolutional feature extraction processing is performed on the original image and the image of the previous frame respectively to obtain the feature map of the current frame and the feature map of the previous frame. For each spatial coordinate position in the current frame feature map, select all spatial coordinate positions in the previous frame feature map in sequence, and perform a dot product operation between the feature vector at the corresponding spatial coordinate position in the current frame feature map and the feature vector at each spatial coordinate position in the previous frame feature map. The scalar value obtained from each dot product operation is stored at a four-dimensional coordinate position determined by the spatial coordinate position in the current frame feature map and the corresponding spatial coordinate position in the previous frame feature map, thus generating a four-dimensional correlation volume. In the RAFT optical flow algorithm, the optical flow estimation result is initialized. The optical flow estimation result is a two-dimensional vector field with the same spatial size as the original image of the current frame. During initialization, the two-dimensional displacement vector at each pixel position is set to zero vector. The two-dimensional displacement vector includes the displacement component in the horizontal direction and the displacement component in the vertical direction. Based on the four-dimensional correlation volume and optical flow estimation results, multiple iterative updates are performed. In each iteration, for each spatial location in the feature map of the current frame, the two-dimensional displacement vector corresponding to the spatial location is read from the optical flow estimation results. Using the spatial position in the current frame feature map as the reference starting position, the reference starting position is translated in the spatial coordinate system along the horizontal and vertical directions according to the two-dimensional displacement vector, which is used as the predicted corresponding position in the previous frame feature map. Centered on the predicted location, a defined neighborhood consisting of multiple adjacent spatial locations is selected in the four-dimensional correlation volume, and multiple scalar values ​​corresponding to the neighborhood are read and combined to generate a local correlation feature vector. Perform a linear mapping operation on the local correlation feature vector to obtain a two-dimensional optical flow update vector corresponding to the current spatial position; The updated two-dimensional optical flow vector is obtained by adding the two-dimensional displacement vector at the corresponding spatial position in the optical flow estimation result element by element. Based on the updated two-dimensional displacement vectors at all spatial locations, an updated optical flow estimation result is generated, and the updated optical flow estimation result is used as the input for the next iteration; After all iterations are completed, the final optical flow estimation result is used as a dense optical flow map, where each pixel position in the dense optical flow map corresponds to a two-dimensional optical flow vector. For each candidate target region in the preliminary target set, all pixel positions within the candidate target region are traversed, and the two-dimensional optical flow vector at the corresponding pixel position in the dense optical flow map is read. The average displacement of the candidate target region in the horizontal and vertical directions is obtained by summing all two-dimensional optical flow vectors in the candidate target region and dividing them by the number of pixels in the candidate target region, thus generating the position-corrected candidate target region. All candidate target regions after position correction are aggregated to generate a compensation target set.

8. The target detection method for an in-vehicle AI camera based on machine vision according to claim 1, characterized in that, Step seven specifically involves: For each candidate target region in the compensation target set, read the initial confidence level of the candidate target region in the preliminary target set; Simultaneously, the average horizontal displacement and the average vertical displacement of the candidate target region in the compensation target set are read. The absolute values ​​of the average displacement in the horizontal direction and the average displacement in the vertical direction are taken respectively, and the two absolute values ​​are added together to obtain the displacement amplitude value of the candidate target area. Based on the relationship between the displacement amplitude value and the set displacement reference value, the confidence of the candidate target region is evaluated. When the displacement amplitude value is greater than the set displacement reference value, the initial confidence of the candidate target region is multiplied by a set reduction factor to obtain the updated confidence. When the displacement amplitude value is less than or equal to the set displacement reference value, the initial confidence of the candidate target region is multiplied by the set amplification factor to obtain the updated confidence. The candidate target regions that have completed credibility assessment and confidence update, along with their corresponding updated confidence levels, are collectively collected to generate a set of credible targets.

9. The target detection method for an in-vehicle AI camera based on machine vision according to claim 1, characterized in that, Step eight specifically involves: All candidate target regions in the set of trusted targets are filtered according to their corresponding updated confidence levels, and candidate target regions whose updated confidence levels are less than a set confidence threshold are deleted. The candidate target regions retained after screening are integrated. When two or more candidate target regions overlap in spatial location, each candidate target region is compared as a whole. Only the candidate target region with the highest confidence after the update is retained, and the other candidate target regions that overlap with it are deleted. All candidate target regions obtained after screening and integration are taken as the final target regions, and the target location and target confidence corresponding to each final target region are output to generate the final detection result. The target location is the spatial coordinate of the final target region in the original image of the current frame, and the target confidence is the updated confidence corresponding to the final target region.

Images