A target detection method and system in a dynamic background

Abstract

The application discloses a target detection method and system in a dynamic background, and the method comprises the following steps: S1, acquiring an image at a moment, and detecting a background region and each target region in the image; S2, performing sparse optical flow calculation on the background region and the image at the moment to obtain the offset of each feature point in the background region; meanwhile, performing sparse optical flow calculation on each target region and the image to obtain the offset of each target region; S3, performing sub-grid division on the image to calculate the average offset of each sub-grid; S4, performing motion compensation on the offset of the target region according to the average offset of the sub-grid where the center point of the target region is located to separate the target autonomous motion offset; and S5, acquiring the target motion state or / and target motion speed. The application can guarantee the calculation efficiency and accurately estimate the target state in the dynamic background.

Description

TECHNICAL FIELD

[0001] The present application mainly relates to the technical field of target detection, and particularly relates to a target detection method and system in a dynamic background. BACKGROUND

[0002] With the deep penetration of intelligent monitoring technology in various industry application scenarios, efficient analysis and accurate extraction of dynamic information of targets in video streams have gradually become an important research topic in the field of computer vision. Taking the high-speed inspection task of unmanned aerial vehicles as an example, due to the complex flight environment and fast flight speed, traditional target dynamic judgment and motion speed estimation methods often face challenges. Therefore, in a dynamic background, how to accurately judge whether a target is in a motion state and estimate its accurate motion speed is crucial for improving intelligent monitoring, path planning and automatic control capabilities in a dynamic environment.

[0003] At present, there are many research results and applications about target dynamic judgment and motion speed estimation, especially in the field of dynamic target detection and tracking based on visual information. Existing technologies include a target dynamic judgment method based on inter-frame difference, a target dynamic judgment method based on background subtraction and a target motion estimation method based on optical flow.

[0004] The target motion estimation method based on inter-frame difference is a classical dynamic target detection technology. Its core idea is to identify the motion region by calculating the pixel-level difference between adjacent frames. This method usually performs pixel-by-pixel difference operation on two (or three) consecutive frames, and sets a threshold to determine which pixels have changed significantly, thereby determining the position and range of the moving target. When the background itself moves, the inter-frame difference will produce a large number of false motion regions, resulting in a significant increase in false detection rate. Therefore, this method is not suitable for estimating the motion state of targets in complex scene dynamic background conditions.

[0005] The background subtraction method establishes a background model to represent the static background, and then compares it with the current frame. Its basic idea is to calculate the difference between the current frame and the background model, and then detect the foreground (moving target). The background model is usually updated and maintained through multiple frames of images. The background subtraction method assumes that the background is static, and uses a static background model to separate targets. Therefore, in a dynamic background condition, the background model may not accurately reflect the current scene, resulting in inaccurate separation of targets and background.

[0006] Target motion estimation based on optical flow infers target motion information by calculating the motion vectors of pixels in an image. Typically, this method estimates the target's displacement and velocity by calculating the optical flow field of the image frame by frame. However, this method often assumes that the background is stationary or that background motion is independent of the target. In videos captured by real-world action cameras, background changes are usually affected by the camera's own motion, causing the target's motion to blend with the background, thus affecting the accuracy of target motion state and velocity estimation. Figure 1 The example demonstrates the visualization results of using optical flow to estimate target motion in drone aerial video. The background in the image is in motion. Figure 1 The upper part of the image visualizes the visible light image, while the lower part shows the corresponding global optical flow (GAF) tracing map. The brightness of the GAF tracing map characterizes the magnitude of pixel displacement in the visible light image; higher brightness indicates greater pixel displacement, and lower brightness indicates smaller pixel displacement. Figure 1 In the image, the areas circled by dashes correspond to three moving targets, and are marked with lines connecting them in the visible light image and the optical flow tracing map. The visualization shows that the pixel displacement amplitudes of the moving target regions and the background regions in the optical flow tracing map are not significantly different. Therefore, it is difficult to accurately determine the target's motion state. Furthermore, traditional target motion estimation based on optical flow methods typically requires solving the global optical flow field, constructing a full-image energy function, and performing iterative optimization. This leads to an exponential increase in time complexity with image resolution, making it difficult to meet the requirements of real-time target state estimation. Summary of the Invention

[0007] To address the technical problems existing in the prior art, this invention provides a target detection method and system under dynamic background that ensures computational efficiency while accurately estimating the target state under dynamic background.

[0008] To solve the above-mentioned technical problems, the technical solution proposed by this invention is as follows:

[0009] A target detection method under dynamic background, comprising the following steps:

[0010] S1, Obtain Time Image Detecting images The target region of each target is then processed in the image. The target region is removed to obtain the background region;

[0011] S2, Combine the background area with... Image of time Perform sparse optical flow calculations to obtain the offset of each feature point in the background region;

[0012] At the same time, each target area is compared with Image of time Perform sparse optical flow calculations to obtain the offset of each target region;

[0013] S3, Image Perform sub-mesh generation and calculate the average offset of each sub-mesh based on the offset of each feature point within the sub-mesh;

[0014] S4. Then, based on the average offset of the subgrid where the center point of the target area is located, motion compensation is performed on the offset of the target area to separate the target autonomous motion offset.

[0015] S5. Obtain the target motion state and / or target motion speed based on the target autonomous motion offset.

[0016] Preferably, in step S2, the specific process of obtaining the offset of the background region is as follows:

[0017] Select image background area The feature points with significant characteristics in the data are used as the tracking benchmark, and the set of feature points extracted from them is denoted as . ;in The number of feature points selected. These are the pixel coordinates corresponding to the feature points;

[0018] Get The set of feature points at time points ,in For the tracked feature points in the image The corresponding pixel coordinates are used to calculate the displacement vector representation of the feature point. ,in and These represent the displacements in the x and y directions, respectively.

[0019] Preferably, in step S2, the process of obtaining the offset of each target region is as follows:

[0020] right Each target area Feature points with significant characteristics are selected and represented by circles, and then tracked using sparse optical flow. The location of the corresponding feature point is determined, and the tracked feature point is represented by a triangle. The offset of the feature point is then calculated using a set. To indicate, among which and For the offset of feature points in the corresponding target region direction and The mean in the direction;

[0021] The displacement of this feature point represents the offset of the target region.

[0022] Preferably, the specific process of step S3 is as follows:

[0023] Image The width is divided into equal parts Divide into equal parts and divide the long side into equal parts Divide into equal parts, and finally form Each sub-grid region, the resulting set of regions is represented as: Average offset of subgrid The set is represented as , and For subgrid The offset of each feature point within the region is direction and The mean of the components in the direction.

[0024] Preferably, in step S3, during the calculation of the average offset of the sub-mesh, interpolation processing is performed on the sub-mesh obscured by the target region, specifically as follows:

[0025] .

[0026] Preferably, in step S4, the target autonomous motion offset set The calculation formula is:

[0027] }

[0028] in and For the offset of feature points in the corresponding target region direction and The mean of the components in the direction, This is the average offset of the subgrid where the center point of the target area is located.

[0029] Preferably, in step S5, the specific process of obtaining the target motion state based on the target autonomous motion offset is as follows:

[0030] The set of motion distances of each target in two frames is calculated based on the target's autonomous motion offset. :

[0031]

[0032] For each target, if its corresponding autonomous motion distance Greater than a certain threshold If the target is in motion, it is considered to be in motion; otherwise, it is considered to be stationary.

[0033] Preferably, in step S5, the specific process of obtaining the target's motion velocity based on the target's autonomous motion offset is as follows:

[0034] The pixel distance moved by each sub-grid was calculated. Then, based on the speed of the action camera Calculate the submesh The corresponding unit pixel motion speed Thus, the target velocity is obtained. .

[0035] Preferably, the formula for calculating the target's velocity is:

[0036] .

[0037] The present invention also discloses a target detection system under dynamic background, including a memory and a processor connected to each other, wherein the memory stores a computer program, and the computer program executes the steps of the method described above when run by the processor.

[0038] Compared with the prior art, the advantages of the present invention are as follows:

[0039] This invention presents a dynamic background target detection method based on the combination of optical flow calculation and target detection, comprising a motion state estimation method and a target motion velocity estimation method. The motion state estimation method estimates optical flow to both the target and background regions in the image and introduces a sub-mesh partitioning strategy to model and compensate for local background motion, thereby extracting the target's autonomous motion offset and achieving accurate determination of whether the target is in motion. The target motion velocity estimation method calculates the offset of the image background and maps the motion camera's own velocity to the background offset, establishing a proportional relationship between the true velocity and the background offset. This, combined with the offset generated by the target's autonomous motion, achieves high-precision inverse mapping calculation of the target's actual velocity.

[0040] This invention innovatively introduces a modeling method that combines sparse optical flow with image spatial subgrids. While ensuring computational efficiency, it achieves low-cost modeling of non-uniform motion fields in the background, and is especially suitable for target motion state estimation tasks in dynamic background and moving platform (such as UAV) scenarios. Attached Figure Description

[0041] Figure 1 This provides a visible light image of a target and its corresponding global optical flow tracking map in existing drone aerial video footage.

[0042] Figure 2 This is a flowchart of an embodiment of the target motion state estimation method under dynamic background in this invention.

[0043] Figure 3 This is a schematic diagram of optical flow calculation for the target region in this invention.

[0044] Figure 4 This is a schematic diagram of subgrid division in this invention. Detailed Implementation

[0045] The present invention will be further described below with reference to the accompanying drawings and specific embodiments.

[0046] like Figure 2 As shown, the target motion state estimation method under dynamic background provided in this embodiment of the invention includes the following steps:

[0047] S1, Obtain Image of time Image detection is performed using the target detection module. The target region in the image; after target detection is completed, the image is... The target region detected in the image is removed to obtain the background region;

[0048] S2, Combine the background area with the next frame image. ( Sparse optical flow calculations are performed on the time-series images to obtain the offsets of significant feature points in the background region, thus representing the motion information of the background region.

[0049] At the same time, for each target area, the same applies... Perform sparse optical flow calculations to obtain the offset of the target region (including the combined motion of the moving camera and the autonomous motion of the target).

[0050] S3. Perform sub-grid division on the image, and calculate the average offset of each sub-grid based on the offset of each feature point within the sub-grid;

[0051] S4. Based on the average offset of the subgrid where the center point of the target area is located, motion compensation is performed on the offset of the target area to separate the target autonomous motion offset.

[0052] S5. After obtaining the target's autonomous motion offset, perform target motion state discrimination and target motion velocity estimation, specifically as follows:

[0053] Target motion state determination: By analyzing the magnitude of the target's autonomous motion offset, it can be determined whether the target is stationary or in motion.

[0054] Target motion velocity estimation: Based on the input motion camera velocity and the offset of each feature point in the background area of ​​the target sub-grid, a velocity mapping relationship is established. This mapping relationship is then applied inversely to the target's autonomous motion offset to estimate the target's motion velocity.

[0055] In summary, this invention achieves accurate estimation of the target's static and dynamic states and motion speed in a dynamic context by integrating target detection and optical flow calculation, combined with sub-mesh generation and motion compensation techniques.

[0056] This invention models the motion vector of the background by using sparse optical flow and sub-mesh partitioning, thereby reducing computational costs while accurately estimating the background offset. Then, by calculating sparse optical flow on the target region detected by the target detection algorithm, the common motion vector of the target and background is accurately obtained. Finally, the target's motion state is estimated by the difference between the background motion vector and the target motion vector. This invention ensures computational efficiency while accurately estimating the target state under dynamic backgrounds.

[0057] To better understand the above technical solutions, the following will provide a detailed explanation of the technical solutions in conjunction with the accompanying drawings and specific implementation methods.

[0058] Specifically, the invention is mainly divided into two parts. The first part is the determination of the target motion state (motion / static determination) based on optical flow and target detection, and the final result is... Figure 2 The first part is the output corresponding to Task 1; the second part is the target motion velocity estimation based on optical flow and target detection, and the final result is... Figure 2 The output corresponding to Task 2.

[0059] The target motion state determination method based on optical flow and target detection includes the following steps:

[0060] S1, such as Figure 2 As shown, firstly Time Image A target detector is used to obtain the target regions corresponding to all targets in the image. ,in Indicates shared ownership One detection target; then, the target area From the image Remove from the middle to obtain the background area. That is, other image regions that do not contain any detected target regions;

[0061] S2, then, the background area With the next frame image ( Sparse optical flow calculations are performed on images at different times.

[0062] For sparse optical flow calculation, the image background region needs to be selected first. The feature points with significant characteristics in the data are used as the tracking benchmark, and the set of feature points extracted from them is denoted as . ;in The number of feature points selected. For feature points The corresponding pixel coordinates; where significant feature points refer to points in an image that are easily identified, located, and stably matched across different images. They possess uniqueness, stability, and repeatability within the image content, exhibiting local uniqueness: 1. High contrast: The grayscale values ​​or colors of the pixels surrounding the point change drastically (high gradient); 2. Rich structure: The point is located in areas with rich local structural information in the image, such as corner points, spots (areas that are significantly different from the surrounding areas), endpoints of edges, or points with high curvature on edges).

[0063] Then calculate the motion displacement of feature points between adjacent frames, and obtain the result using this method. The set of feature points at time points ,in For the tracked feature points in the image The corresponding pixel coordinates are used to calculate the set of displacement vectors representing the feature points. ,in For feature points The displacement vector, and Feature points The displacement in the x and y directions.

[0064] Then, the offset of each target region is calculated, as follows: Figure 3 As shown. First, regarding The area where each target is located Feature points with significant characteristics are selected and represented by circles, and then tracked using sparse optical flow. The corresponding feature point should be located in the target area. The tracked feature points are represented by triangles, and then the offset of the feature points is calculated. Here, the offset of the target region (including the combined motion of the camera and the target's autonomous motion) is calculated, which can be represented by a set of... To indicate, For the feature points in the corresponding target region The offset, and For the feature points in the corresponding target region The offset at direction and The mean value in the direction. Here, the offset of each target region is a two-dimensional vector.

[0065] S3. Due to the changing perspective of the action camera, the background offset varies at different locations in the image; that is, the offset of each pixel is different. To accurately obtain the autonomous motion state of the target, the offset of the target region needs to be subtracted from the offset of the corresponding background region. Therefore, image processing is required here. Sub-mesh generation is performed to compensate for different offsets in target areas at different locations. For example... Figure 4 As shown, the image width is divided into equal parts. Divide into equal parts and divide the long side into equal parts Divided into equal parts, finally formed The set of sub-grid regions is represented as: subgrid average offset The set is represented as The average offset of each subgrid is a two-dimensional vector. and background subgrid Offset of feature points within the region (obtained above) (at the time) direction and The mean of the components in the direction.

[0066] During the calculation of the average subgrid offset, there may be instances where the target region occludes the entire subgrid. Therefore, interpolation is required for these special subgrids. Specifically, the offset of the subgrid requiring interpolation can be calculated using the following formula:

[0067]

[0068] Therefore, the average offset of all sub-grid regions can be calculated.

[0069] S4. Next, it is necessary to compensate for the offset of each target area caused by camera movement, thereby calculating the offset of the target caused by autonomous movement. For the set of center points of all target regions Can be recorded as First, it is necessary to locate the center point of the target area. Which subgrid is it in? Because the offset of a target varies in different regions of the image due to camera motion, the compensation for the target offset should be the average offset of its corresponding sub-grid region. Then, the final set of autonomous target motion offsets can be calculated using the difference between these two offsets. The calculation formula is as follows:

[0070] }

[0071] in The offset of the target's autonomous motion. and For the offset of feature points in the corresponding target region direction and The mean of the components in the direction, This is the average offset of the subgrid where the center point of the target area is located;

[0072] S5. Finally, based on the target's autonomous motion offset... Calculate the distance the target has traveled in the two frames. The corresponding set for:

[0073]

[0074] For each target, if its corresponding autonomous motion distance Greater than a certain threshold If the target is in motion, it is considered to be in motion; otherwise, it is considered to be stationary. The threshold T can be adjusted according to the current image size. In the image The recommended default value is 2. Therefore, this invention enables the determination of whether a target is in motion or stationary.

[0075] When estimating the target's velocity, the set of target autonomous motion offsets and background subgrid average offset set The calculation method is as described above. After obtaining... Then it is also necessary to calculate the pixel distance of the sub-mesh background motion. The corresponding set for:

[0076]

[0077] The pixel distance moved by each sub-grid was calculated. Finally, the speed of an action camera is also needed. The input. At this point, the subgrid can be calculated. Medium unit pixel movement The corresponding real-world background relative motion velocity Corresponding set for:

[0078]

[0079] The above yielded the actual distance traveled by the target's autonomous movement. Once the background subgrid containing the target area is determined, the true velocity of the target's autonomous movement can be calculated using the unit pixel velocity of the subgrid. Corresponding set for:

[0080] .

[0081] Thus, this invention achieves autonomous motion speed estimation of the target by mapping the speed of the motion camera to image pixels and combining this with the autonomous pixel movement distance of the target for inverse mapping.

[0082] This invention presents a dynamic background target detection method based on the combination of optical flow calculation and target detection, comprising a motion state estimation method and a target motion velocity estimation method. The motion state estimation method estimates optical flow to both the target and background regions in the image and introduces a sub-mesh partitioning strategy to model and compensate for local background motion, thereby extracting the target's autonomous motion offset and achieving accurate determination of whether the target is in motion. The target motion velocity estimation method calculates the offset of the image background and maps the motion camera's own velocity to the background offset, establishing a proportional relationship between the true velocity and the background offset. This, combined with the offset generated by the target's autonomous motion, achieves high-precision inverse mapping calculation of the target's actual velocity.

[0083] This invention innovatively introduces a modeling method that combines sparse optical flow with image spatial subgrids. While ensuring computational efficiency, it achieves low-cost modeling of non-uniform motion fields in the background, and is especially suitable for target motion state estimation tasks in dynamic background and moving platform (such as UAV) scenarios.

[0084] The present invention provides a method for determining target motion / static status in dynamic backgrounds based on optical flow and target detection. This method performs optical flow analysis on the background and target from the perspective of a moving camera, calculating the offsets of both. By comparing the background and target offsets, it accurately determines whether the target is stationary or in motion, thus overcoming the problems of background-target confusion and misjudgment of motion / static states in existing technologies. The present invention also provides a method for estimating target motion velocity based on optical flow and target detection. This method calculates the offset of the image background, maps the camera motion velocity onto the background, and then uses the proportional relationship between the target offset and the background offset to accurately estimate the target motion velocity.

[0085] This invention also discloses a target detection system under dynamic background, including an interconnected memory and a processor. The memory stores a computer program, which, when run by the processor, executes the steps of the method described above. The system of this invention corresponds to the method described above and also possesses the advantages described therein.

[0086] The present invention can implement all or part of the processes in the methods of the above embodiments, or it can be implemented by hardware related to computer program instructions. The computer program can be stored in a computer-readable storage medium. When the computer program is executed by a processor, it can implement the steps of the above method embodiments. The computer program includes computer program code, which can be in the form of source code, object code, executable file, or some intermediate form. The computer-readable storage medium includes: any entity or device capable of carrying computer program code, recording media, USB flash drive, portable hard drive, magnetic disk, optical disk, computer memory, read-only memory (ROM), random access memory (RAM), electrical carrier signals, telecommunication signals, and software distribution media, etc. The memory is used to store computer programs and / or modules. The processor implements various functions by running or executing the computer programs and / or modules stored in the memory, and by calling data stored in the memory. The memory may include high-speed random access memory, as well as non-volatile memory, such as hard disks, RAM, plug-in hard disks, smart media cards (SMC), secure digital (SD) cards, flash cards, at least one disk storage device, flash memory device, or other volatile solid-state storage devices.

[0087] The above are merely preferred embodiments of the present invention. The scope of protection of the present invention is not limited to the above embodiments. All technical solutions falling within the scope of the present invention's concept are within the scope of protection of the present invention. It should be noted that for those skilled in the art, any improvements and modifications made without departing from the principles of the present invention should be considered within the scope of protection of the present invention.

Claims

1. A target detection method under dynamic background, characterized in that, Including the following steps: S1, Obtain Time Image Detecting images The target region of each target is then processed in the image. The target region is removed to obtain the background region; S2, Combine the background area with... Image of time Perform sparse optical flow calculations to obtain the offset of feature points within the background region; At the same time, each target area is compared with Image of time Perform sparse optical flow calculations to obtain the offset of each target region; S3, Image Perform sub-mesh generation and calculate the average offset of each sub-mesh based on the offset of each feature point within the sub-mesh; S4. Based on the average offset of the subgrid where the center point of the target area is located, motion compensation is performed on the offset of the target area to separate the target autonomous motion offset. S5. Obtain the target motion state and / or target motion speed based on the target autonomous motion offset; The specific process of step S3 is as follows: Image The width is divided into equal parts Divide into equal parts and divide the long side into equal parts Divide into equal parts, and finally form Each sub-grid region, the resulting set of regions is represented as: Average offset of subgrid The set is represented as , and For subgrid The offset of each feature point within the region is direction and The mean of the components in the direction; In step S3, during the calculation of the average offset of the sub-mesh, interpolation processing is performed on the sub-mesh that is occluded by the target region, specifically as follows: ; In step S2, the specific process of obtaining the offset of each feature point in the background region is as follows: Select image background area The feature points with significant characteristics in the data are used as the tracking benchmark, and the set of feature points extracted from them is denoted as . ;in The number of feature points selected. These are the pixel coordinates corresponding to the feature points; Get The set of feature points at time points ,in For the tracked feature points in the image The corresponding pixel coordinates are used to calculate the displacement vector representation of the feature point. ,in and These represent the displacements in the x and y directions, respectively.

2. The target detection method under dynamic background according to claim 1, characterized in that, In step S2, the process of obtaining the offset of each target region is as follows: right Each target area Feature points with significant characteristics are selected and represented by circles, and then tracked using sparse optical flow. The location of the corresponding feature point is determined, and the tracked feature point is represented by a triangle. The offset of the feature point is then calculated using a set. To indicate, among which and For the offset of feature points in the corresponding target region direction and The mean in the direction; The displacement of this feature point represents the offset of the target region.

3. The target detection method under dynamic background according to claim 2, characterized in that, In step S4, the target autonomous motion offset set The calculation formula is: } in and For the offset of feature points in the corresponding target region direction and The mean of the components in the direction, This is the average offset of the subgrid where the center point of the target area is located.

4. The target detection method under dynamic background according to claim 3, characterized in that, In step S5, the specific process of obtaining the target motion state based on the target autonomous motion offset is as follows: The set of motion distances of each target in two frames is calculated based on the target's autonomous motion offset. : For each target, if its corresponding autonomous motion distance Greater than a certain threshold If the target is in motion, it is considered to be in motion; otherwise, it is considered to be stationary.

5. The target detection method under dynamic background according to claim 3, characterized in that, In step S5, the specific process of obtaining the target's motion velocity based on the target's autonomous motion offset is as follows: The pixel distance moved by each sub-grid was calculated. Then, based on the speed of the action camera Calculate the submesh The corresponding unit pixel motion speed Thus, the target velocity is obtained. .

6. The target detection method under dynamic background according to claim 5, characterized in that, The formula for calculating the target's velocity is: 。 7. A target detection system under dynamic background, comprising a memory and a processor interconnected, wherein the memory stores a computer program, characterized in that, The computer program, when run by a processor, performs the steps of the method as described in any one of claims 1-6.

Images