Target detection method and system under dynamic background

Abstract

The invention discloses a target detection method and system under a dynamic background, and the method comprises the steps: S1, obtaining a time image, and detecting a background region and each target region in the image; s2, sparse optical flow calculation is carried out on the background area and the image of the moment, and the offset of each feature point in the background area is obtained; meanwhile, sparse optical flow calculation is carried out on each target area and the image, and the offset of each target area is obtained; s3, performing sub-grid division on the image, and calculating the average offset of each sub-grid; s4, performing motion compensation on the offset of the target area according to the average offset of the sub-grid where the central point of the target area is located, and separating out the autonomous motion offset of the target; and S5, obtaining a target motion state or / and a target motion speed. According to the method, the target state under the dynamic background can be accurately estimated while the calculation efficiency can be ensured.

Description

Technical Field

[0001] The present invention mainly relates to the field of target detection technology, and in particular to a target detection method and system under dynamic background. Background Art

[0002] With the increasing penetration of intelligent monitoring technology across multiple industries, efficient analysis and precise extraction of dynamic target information in video streams have become a key research topic in computer vision. For example, high-speed drone inspections often present challenges due to the complex flight environment and high speeds. Therefore, accurately determining whether a target is in motion and estimating its precise speed in dynamic environments are crucial for improving intelligent monitoring, path planning, and automated control capabilities in these environments.

[0003] Currently, there are numerous research results and applications in target motion determination and motion speed estimation, particularly in the field of dynamic target detection and tracking based on visual information. Existing technologies include methods for detecting and tracking target motion based on inter-frame differences, background subtraction, and optical flow-based target motion estimation.

[0004] Target motion estimation based on inter-frame differences is a classic dynamic target detection technique. Its core concept is to identify moving areas by calculating pixel-level differences between adjacent frames. This method typically performs a pixel-by-pixel difference operation on two (or three) consecutive frames and sets a threshold to determine which pixels have significantly changed, thereby determining the location and range of the moving target. When the background itself is moving, inter-frame differences can generate a large number of false motion regions, significantly increasing the false detection rate. Therefore, this method is not suitable for target motion estimation in complex scenes with dynamic backgrounds.

[0005] Background subtraction methods build a background model to represent the static background and then compare it with the current frame. The basic idea is to calculate the difference between the current frame and the background model to detect the foreground (moving objects). The background model is typically updated and maintained across multiple frames. Background subtraction methods assume a static background and use the static background model to separate objects. Therefore, in dynamic background conditions, the background model may not accurately reflect the current scene, resulting in inaccurate object-background separation.

[0006] Target motion estimation based on optical flow infers target motion information by calculating the motion vectors of pixels in an image. Typically, this method estimates the target's displacement and velocity by calculating the optical flow field frame by frame. However, this method often assumes a stationary background or that background motion is unrelated to the target. In videos captured by actual motion cameras, background changes are often affected by the camera's own motion, resulting in a blend of target and background motion, which in turn affects the accuracy of the target's motion state and velocity estimation. Figure 1 This example shows the visualization of target motion estimation using optical flow in a drone aerial video. In this case, the background in the image is in motion. Figure 1 In the figure, the upper part is the visualization of the visible light image, and the lower part is the corresponding global optical flow tracking map. The brightness of the global optical flow tracking map is used to represent the amplitude of the pixel displacement in the visible light image. The higher the brightness, the larger the pixel displacement; the lower the brightness, the smaller the pixel displacement. Figure 1 In the figure, the areas outlined by dashed circles correspond to three moving targets, and these are marked with lines connecting the visible light image and the optical flow tracking image. The visualization shows that the difference in pixel displacement amplitude between the moving target area and the background area in the optical flow tracking image is not obvious. Therefore, it is difficult to accurately determine the target's motion state. Furthermore, traditional target motion estimation based on optical flow methods typically requires solving the problem based on the global optical flow field, constructing a full-image energy function, and performing iterative optimization. This results in a time complexity that increases exponentially with image resolution, making it difficult to meet the requirements of real-time target state estimation. Summary of the Invention

[0007] In response to the technical problems existing in the prior art, the present invention provides a method and system for detecting targets in a dynamic background, which can ensure computational efficiency while accurately estimating the target state in the dynamic background.

[0008] In order to solve the above technical problems, the technical solution proposed by the present invention is: A method for detecting an object in a dynamic background comprises the following steps: S1. Acquisition Moment Image , detection image The target area of ​​each target in the image Eliminate the target area in the image and get the background area; S2, the background area and Image of the moment Perform sparse optical flow calculation to obtain the offset of each feature point in the background area; At the same time, each target area is Image of the moment Perform sparse optical flow calculation to obtain the offset of each target area; S3. Image Perform sub-grid division and calculate the average offset of each sub-grid based on the offset of each feature point in the sub-grid; S4. Then, according to the average offset of the subgrid where the center point of the target area is located, motion compensation is performed on the offset of the target area, thereby separating the target autonomous motion offset; S5. Obtain the target motion state and / or target motion speed according to the target autonomous motion offset.

[0009] Preferably, in step S2, the specific process of obtaining the offset of the background area is: Select the background area of ​​the image The feature points with significant features in the tracking are used as the tracking benchmark, and the extracted feature point set is recorded as ;in is the number of selected feature points, is the pixel coordinate corresponding to the feature point; Get The set corresponding to the feature points at each moment ,in The feature points after tracking are in the image The corresponding pixel coordinates in the final calculation are the displacement vector representation of the feature point ,in and are the displacements in the x and y directions respectively.

[0010] Preferably, in step S2, the process of obtaining the offset of each target area is: right Each target area Select feature points with significant features, represented by circles, and then track them using sparse optical flow The position of the corresponding feature point in the tracked feature point is represented by a triangle, and then the offset of the feature point is calculated using the set To indicate that and The offset of the feature points in the corresponding target area is Direction and Mean in direction; The displacement of this feature point represents the offset of the target area.

[0011] Preferably, the specific process of step S3 is: The image The wide side is divided into Equal parts and long sides divided equally Equal parts, finally formed sub-grid areas, and the divided area set is expressed as , the average offset of the subgrid The set of , and For subgrid The offset of each feature point in the region is Direction and The mean of the components in the direction.

[0012] Preferably, in step S3, during the calculation of the average offset of the sub-grids, the sub-grids blocked by the target area are interpolated, specifically as follows: .

[0013] Preferably, in step S4, the target autonomous motion offset set The calculation formula is: } in and The offset of the feature points in the corresponding target area is Direction and The mean of the components in the direction, is the average offset of the sub-grid where the center point of the target area is located.

[0014] Preferably, in step S5, the specific process of obtaining the target motion state according to the target autonomous motion offset is: Calculate the set of movement distances of each target in two frames of images based on the target's autonomous movement offset :

[0015] For each target, if its corresponding autonomous motion distance Greater than a certain threshold , then the target is considered to be in motion, otherwise the target is in a stationary state.

[0016] Preferably, in step S5, the specific process of obtaining the target motion speed according to the target autonomous motion offset is: Calculate the pixel distance moved by each sub-grid , and then according to the speed of the motion camera , calculate the subgrid The corresponding unit pixel motion speed , and then get the target motion speed .

[0017] Preferably, the target motion speed is calculated as follows: .

[0018] The present invention also discloses a target detection system under dynamic background, comprising a memory and a processor connected to each other, wherein a computer program is stored on the memory, and when the computer program is run by the processor, the steps of the above method are executed.

[0019] Compared with the prior art, the advantages of the present invention are: The present invention combines optical flow calculation with target detection to detect targets in dynamic backgrounds, including a motion state estimation method and a target motion speed estimation method. The motion state estimation method performs optical flow estimation on the target and background regions in an image, introduces a subgrid partitioning strategy, and models and compensates for local background motion, thereby extracting the target's autonomous motion offset and accurately determining whether the target is in motion. The target motion speed estimation method calculates the offset of the image background and maps the motion camera's own speed to the background offset, establishing a proportional relationship between the actual speed and the background offset. This, combined with the offset generated by the target's autonomous motion, achieves a high-precision inverse mapping calculation of the target's actual speed.

[0020] This paper innovatively introduces a modeling method that combines sparse optical flow with image space subgrids. While ensuring computational efficiency, it achieves low-cost modeling of background non-uniform motion fields. It is particularly suitable for target motion state estimation tasks in scenarios with dynamic backgrounds and moving platforms (such as drones). BRIEF DESCRIPTION OF THE DRAWINGS

[0021] Figure 1 It is the visible light image of the target in the drone aerial video and the corresponding global optical flow tracking map in the existing technology.

[0022] Figure 2 This is a flowchart of an embodiment of a method for estimating target motion state in a dynamic background in the present invention.

[0023] Figure 3 Schematic diagram of optical flow calculation for a target area in the present invention.

[0024] Figure 4 Schematic diagram of subgrid division in the present invention. DETAILED DESCRIPTION

[0025] The present invention will be further described below with reference to the accompanying drawings and specific embodiments.

[0026] like Figure 2 As shown, the target motion state estimation method under dynamic background provided by the embodiment of the present invention includes the following steps: S1. Acquisition Image of the moment , detect the image through the target detection module After the target detection is completed, the image The target area detected in the image is removed to obtain the background area; S2, the background area and the next frame image ( Sparse optical flow calculation is performed on the image at each moment to obtain the offset of significant feature points in the background area and represent the motion information of the background area; At the same time, for each target area, Perform sparse optical flow calculation to obtain the offset of the target area (including the joint motion of the moving camera and the target's autonomous motion); S3, performing sub-grid division on the image, and calculating the average offset of each sub-grid based on the offset of each feature point in the sub-grid; S4. Perform motion compensation on the offset of the target area according to the average offset of the subgrid where the center point of the target area is located, thereby separating the target autonomous motion offset; S5. After obtaining the target autonomous motion offset, target motion state discrimination and target motion speed estimation are performed, specifically: Target motion state discrimination: By analyzing the size of the target's autonomous motion offset, it is determined whether the target is stationary or in motion.

[0027] Target motion speed estimation: Based on the input motion camera speed and the offset of each feature point in the background area of ​​the sub-grid where the target is located, a speed mapping relationship is established. This mapping relationship is then reversely applied to the target's autonomous motion offset to estimate the target's motion speed.

[0028] In summary, the present invention achieves accurate estimation of the target's dynamic and static state and motion speed in a dynamic background by fusing target detection and optical flow calculation, combining sub-grid division and motion compensation technology.

[0029] The present invention uses sparse optical flow and subgrid division to model the background motion vector, reducing computational cost while accurately estimating background offset. Sparse optical flow calculations are then performed on the target area detected by the target detection algorithm to accurately obtain the common motion vector of the target and background. Finally, the target motion state is estimated based on the difference between the background motion vector and the target motion vector. This method achieves precise estimation of target states in dynamic backgrounds while maintaining computational efficiency.

[0030] In order to better understand the above technical solution, the above technical solution will be described in detail below with reference to the accompanying drawings and specific implementation methods.

[0031] Specifically, the invention is divided into two parts. The first part is the target motion state judgment (motion and stillness judgment) based on optical flow and target detection. The final result is Figure 2 The output corresponding to task 1 in the second part is the target motion speed estimation based on optical flow and target detection. The final result is Figure 2 The output corresponding to Task 2.

[0032] The target motion state judgment method based on optical flow and target detection includes the following steps: S1, such as Figure 2 As shown, first Moment Image Obtain the target areas corresponding to all targets in the image through an object detector ,in Indicates shared detection targets; then, the target area From the image Eliminate the background area , that is, other image areas that do not contain any detection target areas; S2. Then, the background area With the next frame image ( The sparse optical flow is calculated based on the image at the moment.

[0033] For sparse optical flow calculation, you need to first select the image background area The feature points with significant features in the tracking are used as the tracking benchmark, and the extracted feature point set is recorded as ;in is the number of selected feature points, is a feature point Corresponding pixel coordinates; Feature points with significant characteristics are points in an image that are easily identifiable, located, and can be stably matched across different images. They are unique, stable, and repeatable in image content, and have local uniqueness: 1. High contrast: The grayscale value or color of the pixels around the point changes dramatically (high gradient); 2. Rich structure: The point is located in an area with rich local structural information in the image, such as corners, spots (areas that are significantly different from the surrounding area), edge endpoints, or points on edges with high curvature). Then calculate the motion displacement of feature points between adjacent frames, and obtain The set corresponding to the feature points at each moment ,in The feature points after tracking are in the image The corresponding pixel coordinates in the final calculation are the displacement vector set representations of the feature points ,in is a feature point The displacement vector, and Feature points The amount of displacement in the x and y directions.

[0034] Then, the offset of each target area is calculated as follows Figure 3 First, The area where each target is located Select feature points with significant features, represented by circles, and then track them using sparse optical flow The position of the corresponding feature point in the tracked feature point is represented by a triangle, and then the offset of the feature point is calculated. The offset of the target area is calculated here (including the joint motion of the motion camera and the target's autonomous motion), which can be set To express, is the feature point in the corresponding target area The offset, and is the feature point in the corresponding target area The offset in Direction and Here, the offset of each target area is a two-dimensional vector.

[0035] S3. Due to the change of the camera's viewing angle, the background deviation at different positions in the image is different, that is, the offset of each pixel is different. To accurately obtain the target's autonomous motion state, the offset of the target area needs to be subtracted from the offset of the background area corresponding to the target. Therefore, the image needs to be Subgrid division is performed to compensate for different offsets of target areas at different locations. Figure 4 As shown, the image width is divided into Equal parts and long sides divided equally Equal parts, finally formed sub-grid areas, and the set of divided sub-grid areas is expressed as , subgrid The average offset The set of , the average offset of each subgrid is a two-dimensional vector, and Background subgrid The offset of the feature points inside the region (the above calculation ) Direction and The mean of the components in the direction.

[0036] When calculating the average offset of a subgrid, there may be a problem where the target area blocks the entire subgrid, so interpolation is required for these special subgrids. Specifically, the offset of the subgrid that needs to be interpolated can be calculated using the following formula:

[0037] From this, the average offset of all sub-grid areas can be calculated.

[0038] S4. Next, it is necessary to compensate for the offset of each target area caused by camera motion, so as to calculate the offset of the target caused by autonomous motion. For the center point set of all target areas Can be recorded as , first you need to locate the center point of the target area Which subgrid is it in? , because the offsets of targets in different areas of the image due to camera motion are different, the compensation for the target offset should be the average offset of the sub-grid area where it is located. Then, the final target autonomous motion offset set can be calculated by the difference between the two offsets , the calculation formula is as follows: } in is the target autonomous motion offset, and The offset of the feature points in the corresponding target area is Direction and The mean of the components in the direction, is the average offset of the sub-grid where the center point of the target area is located; S5, finally according to the target autonomous motion offset Calculate the distance the target moves in two frames , the corresponding set for:

[0039] For each target, if its corresponding autonomous motion distance Greater than a certain threshold , then the target is considered to be in motion, otherwise the target is in a stationary state. The threshold T can be adjusted according to the current image size. In the image The recommended default value of is 2. Thus, the present invention realizes the judgment of the motion state of the target as moving or stationary.

[0040] When estimating the target motion speed, the target autonomous motion offset set and background subgrid The average offset set The calculation method of is as described above. Finally, we need to calculate the pixel distance of the sub-grid background movement , the corresponding set for:

[0041] Calculate the pixel distance moved by each sub-grid Finally, you need the speed of a sports camera At this point, the subgrid can be calculated Medium unit pixel movement The corresponding real-world background relative motion speed , corresponding to the set for:

[0042] The above results in the actual movement distance corresponding to the target's autonomous movement. After determining the background sub-grid where the target area is located, the actual speed of the target's autonomous movement can be calculated through the sub-grid unit pixel movement speed. , corresponding to the set for: .

[0043] Thus, the present invention uses the mapping relationship between the speed of the motion camera and the image pixels and combines it with the target's autonomous pixel movement distance to perform inverse mapping, thereby ultimately achieving the target's autonomous motion speed estimation.

[0044] The present invention combines optical flow calculation with target detection to detect targets in dynamic backgrounds, including a motion state estimation method and a target motion speed estimation method. The motion state estimation method performs optical flow estimation on the target and background regions in an image, introduces a subgrid partitioning strategy, and models and compensates for local background motion, thereby extracting the target's autonomous motion offset and accurately determining whether the target is in motion. The target motion speed estimation method calculates the offset of the image background and maps the motion camera's own speed to the background offset, establishing a proportional relationship between the actual speed and the background offset. This, combined with the offset generated by the target's autonomous motion, achieves a high-precision inverse mapping calculation of the target's actual speed.

[0045] This paper innovatively introduces a modeling method that combines sparse optical flow with image space subgrids. While ensuring computational efficiency, it achieves low-cost modeling of background non-uniform motion fields. It is particularly suitable for target motion state estimation tasks in scenarios with dynamic backgrounds and moving platforms (such as drones).

[0046] The present invention's method for determining target motion and stillness in a dynamic background based on optical flow and target detection performs optical flow analysis on the background and target from a moving camera's perspective, calculating their respective offsets. By comparing the background offset with the target offset, the method accurately determines whether the target is stationary or in motion, thereby overcoming the prior art issues of background-target confusion and misjudgment of motion and stillness. The present invention's method for estimating target motion speed based on optical flow and target detection calculates the offset of the image background, maps the camera's motion speed to the background, and then accurately estimates the target's motion speed based on the proportional relationship between the target offset and the background offset.

[0047] The present invention also discloses a system for detecting objects in a dynamic environment, comprising a memory and a processor connected to each other, wherein the memory stores a computer program that, when executed by the processor, performs the steps of the method described above. The system of the present invention corresponds to the method described above and also has the advantages described above.

[0048] The present invention can implement all or part of the process steps in the above-described method embodiments through hardware associated with computer program instructions. The computer program can be stored in a computer-readable storage medium. When executed by a processor, the computer program can implement the steps of the above-described method embodiments. The computer program includes computer program code, which can be in source code form, object code form, executable file, or some intermediate form. Computer-readable storage media include any entity or device capable of carrying computer program code, recording media, USB flash drives, removable hard drives, magnetic disks, optical disks, computer memory, read-only memory (ROM), random access memory (RAM), electrical carrier signals, telecommunication signals, and software distribution media. Memory is used to store computer programs and / or modules. The processor implements various functions by running or executing the computer programs and / or modules stored in the memory and accessing data stored in the memory. The memory may include a high-speed random access memory and may also include a non-volatile memory, such as a hard disk, a memory, a plug-in hard disk, a Smart Media Card (SMC), a Secure Digital (SD) card, a Flash Card, at least one disk storage device, a flash memory device, or other volatile solid-state storage device.

[0049] The above are merely preferred embodiments of the present invention. The scope of protection of the present invention is not limited to the above embodiments. All technical solutions based on the principles of the present invention are within the scope of protection of the present invention. It should be noted that for those skilled in the art, various improvements and modifications that do not depart from the principles of the present invention should be considered within the scope of protection of the present invention.

Claims

1. A method for detecting an object in a dynamic background, characterized in that: Including steps: S1. Acquisition Moment Image , detection image The target area of ​​each target in the image Eliminate the target area in the image and get the background area; S2, the background area and Image of the moment Perform sparse optical flow calculation to obtain the offset of feature points in the background area; At the same time, each target area is Image of the moment Perform sparse optical flow calculation to obtain the offset of each target area; S3. Image Perform sub-grid division and calculate the average offset of each sub-grid based on the offset of each feature point in the sub-grid; S4. Perform motion compensation on the offset of the target area according to the average offset of the subgrid where the center point of the target area is located, thereby separating the target autonomous motion offset; S5. Obtain the target motion state and / or target motion speed according to the target autonomous motion offset.

2. The target detection method under dynamic background according to claim 1, characterized in that In step S2, the specific process of obtaining the offset of each feature point in the background area is as follows: Select the background area of ​​the image The feature points with significant features in the tracking are used as the tracking benchmark, and the extracted feature point set is recorded as ;in is the number of selected feature points, is the pixel coordinate corresponding to the feature point; Get The set corresponding to the feature points at each moment ,in The feature points after tracking are in the image The corresponding pixel coordinates in the final calculation are the displacement vector representation of the feature point ,in and are the displacements in the x and y directions respectively.

3. The target detection method under dynamic background according to claim 2, characterized in that In step S2, the process of obtaining the offset of each target area is as follows: right Each target area Select feature points with significant features, represented by circles, and then track them using sparse optical flow The position of the corresponding feature point in the tracked feature point is represented by a triangle, and then the offset of the feature point is calculated using the set To indicate that and The offset of the feature points in the corresponding target area is Direction and Mean in direction; The displacement of this feature point represents the offset of the target area.

4. The target detection method under dynamic background according to claim 3, characterized in that The specific process of step S3 is: The image The wide side is divided into Equal parts and long sides divided equally Equal parts, finally formed sub-grid areas, and the divided area set is expressed as , the average offset of the subgrid The set of , and For subgrid The offset of each feature point in the region is Direction and The mean of the components in the direction.

5. The method for detecting objects in a dynamic background according to claim 4, wherein: In step S3, during the calculation of the average offset of the sub-grids, the sub-grids blocked by the target area are interpolated, specifically: .

6. The method for detecting objects in a dynamic background according to claim 5, wherein: In step S4, the target autonomous motion offset set The calculation formula is: } in and The offset of the feature points in the corresponding target area is Direction and The mean of the components in the direction, is the average offset of the sub-grid where the center point of the target area is located.

7. The target detection method under dynamic background according to claim 6, characterized in that: In step S5, the specific process of obtaining the target motion state according to the target autonomous motion offset is as follows: Calculate the set of movement distances of each target in two frames of images based on the target's autonomous movement offset : For each target, if its corresponding autonomous motion distance Greater than a certain threshold , then the target is considered to be in motion, otherwise the target is in a stationary state.

8. The target detection method under dynamic background according to claim 6, characterized in that: In step S5, the specific process of obtaining the target motion speed according to the target autonomous motion offset is as follows: Calculate the pixel distance moved by each sub-grid , and then according to the speed of the motion camera , calculate the subgrid The corresponding unit pixel motion speed , and then get the target motion speed .

9. The target detection method under dynamic background according to claim 8, characterized in that The calculation formula for target movement speed is: 。 10. A target detection system in a dynamic background, comprising a memory and a processor connected to each other, wherein a computer program is stored in the memory, characterized in that: When the computer program is executed by a processor, the computer program performs the steps of the method according to any one of claims 1 to 9.

Images