A multi-motion target tracking counting method, device, equipment and storage medium

Abstract

The application is suitable for the field of detection, and discloses a multi-motion target tracking counting method, device, equipment and storage medium.The method comprises the following steps: acquiring a current frame of an input image sequence in time sequence, performing foreground detection and morphological operation, then performing connected domain analysis to obtain multiple connected domains, and screening out connected domains with an area greater than a preset connected domain area threshold as current frame motion targets; when the size of the boundary box of the current frame motion target is greater than a preset boundary box splitting threshold, the current frame motion target is split into at least two current frame sub-targets; the current frame motion target and the current frame sub-targets that are not split are collectively referred to as current frame effective targets, the state of the current frame effective targets is updated through an association, creation and removal mechanism, and after the state updating of all the current frame effective targets is completed, the number of the current frame effective targets is counted.The method can split oversized targets, and ensures that each sub-target can be independently recognized and tracked.

Description

Technical Field

[0001] This invention relates to the field of detection, and more particularly to a method, apparatus, device, and storage medium for tracking and counting multiple moving targets. Background Technology

[0002] In scenarios such as biological detection and industrial inspection, it is often necessary to automatically track and count small moving targets. Existing technologies mainly employ traditional target detection and tracking algorithms, including feature point matching, optical flow methods, and Kalman filtering. These methods have significant drawbacks when dealing with small targets: the target size is too small, and the target has high similarity to the background, making it difficult to effectively extract target features for tracking; when dealing with complex scenarios such as large target size variations, dense targets, occlusion, or target adhesion, problems such as missed detections, false detections, or inaccurate counting easily occur. Especially when a single target area is large or multiple targets are adhered, existing technologies cannot reasonably separate adhered targets, leading to severely distorted counting results. Furthermore, existing technologies typically use connected component analysis with fixed thresholds, which cannot adaptively handle targets of different sizes and shapes, and lack effective management of target lifecycles, resulting in short-term noisy targets interfering with the counting results, seriously affecting the accuracy and robustness of tracking and counting.

[0003] Therefore, existing technologies still need improvement and development. Summary of the Invention

[0004] The first objective of this invention is to provide a multi-moving target tracking and counting method, which aims to solve the technical problem in the prior art that the inability to reasonably separate adhered targets leads to serious distortion of the counting results.

[0005] To achieve the above objectives, the solution provided by the present invention is as follows:

[0006] A method for tracking and counting multiple moving targets includes: acquiring the current frame of an input image sequence in chronological order, preprocessing the current frame to obtain a preprocessed image of the current frame; performing foreground detection on the preprocessed image of the current frame to obtain a binarized foreground mask of the current frame; performing morphological operations on the binarized foreground mask of the current frame to obtain a processed foreground mask of the current frame, performing connected component analysis on the processed foreground mask of the current frame to obtain multiple connected components, and selecting connected components with an area greater than a preset connected component area threshold as moving targets of the current frame; assigning an ID to the moving target of the current frame. The bounding box information of the moving target in the current frame is recorded. When the bounding box size of the moving target in the current frame is greater than the preset bounding box splitting threshold, the corresponding moving target in the current frame is split into at least two sub-targets in the current frame according to the bounding box information of the moving target in the current frame. The unsplit moving targets in the current frame and the two or more sub-targets in the current frame are collectively referred to as the valid targets in the current frame. Based on the tracking and counting method of target matching and state management, the state of the valid targets in the current frame is updated through association, creation and removal mechanisms. After the state update of all valid targets in the current frame is completed, the number of valid targets in the current frame is counted.

[0007] Preferably, the step of performing foreground detection on the preprocessed image of the current frame to obtain a binarized foreground mask of the current frame includes: dynamically constructing a background difference model using a moving average method and updating the background difference model in real time; calculating the absolute difference image between the preprocessed image of the current frame and the background difference model of the previous frame; determining whether the pixel grayscale in the absolute difference image is greater than a preset foreground detection threshold; if so, outputting a binarized foreground mask of the current frame with a pixel mask value of 255; otherwise, outputting a binarized foreground mask of the current frame with a pixel mask value of 0.

[0008] Preferably, the update formula for the background difference model is:

[0009]

[0010]

[0011] In the formula, Indicates the first t Frame-to-background difference model, Indicates the first Frame-to-background difference model, t Indicates the current frame number. Indicates the background model update rate. Indicates the current frame preprocessed image in Pixel value at that location, Indicates the first t Pixel mask values ​​of the frame binarized foreground mask. α 0 represents the initial background model update rate. αs This represents the update rate of the stable background model. N Indicates the stable frame rate.

[0012] Preferably, the step of performing morphological operations on the binarized foreground mask of the current frame to obtain a processed foreground mask of the current frame, and performing connected component analysis on the processed foreground mask of the current frame to obtain multiple connected components, and selecting connected components with an area greater than a preset connected component area threshold as the moving target of the current frame, includes: performing morphological operations on the binarized foreground mask of the current frame to obtain a processed foreground mask of the current frame; performing connected component labeling on the processed foreground mask of the current frame using an 8-connectivity or 4-connectivity criterion to obtain multiple connected components; extracting the attributes of each connected component to obtain the bounding box information of the connected component; and selecting connected components with an area greater than a preset connected component area threshold as the moving target of the current frame based on the bounding box information of the connected components.

[0013] Preferably, the formula for calculating the number of sub-targets in the current frame is expressed as:

[0014]

[0015]

[0016] In the formula, Indicates the first The area of ​​each undivided target, where n represents the number of undivided targets. This represents the average area of ​​all targets that were not split. N i This represents the count of moving targets in each current frame. λ This indicates the splitting quantity control factor. This represents the width of the bounding box of the moving target in the current frame. This indicates the length of the bounding box of the moving target in the current frame. This represents the bounding box size of the moving target in the current frame. w s , h s ) represents the bounding box splitting threshold.

[0017] Preferably, the step of collectively referring to the unsplit current frame moving targets and two or more current frame sub-targets as current frame valid targets, and using a tracking and counting method based on target matching and state management to update the state of current frame valid targets through association, creation, and removal mechanisms, and counting the number of current frame valid targets after completing the state update of all current frame valid targets, includes: collectively referring to the unsplit current frame moving targets and current frame sub-targets as current frame valid targets; matching each current frame valid target one by one with the valid targets in the previous frame active target tracking list, and updating the state according to the matching results to form a current active target tracking list; and counting the number of current frame valid targets in the current active target tracking list.

[0018] Preferably, the step of matching each valid target in the current frame with valid targets in the active target tracking list of the previous frame one by one, and updating the status according to the matching results to form a current active target tracking list, includes: recording the bounding box coordinates of each valid target in the current frame, and calculating the center point coordinates of the valid target in the current frame based on the bounding box coordinates; for each valid target in the current frame, calculating the center point distance between the valid target in the current frame and each valid target in the previous active target tracking list based on the center point coordinates; if the center point distance between the valid target in the current frame and any valid target in the previous active target tracking list is less than a preset merging distance, then it is determined that the valid target in the current frame has been successfully matched, and the center point distance in the previous active target tracking list is used. The ID of the smallest valid target is updated to match the ID of the valid target in the current frame, and the active frame count of the valid target in the current frame is increased. It is then added to the active target tracking list of the current frame. If the distance between the center point of the valid target in the current frame and all valid targets in the previous active target tracking list is not less than the preset merging distance, the match is considered to have failed. A new ID is assigned to the valid target in the current frame that failed to match, and it is added to the active target tracking list of the current frame. For valid targets in the previous active target tracking list that were not matched by valid targets in the current frame, the active frame count is reduced, and they are added to the active target tracking list of the current frame. When the active frame count of a valid target is 0, the corresponding valid target is deleted from the active target tracking list of the current frame.

[0019] A second aspect of the present invention provides a multi-moving target tracking and counting device, comprising: a preprocessing module, configured to acquire the current frame of an input image sequence in chronological order, and preprocess the current frame to obtain a preprocessed image of the current frame; a foreground detection module, configured to perform foreground detection on the preprocessed image of the current frame to obtain a binarized foreground mask of the current frame; an analysis module, configured to perform morphological operations on the binarized foreground mask of the current frame to obtain a processed foreground mask of the current frame, and perform connected component analysis on the processed foreground mask of the current frame to obtain multiple connected components, and select connected components with an area greater than a preset connected component area threshold as moving targets of the current frame; and a splitting module. The module is used to assign IDs to the moving targets in the current frame and record the bounding box information of the moving targets in the current frame. When the bounding box size of the moving targets in the current frame is greater than a preset bounding box splitting threshold, the corresponding moving targets in the current frame are split into at least two sub-targets in the current frame according to the bounding box information of the moving targets in the current frame. The counting module is used to refer to the unsplit moving targets in the current frame and two or more sub-targets in the current frame as valid targets in the current frame. Based on the tracking and counting method of target matching and state management, the module updates the state of valid targets in the current frame through association, creation and removal mechanisms, and counts the number of valid targets in the current frame after the state update of all valid targets in the current frame is completed.

[0020] A third aspect of the present invention provides a multi-moving target tracking and counting device, comprising: a memory and at least one processor, wherein the memory stores computer-readable instructions, and the memory and the at least one processor are interconnected via a circuit; the at least one processor invokes the computer-readable instructions in the memory to cause the multi-moving target tracking and counting device to perform the various steps of the multi-moving target tracking and counting method described above.

[0021] A fourth aspect of the present invention provides a computer-readable storage medium storing computer-readable instructions that, when executed on a computer, cause the computer to perform the steps of the multi-moving target tracking and counting method described above.

[0022] The multi-moving target tracking and counting method provided by this invention accurately extracts the moving target region through foreground detection and morphological operations, and combines connected component analysis with area filtering to eliminate interference, ensuring the accuracy of initial target recognition. Moreover, for large targets whose bounding box size exceeds the bounding box splitting threshold, the method flexibly splits the oversized targets according to the bounding box splitting threshold, ensuring that each sub-target can be independently identified and tracked, greatly improving the precision and completeness of target detection. It effectively solves the problem of recognition omissions caused by target overlap, occlusion, or excessive size, and significantly improves the completeness of target detection in complex scenes. In addition, during the tracking and counting stage, by recording the bounding box information of valid targets, and with the dynamic state management mechanism of association, creation, and removal, continuous tracking and real-time state updates of valid targets are achieved. This not only ensures the continuity of target tracking, but also promptly handles the addition and disappearance of targets, avoiding duplicate or missed counts. Attached Figure Description

[0023] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on the structures shown in these drawings without creative effort.

[0024] Figure 1 This is a flowchart of the multi-moving target tracking and counting method provided in the embodiments of the present invention;

[0025] Figure 2 This is a flowchart of the multi-moving target tracking and counting device provided in an embodiment of the present invention;

[0026] Figure 3 This is a flowchart of a multi-moving target tracking and counting device provided in an embodiment of the present invention. Detailed Implementation

[0027] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only a part of the embodiments of the present invention, and not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those of ordinary skill in the art without creative effort are within the scope of protection of the present invention.

[0028] It should be noted that all directional indications (such as up, down, left, right, front, back, etc.) in the embodiments of the present invention are only used to explain the relative positional relationship and movement of each component in a certain specific posture (as shown in the figure). If the specific posture changes, the directional indication will also change accordingly.

[0029] It should also be noted that when a component is described as "fixed to" or "set on" another component, it can be directly on the other component or there may be an intervening component present. When a component is described as "connected to" another component, it can be directly connected to the other component or there may be an intervening component present.

[0030] Furthermore, the use of terms such as "first" and "second" in this invention is for descriptive purposes only and should not be construed as indicating or implying their relative importance or implicitly specifying the number of technical features indicated. Therefore, a feature defined with "first" or "second" may explicitly or implicitly include at least one of that feature. Additionally, the technical solutions of the various embodiments can be combined with each other, but only on the basis of being achievable by those skilled in the art. When the combination of technical solutions is contradictory or impossible to implement, such a combination of technical solutions should be considered non-existent and not within the scope of protection claimed by this invention.

[0031] like Figure 1 As shown, this is an embodiment of the multi-moving target tracking and counting method of the present invention.

[0032] Please see Figure 1 As shown, the multi-moving target tracking and counting method of this invention includes:

[0033] S101. Obtain the current frame of the input image sequence in chronological order, and preprocess the current frame to obtain the preprocessed image of the current frame;

[0034] S102. Perform foreground detection on the preprocessed image of the current frame to obtain the binarized foreground mask of the current frame;

[0035] S103. Perform morphological operations on the binarized foreground mask of the current frame to obtain the processed foreground mask of the current frame, and perform connected component analysis on the processed foreground mask of the current frame to obtain multiple connected components, and select the connected components with an area greater than the preset connected component area threshold as the moving target of the current frame.

[0036] S104. Assign an ID to the moving target in the current frame and record the bounding box information of the moving target in the current frame. When the bounding box size of the moving target in the current frame is greater than the preset bounding box splitting threshold, split the corresponding moving target in the current frame into at least two sub-targets in the current frame according to the bounding box information of the moving target in the current frame.

[0037] S105. The unsplit moving targets in the current frame and two or more sub-targets in the current frame are collectively referred to as valid targets in the current frame. Based on the tracking and counting method of target matching and state management, the state of valid targets in the current frame is updated through association, creation and removal mechanisms. After the state update of all valid targets in the current frame is completed, the number of valid targets in the current frame is counted.

[0038] The multi-moving target tracking and counting method of this invention accurately extracts the moving target region through foreground detection and morphological operations, and combines connected component analysis with area filtering to eliminate interference, ensuring the accuracy of initial target recognition. Moreover, for large targets whose bounding box size exceeds the bounding box splitting threshold, the method flexibly splits the oversized targets according to the bounding box splitting threshold, ensuring that each sub-target can be independently identified and tracked, greatly improving the precision and completeness of target detection. It effectively solves the problem of recognition omissions caused by target overlap, occlusion, or excessive size, and significantly improves the completeness of target detection in complex scenes. In addition, during the tracking and counting stage, by recording the bounding box information of valid targets and cooperating with the dynamic state management mechanism of association, creation, and removal, continuous tracking and real-time state updates of valid targets are achieved. This not only ensures the continuity of target tracking but also promptly handles the addition and disappearance of targets, avoiding count duplication or omission.

[0039] In this embodiment, in step S101, for video streams or real-time acquisition devices, the frames are read sequentially according to their timestamps or acquisition order; for image sequence files (such as JPEG, PNG, etc. files arranged according to naming rules), the system restores the time sequence relationship according to the file name sorting or timestamp information to ensure the temporal continuity of image frames and provide a time reference for subsequent inter-frame target tracking.

[0040] In this embodiment, the current frame is preprocessed to obtain a preprocessed image of the current frame, including but not limited to one or more of the following:

[0041] a. Noise reduction: Gaussian filtering, median filtering, or bilateral filtering are used to smooth the image, suppress random noise in the image, and reduce false detections in subsequent foreground segmentation.

[0042] b. Contrast Enhancement: Improve image contrast through histogram equalization, adaptive histogram equalization, or gamma correction methods to enhance the distinction between small targets and the background.

[0043] c. Image sharpening: Employ the Laplacian operator or an unsharpened mask to enhance target edge information and improve target boundary detection accuracy.

[0044] In this embodiment, noise reduction effectively improves the signal-to-noise ratio. Gaussian filtering is suitable for processing Gaussian noise, median filtering is effective against salt-and-pepper noise, and bilateral filtering preserves edge information while denoising. In scenes with uneven lighting or low contrast, contrast enhancement can highlight target features. Sharpening enhances the contour features of the target, helping to obtain more accurate target boundary and shape information in subsequent processing.

[0045] In this embodiment, step S102 involves performing foreground detection on the preprocessed image of the current frame to obtain a binarized foreground mask for the current frame. This includes: dynamically constructing a background difference model using a moving average method and updating the background difference model in real time; calculating the absolute difference image between the preprocessed image of the current frame and the background difference model of the previous frame; determining whether the pixel grayscale in the absolute difference image is greater than a preset foreground detection threshold. If so, the binarized foreground mask of the current frame with a pixel mask value of 255 is output; otherwise, the binarized foreground mask of the current frame with a pixel mask value of 0 is output.

[0046] Understandably, the essence of the moving average method (also called the moving average method) is to continuously update the background model by using a weighted average of the current frame image and the historical background model, so that the background model always closely matches the real static state of the current scene.

[0047] In this embodiment, the update formula for the background difference model is:

[0048]

[0049]

[0050] In the formula, Indicates the first t Frame-to-background difference model, Indicates the first Frame-to-background difference model, t Indicates the current frame number. Indicates the background model update rate. Indicates the current frame preprocessed image in grayscale value at that location Indicates the first t Pixel mask values ​​of the frame binarized foreground mask. This represents the initial background model update rate. α s This represents the update rate of the stable background model. N Indicates the stable frame rate.

[0051] That is, if the current frame number t Less than stable frame rate N The update rate then decays linearly if the current frame number t Greater than or equal to stable frame rate N If so, a stable update rate will be used.

[0052] In this embodiment, the average grayscale value of the first frame or the first few frames of the input image sequence is used as the initial background model B0.

[0053] In this embodiment, moving targets in a video sequence cause significant changes in the pixel grayscale values ​​of their respective regions within a short period, while the pixel values ​​of the static background remain relatively stable. Therefore, by calculating the difference between the current frame and the background, the moving region can be effectively separated from the background.

[0054] In this embodiment, the core purpose of calculating the absolute difference image between the current frame preprocessed image and the previous frame background difference model is to quantify the pixel-level difference between the current frame preprocessed image and the background reference, that is, to calculate the absolute difference between two gray values ​​at the same coordinate, and the result is used as the gray value of the corresponding pixel in the difference image. The OpenCV function `absdiff` can be used to calculate the absolute difference image between the current frame preprocessed image and the previous frame background difference model, and returns a difference array of the same size as the current frame preprocessed image.

[0055] In this embodiment, step S103 involves performing morphological operations on the binarized foreground mask of the current frame to obtain the processed foreground mask of the current frame, and performing connected component analysis on the processed foreground mask of the current frame to obtain multiple connected components. Connected components with an area greater than a preset connected component area threshold are then selected as the moving targets of the current frame. This includes: performing morphological operations on the binarized foreground mask of the current frame to obtain the processed foreground mask of the current frame; performing connected component labeling on the processed foreground mask of the current frame using an 8-connectivity or 4-connectivity criterion to obtain multiple connected components; extracting the attributes of each connected component to obtain the bounding box information of the connected components; and selecting connected components with an area greater than a preset connected component area threshold as the moving targets of the current frame based on the bounding box information of the connected components.

[0056] In this embodiment, morphological operations are performed on the binarized foreground mask of the current frame to obtain the processed foreground mask of the current frame, including but not limited to one or more of the following:

[0057] a. Erosion operation: The center point of the structuring element that is completely contained within the image area is retained as the foreground, the foreground area is "shrunken", and isolated points and small protrusions are removed.

[0058] b. Dilation operation: The center point where the structuring element intersects with the foreground of the image is set as the foreground, which "expands" the foreground area, fills the holes, and connects the broken parts.

[0059] c. Opening operation: First perform erosion, then perform dilation to remove isolated small points, burrs and noise, while keeping the overall shape and size of the object basically unchanged.

[0060] d. Closing operation: First perform the dilation operation, then the erosion operation to fill small holes and cracks, connect adjacent but disconnected objects, smooth the boundaries, and maintain the overall shape.

[0061] In this embodiment, 8-connectivity means that adjacent pixels in the top, bottom, left, right and four diagonal directions of a pixel are considered connected, while 4-connectivity means that only adjacent pixels in the top, bottom, left and right directions of a pixel are considered connected.

[0062] In this embodiment, a two-pass scanning method, a seed filling method, or a run-length-based connected component analysis method is used.

[0063] In this embodiment, the attributes of the connected domain also include the total number of pixels included in the connected domain, the geometric center position of the connected domain, and the sequence of external boundary points of the connected domain.

[0064] In this embodiment, after foreground detection and morphological operations, a large number of tiny, non-target areas still exist in the image. These may originate from image sensor noise, sudden changes in illumination, slight perturbations in the background subtraction model, and isolated points not completely eliminated by morphological operations. Examples include isolated bright / dark spots caused by thermal noise, and subtle changes in reflections and shadow edges.

[0065] These noise regions share the common characteristic of being extremely small, differing by orders of magnitude from the area of ​​a real moving target (even a tiny one). By setting a reasonable lower limit threshold for the area (i.e., a preset connected component area threshold), these regions can be directly identified as noise and filtered out, thus avoiding misclassification as moving targets, fundamentally reducing false detections, and ensuring the accuracy of the final counting results.

[0066] Understandably, the area lower limit threshold is based on the essential difference between the area of ​​noise and the area of ​​the real target. That is, the area lower limit threshold is related to the size of the real moving target. The area of ​​the noise region (sensor thermal noise, residual light abrupt changes, isolated points, etc.) is usually only 1 to 10 pixels, while the area of ​​the real moving target (even mites, tiny parts, etc.) is often more than 10 times that of the noise (order of magnitude difference).

[0067] In this embodiment, the formula for calculating the number of sub-targets in the current frame in step S104 is expressed as follows:

[0068]

[0069]

[0070] In the formula, Indicates the first The area of ​​each undivided target, where n represents the number of undivided targets. This represents the average area of ​​all targets that were not split. N i This represents the count of moving targets in each current frame. λ This indicates the splitting quantity control factor. This represents the width of the bounding box of the moving target in the current frame. This indicates the length of the bounding box of the moving target in the current frame. This represents the bounding box size of the moving target in the current frame. w s , h s ) represents the bounding box splitting threshold.

[0071] λ A value greater than 0 indicates that when λ > 1, the number of sub-targets will be relatively small, which is suitable for scenarios where the target sizes are significantly different or densely packed. When λ < 1, the number of sub-targets will be relatively large, which is suitable for scenarios requiring more precise subdivision and avoiding omissions.

[0072] During the segmentation process, the size of other normal targets in the image is used as a reference to dynamically determine how many reasonable virtual units each bounding box with a size greater than the preset bounding box segmentation threshold should be segmented into, thereby effectively solving the problem of inaccurate counting caused by target adhesion.

[0073] It should be noted that the preset bounding box splitting threshold is not a fixed value, but is adjusted based on the actual size of the target to be tracked.

[0074] In this embodiment, in step S105, the unsplit current frame moving targets and two or more current frame sub-targets are collectively referred to as current frame valid targets. Based on the target matching and state management tracking counting method, the state of the current frame valid targets is updated through association, creation, and removal mechanisms. After completing the state update of all current frame valid targets, the number of current frame valid targets is counted, including: collectively referring to the unsplit current frame moving targets and current frame sub-targets as current frame valid targets; matching each current frame valid target one by one with the valid targets in the previous frame active target tracking list, and updating the state according to the matching results to form a current active target tracking list; and counting the number of current frame valid targets in the current active target tracking list.

[0075] In this embodiment, each valid target in the current frame is matched one by one with valid targets in the active target tracking list of the previous frame, and the status is updated according to the matching results to form the current active target tracking list. This includes: recording the bounding box coordinates of each valid target in the current frame, and calculating the center point coordinates of the valid targets in the current frame based on the bounding box coordinates; for each valid target in the current frame, calculating the center point distance between the valid target in the current frame and each valid target in the previous active target tracking list based on the center point coordinates; if the center point distance between the valid target in the current frame and any valid target in the previous active target tracking list is less than a preset merging distance, then the valid target in the current frame is determined to be successfully matched, and the minimum center point distance in the previous active target tracking list is used. The ID of the valid target is updated to match the ID of the valid target in the current frame, and the active frame count of the valid target in the current frame is increased. It is then added to the active target tracking list of the current frame. If the distance between the center point of the valid target in the current frame and the center point of all valid targets in the previous active target tracking list is not less than the preset merging distance, the match is determined to be unsuccessful. A new ID is assigned to the unmatched valid target in the current frame, and it is added to the active target tracking list of the current frame. For valid targets in the previous active target tracking list that are not matched by valid targets in the current frame, the active frame count is reduced, and they are added to the active target tracking list of the current frame. When the active frame count of a valid target is 0, the corresponding valid target is deleted from the active target tracking list of the current frame.

[0076] Understandably, the preset merging distance is the core judgment threshold for inter-frame target matching. Essentially, it is the Euclidean distance threshold (unit: pixels) in the pixel coordinate system, used to determine whether the effective target in the current frame and the active target in the previous frame are the same target.

[0077] The larger the target size and the higher the imaging resolution, the greater the merging distance can be; conversely, the smaller the target size and the lower the imaging resolution, the greater the merging distance can be. The preset merging distance = k × target feature size (k is a scaling factor, ranging from 0.3 to 0.8). For example, assuming the target is a mite, the diagonal length of the bounding box of a single mite is selected as the target feature size. The median width of a single mite is 10 pixels, and the median height is 8 pixels, so the target feature size is approximately 12.8 pixels. Since mites move very slowly, with a maximum inter-frame displacement typically only 2-4 pixels, and k is set to 0.35, the preset merging distance is approximately 4 pixels.

[0078] In this embodiment, the lifecycle management mechanism effectively distinguishes between valid targets that temporarily disappear and those that have truly left. By gradually reducing the number of active frames instead of immediately deleting them, it can tolerate situations where valid targets are temporarily occluded or fail to be detected, thus improving the robustness of tracking. When a valid target remains unmatched, the number of active frames drops to zero, indicating that the valid target has left or disappeared, and it is removed from the active target tracking list of the current frame.

[0079] The multi-moving target tracking and counting method in the embodiments of the present invention has been described above. The apparatus in the embodiments of the present invention is described below. Please refer to [link / reference]. Figure 2 The embodiments of the multi-moving target tracking and counting device in this invention include:

[0080] The preprocessing module 201 is used to acquire the current frame of the input image sequence in chronological order, and to preprocess the current frame to obtain the preprocessed image of the current frame;

[0081] Foreground detection module 202 is used to perform foreground detection on the preprocessed image of the current frame to obtain a binarized foreground mask of the current frame;

[0082] The analysis module 203 is used to perform morphological operations on the binarized foreground mask of the current frame to obtain the processed foreground mask of the current frame, and to perform connected component analysis on the processed foreground mask of the current frame to obtain multiple connected components, and to select the connected components with an area greater than a preset connected component area threshold as the moving target of the current frame.

[0083] The splitting module 204 is used to assign an ID to the current frame moving target and record the bounding box information of the current frame moving target. When the bounding box size of the current frame moving target is greater than a preset bounding box splitting threshold, the corresponding current frame moving target is split into at least two current frame sub-targets according to the bounding box information of the current frame moving target.

[0084] The counting module 205 is used to collectively refer to the unsplit current frame motion targets and two or more current frame sub-targets as current frame valid targets. Based on the tracking and counting method of target matching and state management, it updates the state of current frame valid targets through association, creation and removal mechanisms, and counts the number of current frame valid targets after completing the state update of all current frame valid targets.

[0085] In this embodiment, the moving target region is accurately extracted through foreground detection and morphological operations. Interference is eliminated by combining connected component analysis and area filtering to ensure the accuracy of initial target recognition. Moreover, for large targets whose bounding box size exceeds the bounding box splitting threshold, the oversized targets are flexibly split according to the bounding box splitting threshold to ensure that each sub-target can be independently identified and tracked, which greatly improves the precision and completeness of target detection. It effectively solves the problem of recognition omission caused by target overlap, occlusion or excessive size, and greatly improves the completeness of target detection in complex scenes. In addition, in the tracking and counting stage, by recording the bounding box information of valid targets, and with the dynamic state management mechanism of association, creation and removal, continuous tracking and real-time state updates of valid targets are realized. This not only ensures the continuity of target tracking, but also handles the addition and disappearance of targets in a timely manner, avoiding duplicate or missed counts.

[0086] Figure 2 The structure of the multi-moving target tracking and counting device shown does not constitute a limitation on the multi-moving target tracking and counting device, and can implement the steps of the multi-moving target tracking and counting methods provided in the above-described method embodiments.

[0087] above Figure 2 The multi-moving target tracking and counting device in the embodiments of the present invention will be described in detail from the perspective of modular functional entities. The multi-moving target tracking and counting device in the embodiments of the present invention will be described in detail from the perspective of hardware processing.

[0088] Figure 3 This is a schematic diagram of the structure of a multi-moving target tracking and counting device provided in an embodiment of the present invention. The device 300 can vary considerably due to different configurations or performance characteristics. It may include one or more central processing units (CPUs) 310 (e.g., one or more processors) and a memory 320, and one or more storage media 330 (e.g., one or more mass storage devices) for storing application programs 333 or data 332. The memory 320 and storage media 330 can be temporary or persistent storage. The program stored in the storage media 330 may include one or more modules (not shown), each module including a series of instruction operations on the device 300. Furthermore, the processor 310 may be configured to communicate with the storage media 330 and execute the series of instruction operations in the storage media on the device 300.

[0089] Device 300 may also include one or more power supplies 340, one or more wired or wireless network interfaces 350, one or more input / output interfaces 360, and / or one or more operating systems 331, such as Windows Server, Mac OS X, Unix, Linux, FreeBSD, etc.

[0090] This invention also provides a computer-readable storage medium, which can be a non-volatile computer-readable storage medium or a volatile computer-readable storage medium. The computer-readable storage medium stores instructions that, when executed on a computer, cause the computer to perform the steps of a multi-moving target tracking and counting method.

[0091] Those skilled in the art will clearly understand that, for the sake of convenience and brevity, the specific working process of the system, device, or unit described above can be referred to the corresponding process in the foregoing method embodiments, and will not be repeated here.

[0092] If the integrated unit is implemented as a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention, in essence, or the part that contributes to the prior art, or all or part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of the present invention. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0093] The above-described embodiments are only used to illustrate the technical solutions of the present invention, and are not intended to limit it. Although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features. Such modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. A method for tracking and counting multiple moving targets, characterized in that, include: The current frame of the input image sequence is obtained in chronological order, and the current frame is preprocessed to obtain the preprocessed image of the current frame. Perform foreground detection on the preprocessed image of the current frame to obtain the binarized foreground mask of the current frame; Morphological operations are performed on the binarized foreground mask of the current frame to obtain the processed foreground mask of the current frame. Connected component analysis is then performed on the processed foreground mask of the current frame to obtain multiple connected components. Connected components with an area greater than a preset connected component area threshold are selected as the moving target of the current frame. An ID is assigned to the moving target in the current frame, and the bounding box information of the moving target in the current frame is recorded. When the bounding box size of the moving target in the current frame is greater than a preset bounding box splitting threshold, the corresponding moving target in the current frame is split into at least two sub-targets in the current frame based on the bounding box information of the moving target in the current frame. The formula for calculating the number of sub-targets in the current frame is expressed as follows: In the formula, Indicates the first The area of ​​each undivided target, where n represents the number of undivided targets. This represents the average area of ​​all targets that were not split. N i This represents the count of moving targets in each current frame. λ Indicates the split quantity control factor. This represents the width of the bounding box of the moving target in the current frame. This indicates the length of the bounding box of the moving target in the current frame. This represents the bounding box size of the moving target in the current frame. w s , h s () indicates the bounding box splitting threshold; The unsplit moving target in the current frame and two or more sub-targets in the current frame are collectively referred to as the valid targets in the current frame. Based on the tracking and counting method of target matching and state management, the state of the valid targets in the current frame is updated through association, creation and removal mechanisms. After the state update of all valid targets in the current frame is completed, the number of valid targets in the current frame is counted.

2. The multi-moving target tracking and counting method as described in claim 1, characterized in that, The step of performing foreground detection on the preprocessed image of the current frame to obtain the binarized foreground mask of the current frame includes: A background difference model is dynamically constructed using the moving average method, and the background difference model is updated in real time. Calculate the absolute difference image between the preprocessed image of the current frame and the background difference model of the previous frame; Determine whether the pixel grayscale in the absolute difference image is greater than a preset foreground detection threshold. If yes, output the current frame binarized foreground mask with a pixel mask value of 255. If no, output the current frame binarized foreground mask with a pixel mask value of 0.

3. The multi-moving target tracking and counting method as described in claim 2, characterized in that, The update formula for the background difference model is: In the formula, Indicates the first t Frame-to-background difference model, Indicates the first Frame-to-background difference model, t Indicates the current frame number. Indicates the background model update rate. Indicates the current frame preprocessed image in grayscale value at that location Indicates the first t Pixel mask values ​​of the frame binarized foreground mask. α 0 represents the initial background model update rate. α s This represents the update rate of the stable background model. N Indicates the stable frame rate.

4. The multi-moving target tracking and counting method as described in claim 1, characterized in that, The step of performing morphological operations on the binarized foreground mask of the current frame to obtain the processed foreground mask of the current frame, and performing connected component analysis on the processed foreground mask of the current frame to obtain multiple connected components, and selecting connected components with an area greater than a preset connected component area threshold as the moving target of the current frame, includes: Perform morphological operations on the binarized foreground mask of the current frame to obtain the processed foreground mask of the current frame; By applying the 8-connectivity or 4-connectivity criterion to perform connected component labeling on the foreground mask after processing the current frame, multiple connected components are obtained; Extract the attributes of each connected component to obtain the bounding box information of the connected components; Based on the bounding box information of connected components, connected components with an area greater than a preset connected component area threshold are selected as moving targets in the current frame.

5. The multi-moving target tracking and counting method as described in claim 1, characterized in that, The unsplit current frame moving target and two or more current frame sub-targets are collectively referred to as current frame valid targets. Based on a target matching and state management tracking counting method, the state of current frame valid targets is updated through association, creation, and removal mechanisms. After updating the state of all current frame valid targets, the number of current frame valid targets is counted, including: Unsplit moving targets and sub-targets in the current frame are collectively referred to as valid targets in the current frame; Each valid target in the current frame is matched one by one with the valid targets in the active target tracking list of the previous frame, and the status is updated according to the matching results to form the current active target tracking list; Count the number of valid targets in the current frame in the current active target tracking list.

6. The multi-moving target tracking and counting method as described in claim 5, characterized in that, The step of matching each valid target in the current frame with the valid targets in the active target tracking list of the previous frame, and updating the status according to the matching results to form the current active target tracking list includes: Record the bounding box coordinates of each valid target in the current frame, and calculate the center point coordinates of the valid targets in the current frame based on the bounding box coordinates; For each valid target in the current frame, the distance between the center point of the valid target in the current frame and the center point of each valid target in the previous active target tracking list is calculated based on the center point coordinates; If the distance between the center point of the current frame's valid target and any valid target in the previous active target tracking list is less than a preset merging distance, then the current frame's valid target is determined to be successfully matched. The ID of the successfully matched current frame's valid target is updated using the ID of the valid target with the smallest center point distance in the previous active target tracking list, and the number of active frames of the successfully matched current frame's valid target is increased and added to the current frame's active target tracking list. If the distance between the center point of the current frame's valid target and the center point of all valid targets in the previous active target tracking list is not less than the preset merging distance, then the matching is determined to be a failure. A new ID is then assigned to the current frame's valid target that failed to match, and it is added to the current frame's active target tracking list. For valid targets in the previous active target tracking list that are not matched by valid targets in the current frame, reduce the number of active frames and add them to the active target tracking list of the current frame. When the number of active frames for a valid target is 0, remove the corresponding valid target from the active target tracking list of the current frame.

7. A multi-moving-target tracking and counting device, characterized in that, include: The preprocessing module is used to acquire the current frame of the input image sequence in chronological order, and to preprocess the current frame to obtain the preprocessed image of the current frame; The foreground detection module is used to perform foreground detection on the preprocessed image of the current frame to obtain the binarized foreground mask of the current frame; The analysis module is used to perform morphological operations on the binarized foreground mask of the current frame to obtain the processed foreground mask of the current frame, and to perform connected component analysis on the processed foreground mask of the current frame to obtain multiple connected components, and to select the connected components with an area greater than a preset connected component area threshold as the moving target of the current frame. The splitting module is used to assign an ID to the moving target in the current frame and record the bounding box information of the moving target in the current frame. When the bounding box size of the moving target in the current frame is greater than a preset bounding box splitting threshold, the corresponding moving target in the current frame is split into at least two sub-targets in the current frame according to the bounding box information of the moving target in the current frame. The formula for calculating the number of sub-targets to be split in the current frame is expressed as follows: In the formula, Indicates the first The area of ​​each undivided target, where n represents the number of undivided targets. This represents the average area of ​​all targets that were not split. N i This represents the count of moving targets in each current frame. λ Indicates the split quantity control factor. This represents the width of the bounding box of the moving target in the current frame. This indicates the length of the bounding box of the moving target in the current frame. This represents the bounding box size of the moving target in the current frame. w s , h s () indicates the bounding box splitting threshold; The counting module is used to collectively refer to the unsplit moving targets in the current frame and two or more sub-targets in the current frame as valid targets in the current frame. Based on the tracking and counting method of target matching and state management, it updates the state of valid targets in the current frame through association, creation and removal mechanisms, and counts the number of valid targets in the current frame after completing the state update of all valid targets in the current frame.

8. A multi-moving-target tracking and counting device, characterized in that, It includes a memory and at least one processor, wherein the memory stores computer-readable instructions; The at least one processor invokes the computer-readable instructions in the memory to perform various steps of the multi-moving target tracking and counting method as described in any one of claims 1-6.

9. A computer-readable storage medium storing computer-readable instructions thereon, characterized in that, When the computer-readable instructions are executed by a processor, they implement the steps of the multi-moving target tracking and counting method as described in any one of claims 1-6.

Images