A residue detection method that integrates dual-background models and improved YOLO-P
By fusing a dual-background model and an improved YOLO-P model, the problems of high model complexity and low detection accuracy in legacy object detection are solved, achieving more efficient and accurate legacy object detection.
Patent Information
- Application Number
- CN202410851408.3
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2024-06-28
- Publication Date
- 2026-11-17
- Estimated Expiration
- 2044-06-28
AI Technical Summary
Existing deep learning networks suffer from high model complexity, low real-time performance, and poor detection accuracy in legacy object detection.
By integrating a dual-background model and an improved YOLO-P model, and by constructing a Gaussian mixture model, adaptive learning rate adjustment, an improved PFSM model, and a YOLO-P model structure, the detection speed and accuracy are improved.
It improves the accuracy and robustness of debris detection, enhances the ability to judge complex scenes and occlusion states, and reduces computational complexity.
Smart Images

Figure CN118864811B_ABST
Abstract
Description
Technical Field
[0001] This invention generally relates to the fields of object detection and deep learning, and more specifically, to a legacy detection method that integrates a dual-background model and an improved YOLO-P. Background Technology
[0002] With the widespread use of video surveillance equipment and the increasing number of devices, relying on manual monitoring to process video data and provide real-time feedback has become impractical. To prevent potential accidents and address them promptly, avoiding unforeseen risks and losses, implementing abandoned object detection methods has become an effective strategy with broad application prospects in the field of security monitoring.
[0003] Abandoned object detection is a key task in the field of video surveillance. Traditional computer vision methods construct background models, such as frame difference, Gaussian mixture models, and the ViBe algorithm. In recent years, with the continuous development of deep learning technology, a large number of deep learning networks have been proposed and used for abandoned object detection, including cascaded convolutional neural network (CNN) models and YOLO models, etc., to perform object detection and thus identify abandoned items.
[0004] However, legacy detection methods that rely entirely on deep learning networks suffer from problems such as high model complexity, low real-time performance, and poor detection accuracy. Summary of the Invention
[0005] According to embodiments of the present invention, a method for detecting remnants by fusing a dual-background model and an improved YOLO-P model is provided. This method can improve the overall detection speed and accuracy of the model.
[0006] The aforementioned method for detecting remnants using a fusion dual-background model and improved YOLO-P includes:
[0007] Obtain the input scene image and construct a dual-background model;
[0008] Obtain the state information of the dual-background model, input it into the improved PFSM model, and output the suspicious object;
[0009] The suspicious object is input into the improved YOLO-P model, and the output is the detection layer;
[0010] The detection layer is subjected to regression, localization, and classification processing to obtain the detection results of the residue.
[0011] Furthermore, the construction of the dual-background model includes:
[0012] Construct a Gaussian mixture model;
[0013] Set an adaptive learning rate;
[0014] The Gaussian mixture model is updated based on an adaptive learning rate to obtain a dual-background model; wherein the adaptive learning rate is used to adjust the update rate.
[0015] Furthermore, the construction of the Gaussian mixture model includes:
[0016]
[0017] Wherein, P(X) t ) represents a Gaussian model; K represents the total number of pixels; t represents time step; j represents the number of pixels; ω j,t η(X) represents the weight of the j-th Gaussian distribution at time t. t ,μ j,t ,∑j,t) represents the Gaussian distribution density at time t; X t μ is the pixel value at time t. j,t Let be the expected value of the j-th Gaussian distribution at time t; ∑j,t is the covariance matrix of the j-th Gaussian distribution at time t; n is the dimension of the pixel; and e is the base of the natural logarithm.
[0018] Furthermore, the Gaussian mixture model is updated based on an adaptive learning rate to obtain a dual-background model, including:
[0019] An adaptive learning rate based on an update constant is constructed, wherein the value range of the update constant is set to a first interval or a second interval; the minimum value of the second interval is greater than the maximum value of the first interval.
[0020] When the value of the first constant is within the first interval, the pixels of the current frame are matched with a Gaussian distribution. The mean, weights and parameters of the successfully matched Gaussian distribution are updated based on the successfully matched pixels to obtain the long-term background model.
[0021] When the value of the first constant is within the second interval, the pixels of the current frame are matched with a Gaussian distribution. The mean, weights and parameters of the successfully matched Gaussian distribution are updated based on the successfully matched pixels to obtain the short-term background model.
[0022] The long-term background model and the short-term background model constitute a dual background model;
[0023] Wherein, if the Gaussian distribution satisfies the matching condition, then the pixel point is successfully matched with the Gaussian distribution; the matching condition is:
[0024] |X t -μ j,t-1 |≤Cσ j,t-1
[0025] Among them, X t μ is the pixel value at time t. j,t-1 Let σ be the expected value of the j-th Gaussian distribution at time t; C is a constant; σ j,t-1 Let be the variance of the j-th Gaussian distribution at the time t before the current time.
[0026] Furthermore, the adaptive learning rate is:
[0027]
[0028] Where α is the learning rate; λ0 is a constant; γ is the influence factor; T is the current frame number; T0 is the frame number threshold; ε is the proportion of the detected target pixel in the current frame to the total pixels in the current frame; δ is the illumination transformation factor; N obj N is the number of pixels identified as the target in the current frame; all H represents the total number of pixels in the current frame. t-1 H represents the information entropy of the previous frame image. t This represents the information entropy of the current frame image.
[0029] Furthermore, it also includes setting a frame period, performing Gaussian distribution weight detection based on the frame period, and removing Gaussian distributions that meet the weight conditions;
[0030] The weighting condition is as follows:
[0031] (ω j,t <ω0)&&((ω j,t / σ j,t )<(ω0 / σ0))&&(K>K min )
[0032] Where, ω j,t ω represents the weights of the j-th Gaussian distribution at time t; ω0 represents the initial weights; σ j,t For σ j,t-1 σj is the variance of the j-th Gaussian distribution at time t; σ0 is the initial variance; K is the number of Gaussian distributions in the current background model; K min This represents the minimum number of Gaussian distributions in the current background model.
[0033] Furthermore, obtaining the state information of the dual-background model includes:
[0034] The long-term background model is binarized to obtain a binary image of the long-term background model, which serves as the state information F of the long-term background model. L ; and binarize the short-term background model to obtain a binary image of the short-term background model as the state information F of the short-term background model. S ;
[0035] Based on the state information F of the long-term background modelL and the state information F of the short-term background model S Calculate the state of the current pixel:
[0036] S i,t =F L (i)(t)F S (i)(t)
[0037] Among them, S i,t The state of the current pixel; F L (i)(t),F S (i)(t)∈{0,1},F L When (i)(t) is 1, it indicates that the current pixel is a foreground pixel in the long-term background model, F L When (i)(t) is 0, it indicates that the current pixel is a background pixel under the long-term background model; F S When (i)(t) is 1, it indicates that the current pixel is a foreground pixel in the short-term background model, F S When (i)(t) is 0, it means that the current pixel is a background pixel under the short-term background model;
[0038] When F L (i)(t)=0 and F S When (i)(t)=0, S i,t A value of 00 indicates that at time t, pixel i is a static background pixel in both the long-term background model and the short-term background model.
[0039] When F L (i)(t)=0 and F S When (i)(t)=1, S i,t A value of 0 or 1 indicates that at time t, pixel i is occluded by other objects.
[0040] When F L (i)(t)=1 and F S When (i)(t)=0, S i,t A value of 10 indicates that at time t, pixel i may be a static foreground pixel;
[0041] When F L (i)(t)=1 and F S When (i)(t)=1, S i,t The value is 11, which means that at time t, pixel i is a dynamic foreground pixel in both the long-term background model and the short-term background model.
[0042] Furthermore, the state information from the dual-background model is input into the improved PFSM model, which outputs suspicious objects, including:
[0043] In response to the current pixel's state value being 11, the state of the PFSM model changes from the Start state to the dynamic foreground state MF.
[0044] If the pixel value of the current pixel remains unchanged within a certain period of time, the short-term background model will update the current pixel into the background while the long-term background model will not be updated. At this time, the state value of the current pixel is 10, which triggers the state of the PFSM model to change from the dynamic foreground state MF to the candidate static foreground state CSF, and the first time period timing is started at the same time.
[0045] During the first time period, if the current pixel's state value becomes 11, and the first time period is less than the first time threshold T... st If the first time period continues, the timing will continue, and the timing for the second time period will start simultaneously, triggering the transformation of the PFSM model's state from the candidate static foreground state (CSF) to the occluded candidate static foreground state (OCSF); this continues until the first time period reaches the first time threshold T. st And the second time period is less than the second time threshold T sh When the state of the PFSM model is triggered, it changes from the occluded candidate static foreground state OCSF to the occluded static foreground object OSFO.
[0046] During the second time period, if the current pixel's state value becomes 10, and the second time period is less than the second time threshold T... sh This indicates that the current pixel is not occluded, triggering the PFSM model's state to change from the occluded candidate static foreground state OCSF to the candidate static foreground state CSF, simultaneously terminating the second time period timing while maintaining the first time period timing; when the first time period reaches the first time threshold T... st When the state of the PFSM model is triggered, it changes from the candidate static foreground state CSF to the static foreground object state SFO.
[0047] When the second time period reaches the second time threshold T sh When the current pixel's state value is 11, the state of the PFSM model is directly converted to the occluded static foreground object OSFO.
[0048] If the state output of the PFSM model is the static foreground object state (SFO) and / or the occluded static foreground object state (OSFO), then the current pixel is considered a suspicious pixel.
[0049] After traversing all pixels, the set of all suspicious pixels is taken as the suspicious object.
[0050] Furthermore, the improved YOLO-P model includes a skeleton, a neck, and a head.
[0051] The backbone component sequentially includes a CBS module, a first GhostConv module, a first C3Ghost module, a second GhostConv module, a second C3Ghost module, a third GhostConv module, a third C3Ghost module, a fourth GhostConv module, a fourth C3Ghost module, a first CBAM module, and an SPPF module, wherein the CBS module serves as the input terminal of the improved YOLO-P model;
[0052] The neck section sequentially includes a fifth GhostConv module, a second CBAM module, a first multi-connection module, a fifth C3Ghost module, a sixth GhostConv module, a third CBAM module, a second multi-connection module, and a sixth C3Ghost module; the output of the SPPF module serves as the input of the fifth GhostConv module; the outputs of the third GhostConv module, the third C3Ghost module, the fourth GhostConv module, and the second CBAM module serve as the input of the first multi-connection module; the outputs of the first C3Ghost module, the second GhostConv module, the second C3Ghost module, and the third CBAM module serve as the input of the second multi-connection module; the output of the sixth C3Ghost module is converted into a first detection layer after a convolution operation.
[0053] The header section sequentially includes a seventh GhostConv module, a fourth CBAM module, a first connection module, the seventh C3Ghost module, an eighth GhostConv module, a second connection module, and the eighth C3Ghost module; the output of the sixth C3Ghost module serves as the input of the seventh GhostConv module; the outputs of the sixth GhostConv module and the fourth CBAM module serve as the input of the first connection module; the outputs of the fifth GhostConv module and the eighth GhostConv module serve as the input of the second connection module; the output of the seventh C3Ghost module generates a second detection layer after a convolution operation; the output of the eighth C3Ghost module generates a third detection layer after a convolution operation.
[0054] The first detection layer, the second detection layer, and the third detection layer serve as the outputs of the improved YOLO-P model.
[0055] Furthermore, the multi-connection module includes an upsampling module, a downsampling module, and a connection module;
[0056] The upsampling module is used to upsample the results obtained from the high-level feature layer and output them to the connection module.
[0057] The downsampling module is used to downsample the results obtained from the low-level feature layer and output them to the connection module.
[0058] The connection module is used to connect the outputs of the upsampling module and the downsampling module, as well as the results obtained from the same feature layer. The connection result is then convolved and used as the output of the multi-connection module.
[0059] It should be understood that the description in the Summary of the Invention is not intended to limit the key or essential features of the embodiments of the present invention, nor is it intended to restrict the scope of the invention. Other features of the invention will become readily apparent from the following description. Attached Figure Description
[0060] The above and other features, advantages, and aspects of the various embodiments of the present invention will become more apparent from the accompanying drawings and the following detailed description. In the drawings, the same or similar reference numerals denote the same or similar elements, wherein:
[0061] Figure 1 A flowchart illustrating the fusion dual-background model and improved YOLO-P residue detection method according to an embodiment of the present invention is shown;
[0062] Figure 2 A block diagram of a dual-background model construction method according to an embodiment of the present invention is shown;
[0063] Figure 3 A block diagram of a method for obtaining state information of a dual-background model according to an embodiment of the present invention is shown;
[0064] Figure 4 A state diagram of an improved PFSM model according to an embodiment of the present invention is shown;
[0065] Figure 5 A structural diagram of a YOLO-P model network according to an embodiment of the present invention is shown;
[0066] Figure 6 A structural diagram of the CBS module of the YOLO-P model network according to an embodiment of the present invention is shown;
[0067] Figure 7 A structural diagram of the GhostConv module of a YOLO-P model network according to an embodiment of the present invention is shown;
[0068] Figure 8 A structural diagram of the C3Ghost module of a YOLO-P model network according to an embodiment of the present invention is shown;
[0069] Figure 9A structural diagram of the GhostBottleneck module of the C3Ghost module according to an embodiment of the present invention is shown;
[0070] Figure 10 A schematic diagram illustrating the GhostModule principle of a YOLO-P model network according to an embodiment of the present invention is shown.
[0071] Figure 11 A first multi-connection module structure diagram of a YOLO-P model network according to an embodiment of the present invention is shown;
[0072] Figure 12 A second multi-connection module structure diagram of a YOLO-P model network according to an embodiment of the present invention is shown;
[0073] Figure 13 A block diagram of the CBAM module of a YOLO-P model network according to an embodiment of the present invention is shown. Detailed Implementation
[0074] To make the objectives, technical solutions, and advantages of the embodiments of the present invention clearer, the technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.
[0075] Furthermore, the term "and / or" in this article is merely a description of the relationship between related objects, indicating that three relationships can exist. For example, A and / or B can represent: A existing alone, A and B existing simultaneously, or B existing alone. Additionally, the character " / " in this article generally indicates that the preceding and following related objects have an "or" relationship.
[0076] This invention improves the dual-background model by proposing a novel adaptive Gaussian mixture model to enhance the reliability of background modeling in complex scenes and under varying lighting conditions. It also refines the pixel-based finite state machine (PFSM) model for accurate extraction of static foreground objects and uses the proposed YOLO-P model for further evaluation of these objects, thus achieving object detection. This approach improves the overall accuracy and robustness of the model, enhances its ability to handle special scenarios such as occlusion, and increases detection accuracy.
[0077] Example 1:
[0078] Figure 1A flowchart illustrating the residue detection method of the fused dual-background model and improved YOLO-P according to an embodiment of the present invention is shown. The method includes:
[0079] S101. Obtain the input scene image and construct a dual-background model.
[0080] Figure 2 A block diagram of a dual-background model construction method according to an embodiment of the present invention is shown.
[0081] Specifically, the dual-background model is an extension of the background-foreground model, a foreground detection model used for dynamic scene analysis. It mainly consists of two parts: a long-term background model and a short-term background model. Both the long-term and short-term background models are built using a Gaussian mixture model. The long-term background model uses a lower learning rate to reduce the update rate, while the short-term background model uses a higher learning rate to accelerate the background update rate. The difference in update rates between the two models allows for the detection of corresponding static foreground objects.
[0082] In this embodiment, constructing the dual-background model includes:
[0083] S201. Construct a Gaussian mixture model.
[0084] In this embodiment, the construction of the Gaussian mixture model involves defining K Gaussian distributions to describe the pixel value for each pixel. Then, at time t, the current pixel value X is described. t The random probability distribution is as follows:
[0085]
[0086] Wherein, P(X) t ) represents a Gaussian model; K represents the total number of pixels; t represents time step; j represents the number of pixels; ω j,t η(X) represents the weight of the j-th Gaussian distribution at time t. t ,μ j,t ,∑j,t) represents the Gaussian distribution density at time t; X t μ is the pixel value at time t. j,t Let ∑ be the expected value of the j-th Gaussian distribution at time t; j,t Let be the covariance matrix of the j-th Gaussian distribution at time t; n is the dimension of the pixel; and e is the base of the natural logarithm.
[0087] By constructing a Gaussian mixture model, a dual-background model can be quickly built, which can effectively describe the current scene content and has good scene adaptability.
[0088] S202, Set an adaptive learning rate.
[0089] In this embodiment, the adaptive learning rate is:
[0090]
[0091] Where α is the learning rate; λ0 is a constant; γ is the influence factor; T is the current frame number; T0 is the frame number threshold; ε is the proportion of the detected target pixel in the current frame to the total pixels in the current frame; δ is the illumination transformation factor; N obj N is the number of pixels identified as the target in the current frame; all H represents the total number of pixels in the current frame. t-1 H represents the information entropy of the previous frame image. t This represents the information entropy of the current frame image.
[0092] Specifically, the long-term background model λ0 is usually ∈ [0.03, 0.06], and the short-term background model λ0 is usually λ0 ∈ [0.1, 0.3]. The proportion of the target pixel detected in the current frame to the total pixels in the current frame, ε, is used to describe the target complexity in the current frame. The higher the proportion of the target pixel, the higher the current target complexity, and the lower the required learning rate. The illumination transformation factor δ is used to determine the illumination change between the current frame and the previous frame. If the light changes drastically in the current frame, then δ is 1, so that the current learning rate is adjusted to a higher value to make the background state as stable as possible. When the state is stable, δ is 0.
[0093] In this embodiment, after initializing the background model, the update rate of the background model is adjusted according to changes in the current scene using a proposed adaptive learning rate. By setting an adaptive learning rate, the background model can better adapt to interference caused by changes in lighting and scene complexity, enhancing its adaptability and improving the accuracy of model detection.
[0094] S203. The Gaussian mixture model is updated based on an adaptive learning rate to obtain a dual-background model; wherein the adaptive learning rate is used to adjust the update rate.
[0095] In this embodiment, updating the Gaussian mixture model based on an adaptive learning rate to obtain a dual-background model includes:
[0096] An adaptive learning rate based on an update constant is constructed, and the value range of the update constant is set to a first interval or a second interval; the minimum value of the second interval is greater than the maximum value of the first interval, wherein the first interval is [0.1, 0.3] and the second interval is [0.03, 0.06].
[0097] When the value of the first constant is within the first interval, the pixels of the current frame are matched with a Gaussian distribution. Based on the successfully matched pixels, the mean, weights, and parameters of the successfully matched Gaussian distribution are updated to obtain the long-term background model.
[0098] When the value of the first constant is within the second interval, the pixels of the current frame are matched with a Gaussian distribution. Based on the successfully matched pixels, the mean, weights, and parameters of the successfully matched Gaussian distribution are updated to obtain the short-term background model.
[0099] The long-term background model and the short-term background model constitute a dual background model.
[0100] Wherein, if the Gaussian distribution satisfies the matching condition, then the pixel point is successfully matched with the Gaussian distribution; the matching condition is:
[0101] |X t -μ j,t-1 |≤Cσ j,t-1 (6)
[0102] Among them, X t μ is the pixel value at time t. j,t-1 Let be the expected value of the j-th Gaussian distribution at time t; C is a constant, typically taken as 2.5-3.5; σ j,t-1 Let be the variance of the j-th Gaussian distribution at the time t before the current time.
[0103] By using an adaptive learning rate to adjust its own value in real time according to changes in the scene, the background model can be updated reasonably, thereby improving the accuracy and reliability of model detection.
[0104] In this embodiment, updating the mean, weights, and parameters of the successfully matched Gaussian distribution based on the successfully matched pixels includes:
[0105] μ j,t =(1-ρ)μ j,t-1 +ρ·X t (7)
[0106]
[0107] ω j,t =(1-α)μ j,t +α·M(t) (9)
[0108]
[0109] ρ=αη(X t ,μ j,t ,∑j,t) (11)
[0110] Where, μ j,t Let μ be the expected value of the j-th Gaussian distribution at time t; j,t-1 X is the expected value of the j-th Gaussian distribution at time t; ρ is the update rate; tω represents the pixel value at time t; M(t) is the matching variable; Match indicates a successful match; DisMatch indicates a failed match; ω j,t α represents the weights of the j-th Gaussian distribution at time t; α is the learning rate. The parameter value at time t; The parameter value at time t-1; η(X) t ,μ j,t ∑j,t) represents the Gaussian distribution density at time t.
[0111] Specifically, if a pixel matches a Gaussian distribution, the matching variable M(t) = 1; if a pixel does not match a Gaussian distribution, M(t) = 0.
[0112] In this embodiment, the method further includes setting a frame period, performing Gaussian distribution weight detection based on the frame period, and removing Gaussian distributions that meet the weight conditions.
[0113] The weighting condition is as follows:
[0114] (ω j,t <ω0)&&((ω j,t / σ j,t )<(ω0 / σ0))&&(K>K min (12)
[0115] Where, ω j,t ω represents the weights of the j-th Gaussian distribution at time t; ω0 represents the initial weights; σ j,t σj is the variance of the j-th Gaussian distribution at time t; σ0 is the initial variance; K is the number of Gaussian distributions in the current background model, and the value of K is generally between 3 and 5 to ensure that there are enough Gaussian distributions to accurately describe the pixel distribution; K min This represents the minimum number of Gaussian distributions in the current background model.
[0116] In this embodiment, the Gaussian mixture model uses K Gaussian distributions to describe pixels. Different K values can be set according to the different needs of the background model; the larger the K value, the stronger the anti-interference capability against the background. However, in practical applications, the object detection scenario is complex and varied. If a fixed number of Gaussian distributions are used to represent each pixel, it cannot accurately describe the pixel value and will also result in a significant waste of computational resources. Therefore, during the background model update process, the number of Gaussian distributions can be adaptively adjusted by performing a weighted detection of the Gaussian distribution every f frames.
[0117] Specifically, if the Gaussian distribution parameters satisfy formula (12), then the Gaussian distribution is removed. By continuously and adaptively adjusting the Gaussian distribution K value, the model's adaptability to complex scenes is improved, while unnecessary waste of computational resources is reduced, thus lowering the model's complexity. By removing the Gaussian distribution that satisfies the weight conditions, the computational cost required to update the background model can be reduced with almost no impact on the detection performance, thereby improving the model's detection rate.
[0118] In this embodiment, as Figure 3 The diagram shown is a block diagram of a method for obtaining state information of a dual-background model according to an embodiment of the present invention. The method for obtaining state information of a dual-background model includes:
[0119] S301. Binarize the long-term background model to obtain a binary image of the long-term background model, which serves as the state information F of the long-term background model. L ; and binarize the short-term background model to obtain a binary image of the short-term background model as the state information F of the short-term background model. S .
[0120] S302, Based on the state information F of the long-term background model L and the state information F of the short-term background model S Calculate the state of the current pixel:
[0121] S i,t =F L (i)(t)F S (i)(t) (13)
[0122] Among them, S i,t The state of the current pixel; F L (i)(t),F S (i)(t)∈{0,1},F L When (i)(t) is 1, it indicates that the current pixel is a foreground pixel in the long-term background model, F L When (i)(t) is 0, it indicates that the current pixel is a background pixel under the long-term background model; F S When (i)(t) is 1, it indicates that the current pixel is a foreground pixel in the short-term background model, F S When (i)(t) is 0, it means that the current pixel is a background pixel under the short-term background model.
[0123] When F L (i)(t)=0 and F S When (i)(t)=0, S i,t A value of 00 indicates that at time t, pixel i is a static background pixel in both the long-term and short-term background models. When F... L(i)(t)=0 and F S When (i)(t)=1, S i,t A value of 0 or 1 indicates that at time t, pixel i is occluded by other objects. When F... L (i)(t)=1 and F S When (i)(t)=0, S i,t A value of 10 indicates that at time t, pixel i may be a static foreground pixel. When F L (i)(t)=1 and F S When (i)(t)=1, S i,t The value is 11, which means that at time t, pixel i is a dynamic foreground pixel in both the long-term background model and the short-term background model.
[0124] By binarizing pixels, the resulting values can be used to make a more accurate judgment on the current pixel state.
[0125] This invention proposes an adaptive Gaussian mixture model and applies it to a long-term and short-term dual-background model, which minimizes the interference from factors such as changes in lighting and target complexity in the object detection scenario, thereby improving the overall accuracy and robustness of the model.
[0126] S102. Obtain the state information of the dual-background model, input it into the improved PFSM model, and output the suspicious object.
[0127] Specifically, if only equation (10) is used to detect static foreground objects, the judgment can only be made based on the state of the pixel at the current moment, and the context cannot be reasonably considered. Therefore, its detection result is inaccurate. Based on the Finite State Machine (FSM) model, an improved pixel-based finite state machine model PFSM is proposed, which transforms the state according to different state information and adds an occlusion state to achieve accurate detection of static foreground objects. Among them, the added occlusion state refers to the state in which the detected object has been transformed into a candidate static foreground object and is then occluded by other objects.
[0128] In this embodiment, the state transition information of pixels at different times is used to construct a sequence pattern that identifies the current state of each pixel, and this pattern is used to detect static foreground objects. Here, MF represents moving foreground, CSF represents candidate static foreground, OCSF represents occluded candidate static foreground, SFO represents static foreground object, and OSFO represents occluded static foreground object. The state information of the dual-background model is input into the improved PFSM model, which outputs suspicious objects.
[0129] Specifically, the improved PFSM model employs two timers, Count1 and Count2, and a time threshold T. st and T shCount1 represents the dwell time of the foreground object, Count2 represents the occlusion time of the candidate static foreground object, and T st T represents the time threshold required for a static foreground object to be identified as a legacy. sh This represents the time required for the object obscuring the remaining items to be recognized as background by the short-term background model.
[0130] like Figure 4 The state diagram of the improved PFSM model according to an embodiment of the present invention is shown. The PFSM model states include:
[0131] (1) The initial state of the PFSM model is the Start state.
[0132] 1) If the response is to the current pixel's state S i,t When the value is 11, the state of the PFSM model changes from the Start state to the dynamic foreground state MF.
[0133] 2) If the response is to the current pixel's state S i,t The value is 00 / 01 / 10, and the state of the PFSM model remains unchanged in the Start state.
[0134] (2) When the PFSM model is in the dynamic foreground state MF, if the pixel value of the current pixel remains unchanged for a certain period of time, the short-term background model will update the current pixel into the background and the long-term background model will not be updated.
[0135] 1) If the current state value of the pixel is S i,t The change from 11 to 10 triggers the transformation of the PFSM model's state from the dynamic foreground state MF to the candidate static foreground state CSF, and simultaneously starts the first time period Count1 timing.
[0136] 2) If the current pixel state value is 11 and remains unchanged, keep the PFSM model state as the dynamic foreground state MF unchanged.
[0137] 3) If the current pixel state value is S i,t When the value changes from 11 to 00 / 01, the state of the PFSM model changes from the dynamic foreground state MF to the Start state.
[0138] (3) When the state of the PFSM model is the dynamic foreground state CSF:
[0139] 1) If the current pixel is no longer stationary or the current object is occluded, then the state value of the current pixel is S. i,t The value changes from 10 to 11. If Count1 in the first time period is less than the first time threshold T... stIf the first time period continues to be timed, the second time period will be timed simultaneously, triggering the PFSM model to change from the candidate static foreground state CSF to the occluded candidate static foreground state OCSF.
[0140] 2) During the first time period, if the current pixel's state value is S i,t When the state changes to 00 / 01, the state of the PFSM model changes from the dynamic foreground state CSF to the Start state.
[0141] (4) When the state of the PFSM model is the dynamic foreground state OCSF:
[0142] 1) If the current pixel's state value is S i,t It becomes 11 when Count1 reaches the first time threshold T in the first time period. st (This indicates that the current pixel may be continuously occluded and the remaining time reaches the threshold), and the second time period Count2 is less than the second time threshold T. sh When the state of the PFSM model is changed from the occluded candidate static foreground state OCSF to the occluded static foreground object OSFO, the PFSM model is triggered to change.
[0143] 2) If the current pixel's state value is S i,t It becomes 11 when Count2 in the second time period reaches the second time threshold T. sh This indicates that the current pixel may be continuously occluded and the occlusion time reaches a threshold, triggering the PFSM model to directly convert the state to the occluded static foreground object OSFO.
[0144] 3) If the current pixel's state value is S i,t The value becomes 10, and Count2 in the second time period is less than the second time threshold T. sh This indicates that the current pixel is not occluded, triggering the PFSM model's state to change from the occluded candidate static foreground state OCSF to the candidate static foreground state CSF. Simultaneously, the second time period Count2 is terminated, while the first time period Count1 continues timing. When the first time period Count1 reaches the first time threshold T... st When this occurs, the state of the PFSM model is triggered to change from the candidate static foreground state CSF to the static foreground object state SFO.
[0145] 4) If the current pixel's state value is S i,t When the state changes to 00 / 01, the state of the PFSM model changes from the dynamic foreground state OCSF to the Start state.
[0146] (5) If the state output of the PFSM model is the static foreground object state SFO and / or the occluded static foreground object OSFO, then the current pixel is considered a suspicious pixel.
[0147] (6) After traversing all pixels, the set of all suspicious pixels is taken as the suspicious object.
[0148] Specifically, when detecting abandoned objects, the focus should be on pixels with a binary representation of 10. This is because, in the long-term background model, this pixel remains a foreground pixel, indicating that it was in motion for a considerable period. In the short-term background model, it is a background object, suggesting that it has stopped moving recently. Therefore, this pixel is a static foreground pixel, meeting the relevant characteristics of abandoned objects, and should be considered a candidate static foreground object.
[0149] In this embodiment, the state information of the background model is input into the improved PFSM model to output suspicious objects. The improved pixel-based finite state machine model (PFSM) enhances the model's ability to judge special scenes such as occlusion states by increasing the search range for suspicious objects. By detecting pixel states through the PFSM model, the temporal information of pixels can be fully utilized and combined with spatial information, thereby more accurately outputting the required candidate static foreground objects (candidate remnants).
[0150] S103. Input the suspected object into the improved YOLO-P model and output the detection layer.
[0151] Figure 5 A structural diagram of the YOLO-P model network according to an embodiment of the present invention is shown.
[0152] In this embodiment, the improved YOLO-P model adopts the CSPDarknet-53 (Cross Stage Partial Network with Darknet-53) feature extraction network as a whole, introduces ghost modules, multi-connection modules and attention modules such as CBAM, and appropriately adjusts the model structure. The model includes a backbone, a neck and a head.
[0153] Specifically, the improved YOLO-P model is as follows:
[0154] (1) The input image is normalized and uniformly processed to a size of 640×640×3 (length×width×number of channels, where the length and width are in pixels).
[0155] (2) The features are initially extracted through the backbone and a feature map of size 20×20×1024 is obtained.
[0156] The backbone includes, in sequence, a CBS module, a first GhostConv module, a first C3Ghost module, a second GhostConv module, a second C3Ghost module, a third GhostConv module, a third C3Ghost module, a fourth GhostConv module, a fourth C3Ghost module, a first CBAM module, and an SPPF module, wherein the CBS module serves as the input terminal of the improved YOLO-P model.
[0157] Specifically, Figure 6 The diagram shows the structure of the CBS module of the YOLO-P model network according to an embodiment of the present invention. Each CBS module includes a convolution module (Conv), a batch normalization module (BN), and an activation module (SiLU) in sequence. The result of the input data of the CBS module passing through the convolution module (Conv), the batch normalization module (BN), and the activation module (SiLU) in sequence is used as the output data of the CBS module.
[0158] Specifically, Figure 7 The diagram shows the structure of the GhostConv module of the YOLO-P model network according to an embodiment of the present invention. Each GhostConv module includes a convolutional module (Conv), a cheap operation, and a connection module (Concat). The output data of the GhostConv module is obtained by inputting the output data of the convolutional module (Conv) and the output data of the cheap operation into the connection module (Concat).
[0159] Specifically, Figure 8 The diagram illustrates the structure of a C3Ghost module in a YOLO-P model network according to an embodiment of the present invention. Each C3Ghost module sequentially includes a CBS module, a Ghost Bottleneck module, and a concat module. The output data obtained after the input data of the C3Ghost module passes through the CBS module and the Ghost Bottleneck module, together with the output data obtained after the input data of the C3Ghost module passes through the CBS module, are used as input data. The result of this input to the concat module and the CBS module is used as the output data of the C3Ghost module.
[0160] in, Figure 9The diagram shows the structure of the Ghost Bottleneck module of the C3Ghost module according to an embodiment of the present invention. The Ghost Bottleneck module includes a GhostConv module, a batch normalization module (BN), and an activation module (SiLU). The result obtained by adding the input data of the Ghost Bottleneck module and the output data of the GhostConv module is used as the output data of the Ghost Bottleneck module.
[0161] (3) After further feature fusion via the Feature Pyramid module SPPF, the input is the neck, and further feature extraction is performed through operations such as convolution, connection, and pooling to obtain three detection branches.
[0162] The neck section sequentially includes a fifth GhostConv module, a second CBAM module, a first multi-connection module, a fifth C3Ghost module, a sixth GhostConv module, a third CBAM module, a second multi-connection module, and a sixth C3Ghost module; the output of the SPPF module serves as the input of the fifth GhostConv module; the outputs of the third GhostConv module, the third C3Ghost module, the fourth GhostConv module, and the second CBAM module serve as the input of the first multi-connection module; the outputs of the first C3Ghost module, the second GhostConv module, the second C3Ghost module, and the third CBAM module serve as the input of the second multi-connection module; the output of the sixth C3Ghost module generates the first detection layer after a convolution operation.
[0163] In this embodiment, as Figure 11 The diagram shown is a structural diagram of the first multi-connection module (Mul-concat) according to an embodiment of the present invention. Figure 12 The diagram shown is a second multi-connection module (Mul-concat) structure according to an embodiment of the present invention. The multi-connection module (Mul-Concat) sequentially includes a convolution module (Conv), an upsampling module (Up-Sample), a downsampling module (Down-Sample), and a connection module (Concat). The first multi-connection module includes a first upsampling module, a first convolution module, a first connection module, a second convolution module, a first downsampling module, and a third convolution module; the second multi-connection module includes a second upsampling module, a fourth convolution module, a second connection module, a fifth convolution module, a sixth convolution module, a second downsampling module, and a seventh convolution module.
[0164] Specifically, the output of the third GhostConv module and the previous layer feature map in the original structure are used as the input of the upsampling module (Up-Sample); the output of the third C3Ghost module is used as the input of the first convolutional module; the output of the third GhostConv module is used as the input of the second convolutional module; the first C3Ghost module and the previous layer feature map in the original structure are used as the input of the upsampling module (Up-Sample); the output of the second GhostConv module is used as the input of the fourth convolutional module; the output of the second C3Ghost module is used as the input of the fifth convolutional module; and the output of the third C3Ghost module is used as the input of the sixth convolutional module.
[0165] The upsampling module is used to upsample the results obtained from the high-level feature layer and output them to the connection module. Upsampling mainly uses nearest-neighbor interpolation to enlarge the high-level feature map to the same size as the low-level features, maximizing the preservation of semantic information of the high-level features, thereby increasing the receptive field of the low-level features during information fusion.
[0166] The downsampling module is used to downsample the results obtained from the lower-level feature layer and output them to the connection module. Downsampling primarily uses mean sampling to preserve as much complete feature information as possible, thereby improving the sensitivity of higher-level features to small targets.
[0167] The connection module is used to connect the outputs of the upsampling module and the downsampling module, as well as the results obtained from the same feature layer. The connection result is then convolved and used as the output of the multi-connection module.
[0168] Specifically, the output data from different feature layers are input into the convolutional module, and the results are classified. The results from higher-level feature layers are upsampled and then input into the connection module (Concat). The results from the same-level feature layers are directly input into the connection module (Concat). The results from lower-level feature layers are downsampled and then input into the connection module (Concat). The output of the connection module (Concat) is then passed through the convolutional module and used as the output data of the multi-connection module (Mul-Concat). The output data of the three C3Ghost modules are convolved and used as the three detection layers of the YOLO-P model to achieve multi-scale object detection.
[0169] The multi-connection module enhances feature fusion and reduces information loss, thereby improving the accuracy of model detection. Furthermore, based on the FPN (Feature Pyramid Networks) structure, the multi-connection module adds one upsampling channel and two downsampling channels, further improving the interaction between the current feature layer and other layers, and accelerating feature information fusion.
[0170] (4) In the head part, the three detection branches are used as inputs, and detection layers of size 20×20×1024, 40×40×512 and 80×80×256 are generated by convolution operation respectively for detection.
[0171] The header portion sequentially includes a seventh GhostConv module, a fourth CBAM module, a first connection module, the seventh C3Ghost module, an eighth GhostConv module, a second connection module, and the eighth C3Ghost module; the output of the sixth C3Ghost module serves as the input of the seventh GhostConv module; the outputs of the sixth GhostConv module and the fourth CBAM module serve as the input of the first connection module; the outputs of the fifth GhostConv module and the eighth GhostConv module serve as the input of the second connection module; the output of the seventh C3Ghost module generates a second detection layer after a convolution operation; the output of the eighth C3Ghost module generates a third detection layer after a convolution operation.
[0172] (5) The first detection layer, the second detection layer and the third detection layer are the outputs of the improved YOLO-P model.
[0173] In this embodiment, the GhostModule is applied to the Backbone and Neck, replacing the original Conv convolutional module and C3 network structure unit module with GhostConv and C3Ghost, which significantly reduces the computational load of the model.
[0174] Specifically, although current mainstream CNN models have introduced smaller convolutional filters (floating-point operations) to build efficient CNNs, 1×1 convolutions still consume a large amount of storage and computational resources. The GhostModule module, however, significantly reduces the computational scale of the model while sacrificing some detection accuracy. Let the input data X be:
[0175] X∈R c×h×w (13)
[0176] Where R is any real number; c is the number of channels; h is the height of the input data; and w is the width of the input data.
[0177] Therefore, the operation of generating N feature maps with a k×k convolution kernel can be represented as:
[0178] Y = X * f + b (14)
[0179] Where Y∈R c×h'×w', is the output feature map with N channels and height and width h' and w' respectively; * is the convolution operation, is the k×k convolution filter of this layer, and k is the number of convolution kernels; X is the input data; f is the convolution kernel; b is the bias term.
[0180] Specifically, therefore, the amount of floating-point calculations required during convolution calculations is c×k×k×h'×w'. Since the number of convolution kernels N and the number of channels c are usually large (generally 256 or 512), the required amount of floating-point calculations is at least hundreds of thousands.
[0181] like Figure 10 The schematic diagram of the GhostModule according to an embodiment of the present invention illustrates that the GhostModule first uses ordinary convolution to generate some intrinsic feature maps, and then uses some inexpensive linear transformations to replace ordinary convolution to generate these redundant feature maps, reducing unnecessary floating-point calculations. For example, suppose m intrinsic feature maps Y'∈R are generated. m×h'×w' ,but:
[0182] Y'=X*f' (15)
[0183] Where Y' is the output intrinsic feature map; f'∈R c×k×k×m Let m be the convolution kernel of this layer, where m ≤ N.
[0184] Specifically, for ease of calculation, the bias term b is omitted here, and the hyperparameters (such as kernel size, padding, and stride) are the same as those in the ordinary convolution in equation (14) to ensure that the output feature map sizes are consistent. To further obtain N feature maps, a series of inexpensive linear operations need to be performed on each intrinsic feature map to obtain s similar feature maps:
[0185]
[0186] Where, Φ i,j For inexpensive linear operations; y p This is the p-th intrinsic feature map of the output intrinsic feature map Y'; y p,q Y' is obtained through a cheap linear operation Φ i,j The generated q-th similar feature map (except the last one); y p,s Let y be the s-th similar feature map generated from the p-th intrinsic feature map through a cheap linear operation. p,s It is an identity mapping of the inherent feature maps, and the resulting identity relation is N = m × s, where m is the total number of inherent feature maps, s is the total number of similar feature maps, and N is the total number of feature maps.
[0187] Therefore, the number of linear operations can be obtained as follows: Let the linear kernel size be d×d. Ideally, the size of the linear kernel can be freely adjusted, but considering more efficient use on the GPU, a linear kernel of the same size is used in GhostModule. Therefore, the theoretical speedup obtained through GhostModule is r. s As shown in equation (17):
[0188]
[0189] Where d·d and k·k are of the same order of magnitude; s << c.
[0190] The YOLO-P model significantly reduces model parameters and computational scale through the ghost module, ensuring real-time detection. At the same time, it improves the accuracy of model detection by utilizing the CBAM module and the multi-connection model.
[0191] Figure 13 A block diagram of the CBAM module of a YOLO-P model network according to an embodiment of the present invention is shown.
[0192] In this embodiment, the CBAM module includes a channel attention module and a spatial attention module. The channel attention module generates a channel attention map using the relationships between channels; the spatial attention module obtains a spatial attention map using spatial relationships.
[0193] Specifically, the input data of the CBAM module and the output data of the channel attention module are multiplied together to obtain the result. The result obtained after passing through the spatial attention module is then multiplied with the original result to obtain the final result, which is used as the output data of the CBAM module.
[0194] Assume the input feature map is X∈R c×h×w Therefore, the output feature map obtained through CBAM is calculated as follows:
[0195]
[0196] Where X' is the output result obtained after passing through the channel attention module; X is the input data; Element-wise multiplication; M c ∈R c×1×1 This is a one-dimensional channel attention diagram; X” represents the final output result; M s ∈R 1×h×w This is a two-dimensional spatial attention diagram.
[0197] The CBAM module, with almost no increase in model size, further extracts important feature information through spatial attention and channel attention, reducing information loss and improving model detection accuracy. The CBAM channel attention module incorporates max pooling, enabling it to acquire some feature information missing from typical attention modules, thus improving the module's attention refinement. The obtained average pooling and max pooling results are fed into a shared network layer (including a multilayer perceptron (MLP) and a hidden layer) to obtain the final channel attention map. CBAM proposes a spatial attention module to compensate for the deficiencies in spatial attention, calculating a two-dimensional spatial attention map, ultimately resulting in a more refined output feature map.
[0198] Suspicious objects are input into the YOLO-P model to further confirm their category, size, and other feature information. An improved YOLO-P model is proposed, addressing the issue of feature information loss during model training. A multi-connection module is introduced to compensate for information loss to some extent by adding feature fusion. Furthermore, the GhostModule and CBAM modules are introduced to improve detection accuracy while maintaining a lightweight model (reducing the algorithm's computational requirements), thus meeting the real-time requirements of video surveillance.
[0199] S104. Perform regression, localization, and classification processing on the detection layer to obtain the detection results of the residue.
[0200] According to embodiments of the present invention, the present invention has the following advantages and effects compared with the prior art:
[0201] (1) This invention divides the background model into two main stages: a break-in period and a stabilization period. During the break-in period, the model is newly established and requires a high learning rate to quickly update the background model and eliminate interference information in the environment. During the stabilization period, in order to reduce interference caused by inappropriate learning rates and improve the model's adaptability to scene target complexity and lighting changes, the influencing factor γ is adjusted according to target complexity and lighting changes to achieve adaptive adjustment of the model's learning rate, thereby improving the model's speed and accuracy.
[0202] (2) Use the state transition information of pixels at different times to construct a sequence pattern that identifies the current state of each pixel, and use this pattern to detect static foreground objects, thereby improving the accuracy of static foreground object detection.
[0203] (3) Using CSPDarknet-53 as the main body, the C3 module is constructed in the network using the Ghost Module, which greatly reduces the size of the feature extraction network; through the multi-connection module in the neck, the expressive power of each feature layer in the network is improved without significantly increasing the computational load, thereby increasing the detection accuracy of the model. At the same time, the attention module CBAM is introduced to improve the overall detection accuracy of the model.
[0204] Example 2
[0205] This method is used to implement the improved dual-background model and YOLO-P-based object detection method described in Example 1. This detection method is applicable to object detection in public scenes. The present invention primarily uses the MS COCO2017 dataset, ADOBA dataset, and PETS2006 dataset to test the improved dual-background model and YOLO-P-based object detection method.
[0206] (1) Dataset
[0207] This invention uses the MS COCO2017 dataset to train and validate the YOLO-P model. The ADOBA dataset is used to validate the overall detection performance of the object detection method. The MS COCO2017 dataset contains 118,287 images across 80 categories. The training, validation, and test sets are divided in an 8:2 ratio, with the training and validation sets further divided in a 9:1 ratio. The ADOBA dataset contains 11 scenes, including indoor, outdoor, and daytime / nighttime scenarios.
[0208] (2) Explanation of the use of default parameters in the experiment:
[0209] This invention uses a YOLO-P model pre-trained on the MS COCO2017 dataset. The model was trained using stochastic gradient descent (SGD) on a GPU for 300 epochs, with a batch size of 16 samples per epoch and an initial learning rate of 0.001.
[0210] (3) Test Results
[0211] The MS COCO2017 dataset uses mean average precision (mAP), floating-point operations per second (FLOPs), and number of parameters as the main reference metrics. Table 1 shows the comparison results of the proposed YOLO-P model with other models on the MS COCO2017 dataset. Compared with the YOLO series models, the YOLO-P model outperforms YOLOv5s by 0.2% and 0.1% in mAP50 and mAP, respectively, and outperforms YOLOv7-Tiny by 1.6% and 0.1%, respectively, at the same scale. Compared with other general object detection models, the YOLO-P model outperforms RefineDet by 2.3% and 4.5% in mAP50 and mAP, respectively, and outperforms SSD by 8.3% and 8.7%, respectively.
[0212] Table 1
[0213]
[0214] Table 2 shows that YOLO-P has 41.6% and 43.1% lower FLOPS and parameter count than YOLOv5s, respectively, and 29.2% and 33.9% lower than YOLOv7-Tiny, respectively. Furthermore, the model size of YOLO-P is significantly smaller than that of YOLOv4. Therefore, the YOLO-P model has significantly lower hardware performance requirements and has broader application prospects in the field of artifact detection.
[0215] Table 2
[0216]
[0217] The ADOBA dataset uses precision (P) and recall (R) as the main reference metrics. Table 3 shows the comparison results of the proposed legacy detection method, which integrates an improved dual-background model and YOLO-P, with other advanced legacy detection methods on the ADOBA dataset. The proposed legacy detection method achieves a recall (R) of 100% on the ADOBA dataset, higher than the NOBS and YOLO-NAS methods, and a precision (P) of 75%, which is 8.4% and 12.4% higher than the NOBS and PFSM With Back-tracing methods, respectively.
[0218] Table 3
[0219]
[0220] The conclusions are as follows: First, an adaptive long-term / short-term background model improves the model's adaptability to complex scenes. Second, a modified PFSM model is used to extract static foreground objects. Finally, a YOLO-P network model is proposed, which reduces the network model size while maintaining detection accuracy. Further judgment using the YOLO-P network model reduces the false alarm rate of the legacy object detection method. The fusion of the adaptive dual-background model and the YOLO-P legacy object detection method can better adapt to complex scene changes and has better application prospects.
[0221] It should be understood that the various forms of processes shown above can be used, with steps reordered, added, or deleted. For example, the steps described in this invention can be executed in parallel, sequentially, or in different orders, as long as the desired result of the technical solution of this invention can be achieved, and this is not limited herein.
[0222] The specific embodiments described above do not constitute a limitation on the scope of protection of this invention. Those skilled in the art should understand that various modifications, combinations, sub-combinations, and substitutions can be made according to design requirements and other factors. Any modifications, equivalent substitutions, and improvements made within the spirit and principles of this invention should be included within the scope of protection of this invention.
Claims
1. A method for detecting residual matter by fusing a dual background model and an improved YOLO-P, characterized in that, include: Obtain the input scene image and construct a dual-background model; Obtain the state information of the dual-background model, input it into the improved PFSM model, and output the suspicious object; The suspicious object is input into the improved YOLO-P model, and the output is the detection layer; The detection layer is subjected to regression, localization, and classification processing to obtain the results of the residue detection; The state information from the dual-background model is input into the improved PFSM model, which outputs suspicious objects, including: In response to the current pixel's state value being 11, the state of the PFSM model changes from the Start state to the dynamic foreground state MF. If the pixel value of the current pixel remains unchanged within a certain period of time, the short-term background model will update the current pixel into the background while the long-term background model will not be updated. At this time, the state value of the current pixel is 10, which triggers the state of the PFSM model to change from the dynamic foreground state MF to the candidate static foreground state CSF, and the first time period timing is started at the same time. If the state value of the current pixel becomes 11 during the first period timing, and the first period is less than the first time threshold , the first period keeps timing, and the second period timing is started at the same time, triggering the state of the PFSM model to be converted from the candidate static foreground state CSF to the occluded candidate static foreground state OCSF; until the first period reaches the first time threshold , and the second period is less than the second time threshold , triggering the state of the PFSM model to be converted from the occluded candidate static foreground state OCSF to the occluded static foreground object OSFO. during the second time period, if the state value of the current pixel becomes 10 and the second time period is less than the second time threshold , indicating that the current pixel is not occluded, triggering the state of the PFSM model to be converted from the occluded candidate static foreground state OCSF to the candidate static foreground state CSF, while terminating the second time period timing and keeping the first time period timing; when the first time period reaches the first time threshold , triggering the state of the PFSM model to be converted from the candidate static foreground state CSF to the static foreground object state SFO; when the second time period reaches a second time threshold and the state value of the current pixel is 11, the state of the PFSM model is directly converted to an occluded static foreground object OSFO. If the state output of the PFSM model is the static foreground object state (SFO) and / or the occluded static foreground object state (OSFO), then the current pixel is considered a suspicious pixel. After traversing all pixels, the set of all suspicious pixels is taken as the suspicious object; The improved YOLO-P model includes a skeletal portion, a neck portion, and a head portion; The backbone component sequentially includes a CBS module, a first GhostConv module, a first C3Ghost module, a second GhostConv module, a second C3Ghost module, a third GhostConv module, a third C3Ghost module, a fourth GhostConv module, a fourth C3Ghost module, a first CBAM module, and an SPPF module, wherein the CBS module serves as the input terminal of the improved YOLO-P model; The neck section sequentially includes a fifth GhostConv module, a second CBAM module, a first multi-connection module, a fifth C3Ghost module, a sixth GhostConv module, a third CBAM module, a second multi-connection module, and a sixth C3Ghost module; the output of the SPPF module serves as the input of the fifth GhostConv module; the outputs of the third GhostConv module, the third C3Ghost module, the fourth GhostConv module, and the second CBAM module serve as the input of the first multi-connection module; the outputs of the first C3Ghost module, the second GhostConv module, the second C3Ghost module, and the third CBAM module serve as the input of the second multi-connection module; the output of the sixth C3Ghost module is converted into a first detection layer after a convolution operation. The header section sequentially includes a seventh GhostConv module, a fourth CBAM module, a first connection module, a seventh C3Ghost module, an eighth GhostConv module, a second connection module, and an eighth C3Ghost module; the output of the sixth C3Ghost module serves as the input of the seventh GhostConv module; the outputs of the sixth GhostConv module and the fourth CBAM module serve as the input of the first connection module; the outputs of the fifth GhostConv module and the eighth GhostConv module serve as the input of the second connection module; the output of the seventh C3Ghost module generates a second detection layer after a convolution operation; the output of the eighth C3Ghost module generates a third detection layer after a convolution operation. The first detection layer, the second detection layer, and the third detection layer serve as the outputs of the improved YOLO-P model.
2. The method of claim 1, wherein, The construction of the dual-background model includes: Construct a Gaussian mixture model; Set an adaptive learning rate; The Gaussian mixture model is updated based on an adaptive learning rate to obtain a dual-background model; wherein the adaptive learning rate is used to adjust the update rate.
3. The method of claim 2, wherein, The construction of the Gaussian mixture model includes: in, The model is Gaussian; K is the total number of pixels; t is time step. For pixels; Let t be the current time. The weights of a Gaussian distribution; For time t A Gaussian distribution density; Let be the pixel value at time t; For time t The expected value of a Gaussian distribution; For time t The covariance matrix of a Gaussian distribution; The dimension of a pixel; is the base of the natural logarithm.
4. The method of claim 2, wherein, The Gaussian mixture model is updated based on an adaptive learning rate to obtain a dual-background model, including: An adaptive learning rate based on an update constant is constructed, wherein the value range of the update constant is set to a first interval or a second interval; the minimum value of the second interval is greater than the maximum value of the first interval. When the value range of the update constant is the first interval, the pixels of the current frame are matched with the Gaussian distribution, and the mean, weight and parameters of the successfully matched Gaussian distribution are updated according to the successfully matched pixels to obtain the long-term background model. When the value of the update constant is in the second interval, the pixels of the current frame are matched with the Gaussian distribution, and the mean, weight and parameters of the successfully matched Gaussian distribution are updated according to the successfully matched pixels to obtain the short-term background model. The long-term background model and the short-term background model constitute a dual background model; Wherein, if the Gaussian distribution satisfies the matching condition, then the pixel point is successfully matched with the Gaussian distribution; the matching condition is: in, for The pixel value at any given time; For a moment The previous moment The expected value of a Gaussian distribution; C is a constant; For the current moment The moment before The variance of a Gaussian distribution.
5. The method according to claim 2, characterized in that, The adaptive learning rate is: in, The learning rate; It is a constant; Impact factor; The current frame number; The frame count threshold; The target pixel is used to determine the proportion of the total pixels in the current frame. This is the illumination transformation factor; This represents the number of pixels identified as the target in the current frame. This represents the total number of pixels in the current frame. The information entropy of the previous frame image; This represents the information entropy of the current frame image.
6. The method according to claim 4, characterized in that, It also includes setting a frame period, performing Gaussian distribution weight detection based on the frame period, and removing Gaussian distributions that meet the weight conditions; The weighting condition is as follows: in, For the current moment No. The weights of a Gaussian distribution; These are the initial weights; for For the current moment No. The variance of a Gaussian distribution; The initial variance; The number of Gaussian distributions in the current background model; This represents the minimum number of Gaussian distributions in the current background model.
7. The method according to claim 4, characterized in that, The process of obtaining the state information of the dual-background model includes: The long-term background model is binarized to obtain a binary image of the long-term background model, which serves as the state information of the long-term background model. Furthermore, the short-term background model is binarized to obtain a binary image of the short-term background model, which serves as the state information of the short-term background model. ; Based on the state information of the long-term background model State information of short-term background models Calculate the state of the current pixel: in, This refers to the current state of the pixel. , When the value is 1, it indicates that the current pixel is a foreground pixel in the long-term background model. When the value is 0, it indicates that the current pixel is a background pixel under the long-term background model; When the value is 1, it indicates that the current pixel is a foreground pixel in the short-term background model. When the value is 0, it indicates that the current pixel is a background pixel under the short-term background model; when =0 and When =0, A value of 00 indicates that in At any given moment, pixels Both the long-term and short-term background models are static background pixels; when =0 and When =1, The value is 01, indicating that at time 1 Below, pixels It is in a state where it is obscured by other objects; when =1 and When =0, The value is 10, indicating that at time 10 Below, pixels Possibly static foreground pixels; when =1 and When =1, The value is 11, indicating that at time 11 Below, pixels Both the long-term and short-term background models are dynamic foreground pixels.
8. The method according to claim 1, characterized in that, The multi-connection module includes an upsampling module, a downsampling module, and a connection module; The upsampling module is used to upsample the results obtained from the high-level feature layer and output them to the connection module. The downsampling module is used to downsample the results obtained from the low-level feature layer and output them to the connection module. The connection module is used to connect the outputs of the upsampling module and the downsampling module, as well as the results obtained from the same feature layer. The connection result is then convolved and used as the output of the multi-connection module.
Citation Information
Patent Citations
Remnant tracing detection and alarm method
CN111062273A
Subway carriage remnant detection method based on two background modeling opportunities
CN117037123A
