Object leaving discrimination method under outdoor monitoring and electronic equipment

Abstract

The invention provides an object leaving discrimination method under outdoor monitoring and electronic equipment, the method is applied to the field of electronic equipment, and the method comprises the steps: initializing a current background image, and obtaining a fast background image and a slow background image; performing weighted fusion on the fast background image and the current background image, and updating the fast background image at a first updating rate; setting an updating flag bit for each pixel in the slow background image based on the dynamic object area and the remaining target area; according to the updating flag bit of each pixel, performing weighted fusion on the current background image and the slow background image, and updating the slow background image by adopting a second updating rate; and determining whether the target object is a legacy object based on the boundary information of the target object in the legacy target area and the image parameters of the target object. According to the method, misjudgment caused by the influence of the environment on the electronic equipment during outdoor monitoring can be avoided, so that the result of judging the left object is more accurate.

Description

Technical Field

[0001] This application relates to the field of electronic devices, and more specifically, to a method and electronic device for identifying objects left behind during outdoor monitoring. Background Technology

[0002] With the popularization and widespread use of network surveillance cameras, the related technology that uses convolutional neural networks to detect each frame of images acquired by surveillance cameras can only obtain the specific target category of objects in each frame of images. However, in actual use, it is impossible to determine the specific real category of the object. In other words, the convolutional neural network is not suitable for objects in the image.

[0003] In related technologies, other traditional methods such as traditional background subtraction or Gaussian background modeling are used to locate the position of pixel changes in each frame of an image. For example, as described in CN 111369529 A, a reference frame background frame is established using background subtraction to locate pixel positions. Further, color, saturation, and brightness (Hue Saturation Value, HSV) and Sobel are used to determine texture information to identify whether an object is a leftover object. However, this method cannot filter out factors such as continuous leaf shaking and changes in lighting, leading to incorrect object location in the image. Furthermore, complex texture information cannot support the determination of whether an object is a leftover object. In other words, if the original background texture is more abundant than the texture of the currently placed object, it is easy to misjudge the object as a leftover object. Summary of the Invention

[0004] This application provides a method and electronic device for identifying objects left behind during outdoor monitoring. This method can avoid erroneous judgments caused by the influence of the environment on electronic devices during outdoor monitoring, making the results of identifying left-behind objects more accurate.

[0005] Firstly, a method for identifying objects left behind during outdoor surveillance is provided, the method comprising: Initialize the current background image to obtain the fast background image and the slow background image; The current background image is weighted and fused with the current background image, and the current background image is globally updated at the first update rate. Based on the dynamic object region output by the object detection model and the residual target region in the object residual alarm chain, an update flag is set for each pixel in the slow background image; the update flag is used to indicate the category to which the pixel belongs. Based on the update flag bit of each pixel, the current background image and the slow background image are weighted and fused, and the slow background image is locally updated using different second update rates; the first update rate is greater than the second update rate. A first binary image is generated by subtracting the updated fast background image from the current background image, and a second binary image is generated by subtracting the updated slow background image from the current background image. The first binary image is an image in which the first difference between the fast background image and the current background image is greater than a first threshold; the second binary image is an image in which the second difference between the slow background image and the current background image is greater than a second threshold. By performing connected component analysis on the first and second binary graphs, the remaining target region is obtained; Based on the boundary information of the target object in the abandoned target area and the image parameters of the target object, it is determined whether the target object is an abandoned object.

[0006] The above scheme initializes the current background image, resulting in a fast background image and a slow background image. The fast background image is then weighted and fused with the current background image, and the fast background image is globally updated at a first update rate. Based on the dynamic object region output by the target detection model and the residual target region in the object residual alarm chain, an update flag is set for each pixel in the slow background image. This update flag indicates the category to which the pixel belongs. According to the update flag of each pixel, the current background image and the slow background image are weighted and fused, and the slow background image is locally updated using different second update rates. The first update rate is greater than the second update rate. The updated fast background image and the current background image are subtracted to generate a first binary image, and the updated slow background image and the current background image are subtracted to generate a second binary image. The first binary image is the one where the first difference between the fast background image and the current background image is greater than a first threshold; the second binary image is the one where the second difference between the slow background image and the current background image is greater than a second threshold. Connectivity analysis is performed on the first and second binary images to obtain the abandoned target region. Based on the boundary information of the target object in the abandoned target region and the image parameters of the target object, it is determined whether the target object is an abandoned object. In other words, updating the flag determines whether the background image is slow, and the boundary information and image parameters of the target object in the abandoned target region are used to jointly determine whether the target object is an abandoned object. This effectively avoids erroneous judgments caused by environmental influences on electronic devices during outdoor monitoring, making the judgment of abandoned objects more accurate and providing a better user experience.

[0007] In conjunction with the first aspect, in some possible implementations, the weighted fusion of the current background image and the slow background image based on the update flag bit of each pixel includes: For pixels whose update flag is set to the first value, the current background image and the slow background image are weighted and summed using the first weighting coefficient; For pixels whose update flag is set to the second value, the current background image and the slow background image are weighted and summed using the second weighting coefficient; For pixels whose update flag is set to the third value, the current background image and the slow background image are weighted and summed using the third weighting coefficient. Wherein, the first value corresponds to the dynamic object region, the second value corresponds to the leftover target region, and the third value corresponds to the static background region. Furthermore, the update rate corresponding to the first weighting coefficient is greater than the update rate corresponding to the second weighting coefficient, and the update rate corresponding to the second weighting coefficient is greater than the update rate corresponding to the third weighting coefficient.

[0008] In conjunction with the first aspect, in some possible implementations, before determining whether a target object is a leftover object based on the boundary information of the target object in the leftover target area and the image parameters of the target object, the process includes: The number of first pixels of the target pixel in the first binary image of each remaining target area is counted, and the number of second pixels of the target pixel in the second binary image of each remaining target area is counted. The target pixel is used to represent the target object. Calculate the first pixel ratio between the number of first pixels and the total number of pixels in each of the remaining target areas, and the second pixel ratio between the number of second pixels and the total number of pixels in each of the remaining target areas; The presence of a target object in the remaining target area is determined based on the proportion of the first pixel and the proportion of the second pixel.

[0009] In conjunction with the first aspect, in some possible implementations, determining whether a target object exists in the remaining target area based on the first pixel proportion and the second pixel proportion includes: If the proportion of the second pixel is greater than the first preset proportion, the object in the remaining target area is tracked and the number of tracking times is counted. If the proportion of the first pixel is greater than the second preset proportion and the number of tracking times is greater than the third threshold, it is determined that there is a target object in the remaining target area; If the number of tracking attempts is less than or equal to the third threshold, it is determined that there is no target object in the remaining target area.

[0010] In conjunction with the first aspect, in some possible implementations, determining whether a target object is a leftover object based on the boundary information of the target object in the leftover target area and the image parameters of the target object includes: Obtain the boundary coordinates of the target object; The boundary gradient of the target object is determined based on these boundary coordinates; Determine whether the boundary gradient is greater than the fourth threshold; If the boundary gradient is greater than the fourth threshold, the number of alarms for the target object is accumulated; and the target object is determined to be a leftover object based on the image parameters of the target object and the number of alarms.

[0011] In conjunction with the first aspect, in some possible implementations, determining whether a target object is a leftover object based on image parameters of the target object and the number of alarms includes: Determine if the number of alarms exceeds the fifth threshold; If the number of alarms is less than or equal to the fifth threshold and the number of times the object in the abandoned target area is tracked is greater than the sixth threshold, the target object is determined to be a non-abandoned object. If the number of alarms exceeds the fifth threshold, the degree of change in the image parameters determines whether the target object is a leftover object.

[0012] In conjunction with the first aspect, in some possible implementations, the image parameters include brightness parameters and color parameters, and determining whether the target object is a abandoned object based on the degree of change of the image parameters includes: If the degree of change of the color parameter is less than the seventh threshold and the degree of change of the brightness parameter is greater than the eighth threshold, the target object is determined to be a non-residual object. If the degree of change of the color parameter is greater than or equal to the seventh threshold and the degree of change of the brightness parameter is greater than the eighth threshold, the target object is determined to be a leftover object.

[0013] In conjunction with the first aspect, in some possible implementations, after determining that the target object is a abandoned object, the method further includes: Update the target object to the object legacy alarm chain list and trigger alarm information for the target object; After determining that the target object is not a leftover object, the following steps are also included: The target object is updated to the object legacy alarm chain list, but no alarm information is triggered for the target object.

[0014] In conjunction with the first aspect, in some possible implementations, the method further includes: During the update process, if the number of pixels differing between the slow background image and the current background image exceeds a predetermined proportion of the total number of pixels in the current background image, the current background image is reinitialized to obtain a new fast background image and a new slow background image; and the target number of updates are performed continuously.

[0015] Secondly, an object residue detection device for outdoor surveillance is provided, the device comprising: The initialization module is used to initialize the current background image and obtain the fast background image and the slow background image. The fusion module is used to perform a weighted fusion of the fast background image and the current background image, and to globally update the fast background image at a first update rate. The setting module is used to set update flag bits for each pixel in the slow background image based on the dynamic object region output by the target detection model and the residual target region in the object residual alarm chain list; the update flag bits are used to indicate the category to which the pixel belongs. The fusion module is further configured to perform weighted fusion of the current background image and the slow background image according to the update flag bit of each pixel, and locally update the slow background image using different second update rates; the first update rate is greater than the second update rate; The difference module is used to perform difference calculations between the updated fast background image and the current background image to generate a first binary image, and to perform difference calculations between the updated slow background image and the current background image to generate a second binary image; wherein, the first binary image is an image in which the first difference value between the fast background image and the current background image is greater than a first threshold; and the second binary image is an image in which the second difference value between the slow background image and the current background image is greater than a second threshold; The analysis module is used to perform connected component analysis on the first binary graph and the second binary graph to obtain the remaining target region; The determination module is used to determine whether a target object is a leftover object based on the boundary information of the target object in the leftover target area and the image parameters of the target object.

[0016] Thirdly, an electronic device is provided, including a memory and a processor. The memory is used to store executable program code, and the processor is used to call and run the executable program code from the memory, causing the electronic device to perform the methods executed by the aforementioned outdoor monitoring object residue detection method.

[0017] Fourthly, a computer program product is provided, comprising: computer program code, which, when run on a computer, causes the computer to execute the method described above for determining object remnants under outdoor monitoring.

[0018] Fifthly, a computer-readable storage medium is provided that stores computer program code, which, when executed on a computer, causes the computer to perform the method described above for determining object remnants under outdoor monitoring. Attached Figure Description

[0019] Figure 1 This is a schematic diagram of the implementation environment of an object residue identification method under outdoor monitoring provided in an embodiment of this application; Figure 2 This is a schematic flowchart illustrating a method for identifying objects left behind under outdoor monitoring, provided in an embodiment of this application. Figure 3 This is a schematic flowchart of another method for identifying objects left behind under outdoor monitoring, provided in an embodiment of this application. Figure 4 This is a schematic flowchart of another method for identifying objects left behind under outdoor monitoring, provided in an embodiment of this application. Figure 5 This is a schematic diagram of the structure of an object residue detection device under outdoor monitoring provided in an embodiment of this application; Figure 6 This is a schematic diagram of the structure of an electronic device provided in an embodiment of this application. Detailed Implementation

[0020] The technical solutions in this application will be clearly and thoroughly described below with reference to the accompanying drawings. In the description of the embodiments of this application, unless otherwise stated, " / " means "or," for example, A / B can mean A or B. "And / or" in the text is merely a description of the relationship between related objects, indicating that three relationships can exist. For example, A and / or B can represent: A existing alone, A and B existing simultaneously, and B existing alone. Furthermore, in the description of the embodiments of this application, "multiple" refers to two or more than two.

[0021] Hereinafter, the terms "first" and "second" are used for descriptive purposes only and should not be construed as implying or suggesting relative importance or implicitly indicating the number of technical features indicated. Thus, a feature defined as "first" or "second" may explicitly or implicitly include one or more of that feature.

[0022] Before introducing the methods of the embodiments of this application, the technical terms that may be involved in the embodiments of this application will be explained first.

[0023] Outdoor surveillance refers to video surveillance camera systems installed in outdoor environments. Outdoor surveillance can monitor public areas and detect intrusions.

[0024] The current background image is the background image determined by the model used to build the background. For example, if an image contains stationary buildings and roads, as well as moving vehicles and pedestrians, then the pixels in the current background image include the stationary buildings and roads.

[0025] A fast background image is a background model that updates rapidly. It quickly incorporates newly appearing stationary objects as part of the background. For example, if an object moves and then stops, the background model will quickly identify the stopped object as part of the new background, thus avoiding continuous false positives of the stopped object as a moving target.

[0026] A slow background image is a background model that updates very slowly. A slow background image only identifies pixels that remain static for extended periods as slow background pixels.

[0027] Object detection models are computer vision models based on deep learning. They can identify object types, output bounding boxes, and define moving objects.

[0028] In the following description of the embodiments of this application, it is used as... Figure 1 Taking an example, the implementation environment of the embodiments of this application will be introduced.

[0029] For example, such as Figure 1 As shown, the implementation environment includes a camera 110 and an electronic device 120.

[0030] Camera 110 is used to acquire video or images of the outdoor environment and send the video or images to electronic device 120 so that electronic device 120 can process the images.

[0031] Electronic device 120 is used to process video or images acquired from camera 110 and determine whether there are any leftover objects in the image.

[0032] Figure 2 This is a schematic flowchart of an object residue identification method under outdoor monitoring provided in an embodiment of this application.

[0033] For example, such as Figure 2 As shown, taking an electronic device as the executing entity as an example, this application provides a method for identifying objects left behind under outdoor monitoring. The method 200 includes the following steps.

[0034] Step 201: Initialize the current background image to obtain a fast background image and a slow background image. The fast background image is used to represent the image in the current background image with an update rate of the first update rate, and the slow background image is used to represent the image in the current background image with an update rate of the second update rate. The first update rate is greater than the second update rate.

[0035] Initialization refers to the preparatory work performed when the electronic device starts up, and the current background image (BgImg) is the image acquired by the electronic device. The fast background image (FastBgImg) is a background model with a faster update speed, and the slow background image is a background model with a slower update speed (SlowBgImg).

[0036] It should be understood that electronic devices initially create two background models based on the current background image: a fast background image model and a slow background image model. The fast background image model updates the fast background image, and the slow background image model updates the slow background image. If no human or moving object is detected in the current background image for a considerable period, the electronic device assumes that the current environment has permanently changed. That is, if no human or moving object is detected in the current background image for an extended period, the electronic device will reinitialize the current background image, obtaining both the fast and slow background images.

[0037] Step 202: Perform a weighted fusion of the fast background image and the current background image, and globally update the fast background image at a first update rate.

[0038] It should be understood that in order for the background model to quickly adapt to changes in the scene, it is necessary to perform a weighted fusion of the fast background image and the current background image to update the fast background image.

[0039] Step 203: Based on the dynamic object region output by the target detection model and the residual target region in the object residual alarm list, set an update flag bit for each pixel in the slow background image; the update flag bit is used to indicate the category to which the pixel belongs; the object residual alarm list is used to store candidate regions; the object residual alarm list includes candidate regions formed after the alarm is triggered, and candidate regions whose update rate is less than the second update rate and cannot confirm whether there is a suspected residual object.

[0040] The object detection model analyzes the current background image to obtain multiple bounding boxes, object categories, and confidence scores. Dynamic object regions are the image regions containing objects whose positions change within the current background image. Update flags determine whether to update the slow-moving background image. The update flags indicate the category of a pixel; for example, if the pixel belongs to the dynamic object category, the update flag will be set to the first value.

[0041] In practical applications, object detection models can identify moving people or vehicles. The object leaving an object alarm chain stores the areas where previous object leaving alarms were triggered, along with candidate areas that are slower to be detected in the background image and cannot be successfully identified. Each pixel in the slow background image represents an object in the slow background image. Traditional fast and slow background image models update all pixels at a certain rate, but this can cause some moving objects to be absorbed into the slow background image, meaning that leftover objects gradually blend into the slow background image. In this case, update flags can be set for each pixel in the slow background image.

[0042] The update flag includes a first value, a second value, and a third value. For example, the first value can be 1, the second value can be 0, and the third value can be 2. An update flag of 1 indicates that the slow background is updated at the fastest rate. An update flag of 0 indicates that the slow background is updated at a slower rate than when the update flag is 1. An update flag of 2 indicates that the slow background is updated at the slowest rate.

[0043] The object-left-behind alarm list is a dynamic list that manages and records the time objects have been left behind. When an electronic device detects an object-left-behind event, it triggers an alarm mechanism. The left-behind target area is the image area occupied by the left-behind object in the current background image.

[0044] Step 204: Based on the update flag bit of each pixel, the current background image and the slow background image are weighted and fused, and the slow background image is locally updated using different second update rates; the first update rate is greater than the second update rate.

[0045] In practical applications, pixels in a slow background image can be updated at a certain rate. If there are changes in the slow background image, the update rate of the slow background image cannot adapt to the changes in the scene in the image, and may mistake the remaining objects for the background. In this case, it is necessary to set an update flag bit according to each pixel in the slow background image, and then determine the update strategy for the slow background image based on the update flag bit.

[0046] The second update rate is less than the first update rate, that is, the update rate for updating the slow background image is less than the update rate for updating the fast background image.

[0047] Step 205: Perform difference calculation between the updated fast background image and the current background image to generate a first binary image, and perform difference calculation between the updated slow background image and the current background image to generate a second binary image; wherein, the first binary image is an image in which the first difference value between the fast background image and the current background image is greater than a first threshold; and the second binary image is an image in which the second difference value between the slow background image and the current background image is greater than a second threshold.

[0048] The difference operation is an image processing operation, specifically a pixel-by-pixel subtraction between the fast and slow background images. Each pixel value in the fast and slow background images represents the degree of difference between the current background image and that location. The greater the difference between the fast and slow background images, the brighter the pixel value in the first binary image; conversely, the greater the difference between the slow and slow background images, the brighter the pixel value in the second binary image.

[0049] Step 206: Perform connected component analysis on the first binary graph and the second binary graph to obtain the residual target area in the object residual alarm chain list.

[0050] In a binary image, a connected component is a region consisting of pixels with the same pixel value and adjacent positions. In image processing techniques, connected components are commonly used to analyze binary images (images with only black and white pixel values). Adjacent pixels can be defined using 4-neighborhoods or 8-neighborhoods. A 4-neighborhood refers to the pixels adjacent to the current pixel in the top, bottom, left, and right directions, while an 8-neighborhood includes pixels adjacent to the current pixel in the top, bottom, left, right, and diagonal directions.

[0051] Step 207: Determine whether the target object is a leftover object based on the boundary information of the target object in the leftover target area and the image parameters of the target object.

[0052] The boundary information includes the boundary gradient and boundary coordinates of the suspected abandoned object in the current background image. The image parameters are converted to YUV (Luma Chrominance) format, along with color, saturation, and luminance (HSV) parameters; where Y is the luminance parameter, UV is the color parameter, V is the luminance parameter, and HS is the color parameter.

[0053] It should be understood that in practical applications, object shadows may cause slow or fast background image updates, but the object shadow is not a legacy object. In this case, whether the target object is a legacy object can be determined jointly based on the boundary information of the target object in the legacy target area and the image parameters of the target object.

[0054] This application provides a method for identifying objects left behind during outdoor surveillance. The method initializes a current background image to obtain a fast background image and a slow background image; it then performs a weighted fusion of the fast background image and the current background image, globally updating the fast background image at a first update rate; based on the dynamic object region output by the target detection model and the left-behind target region in the object left-behind alarm list, it sets an update flag for each pixel in the slow background image; the update flag indicates the category to which the pixel belongs; according to the update flag of each pixel, it performs a weighted fusion of the current background image and the slow background image, locally updating the slow background image at different second update rates; the first update rate is greater than the second update rate. A first binary image is generated by subtracting the updated fast background image from the current background image, and a second binary image is generated by subtracting the updated slow background image from the current background image. The first binary image is defined as an image where the first difference between the fast background image and the current background image is greater than a first threshold; the second binary image is defined as an image where the second difference between the slow background image and the current background image is greater than a second threshold. Connectivity analysis is performed on the first and second binary images to obtain the abandoned target region. Based on the boundary information of the target object in the abandoned target region and the image parameters of the target object, it is determined whether the target object is an abandoned object, and the target object is identified as a suspected abandoned object in the current background image. In other words, updating the flag determines whether the background image is slow, and the boundary information of the target object in the abandoned target region determines whether the target object is an abandoned object. This effectively avoids erroneous judgments caused by environmental influences on electronic devices during outdoor monitoring, making the identification of abandoned objects more accurate and providing a better user experience.

[0055] Figure 3 This is a schematic flowchart of an object residue identification method under outdoor monitoring provided in an embodiment of this application.

[0056] For example, such as Figure 3 As shown, taking an electronic device as the executing entity as an example, this application describes a method for identifying objects left behind under outdoor monitoring. The method 300 includes the following steps.

[0057] Step 301: Initialize the current background image to obtain the fast background image and the slow background image.

[0058] The current background image is a grayscale image. Initializing the current background image means saving it separately into a fast background image model and a slow background image model, resulting in a fast background image and a slow background image. The resolution of the current background image can be 320*192.

[0059] In some embodiments, if no dynamic object is detected in outdoor monitoring within a preset time period, then the current background image has changed, and the current background image is re-initialized to adapt to long-term environmental changes.

[0060] Step 302: Perform a weighted fusion of the fast background image and the current background image, and globally update the fast background image at a first update rate.

[0061] The phrase "update the fast background image globally at the first update rate" indicates that the fast background image updates relatively quickly. For example, in practical applications, if the fast background image is updated at a high frame rate, then objects in the fast background image will appear to move more smoothly and continuously.

[0062] In some embodiments, the fast background image and the current background image are weighted and fused based on a fourth weighting coefficient.

[0063] For example, the fourth weighting coefficient can include a current background image coefficient of 0.9 and a fast background image coefficient of 0.1. That is, the specific formula for weighted fusion of the fast background image and the current background image is: 0.9 * current background image + 0.1 * fast background image. In other words, within the update cycle, 90% of the current background image and 10% of the fast background image are fused.

[0064] Step 303: Based on the dynamic object region output by the target detection model and the residual target region in the object residual alarm chain, set an update flag bit for each pixel in the slow background image; the update flag bit is used to indicate the category to which the pixel belongs.

[0065] The object detection model includes the YOLO model. The dynamic object region output by this object detection model may include people and vehicles in practical applications.

[0066] In some embodiments, it is determined whether the legacy target region intersects with the dynamic object region. If there is an intersection between the legacy target region and the dynamic object region, the update flag of each pixel in the background image is set to a first value.

[0067] Step 304: Based on the update flag bit of each pixel, the current background image and the slow background image are weighted and fused, and the slow background image is locally updated using different second update rates.

[0068] The second update rate is lower than the first update rate. Using different second update rates to locally update the slow background image indicates that the slow background image is updated at a slower rate. For example, in practical applications, if the slow background image is updated at a low frame rate, objects in the slow background image will appear to move choppy. Different second update rates mean that the rates at which the slow background image is updated are not the same.

[0069] In one possible implementation, for a pixel whose update flag is a first value, the current background image and the slow background image are weighted and summed using a first weighting coefficient.

[0070] The first value is automatically determined by the electronic device, and this embodiment does not limit it. For example, the first value can be 1. This first value corresponds to the dynamic object area.

[0071] It should be understood that an update flag of 1 indicates that the slow background is updated the fastest, that is, the current background image and the slow background image are weighted and summed using the first weighting coefficient.

[0072] For example, the first weighting coefficient may include a current background image coefficient of 0.5 and a slow background image coefficient of 0.5. That is, the specific formula for weighted fusion of the slow background image and the current background image is: 0.5 * current background image + 0.5 * slow background image. In other words, 50% of the current background image and 50% of the slow background image are fused.

[0073] In one possible implementation, for pixels whose update flag is set to the second value, the current background image and the slow background image are weighted and summed using a second weighting coefficient.

[0074] The second value is automatically determined by the electronic device, and this embodiment does not limit this value; for example, the second value can be 0. This second value corresponds to the remaining target area. Furthermore, the update rate corresponding to the first weighting coefficient is greater than the update rate corresponding to the second weighting coefficient.

[0075] It should be understood that an update flag of 0 indicates that the update rate of the slow background is slower than the update rate of the update flag of 1. Therefore, the pixels in the slow background image may have changed. Thus, the slow background image is updated at a moderate update rate, which means that the current background image and the slow background image are weighted and summed using the second weighting coefficient.

[0076] For example, the second weighting coefficient can include a current background image coefficient of 0.001 and a slow background image coefficient of 0.999. That is, the specific formula for weighted fusion of the slow background image and the current background image is: 0.001 * current background image + 0.999 * slow background image. In other words, 0.1% of the current background image is fused with 99.9% of the slow background image.

[0077] In one possible implementation, for pixels whose update flag is a third value, the current background image and the slow background image are weighted and summed using a third weighting coefficient.

[0078] The third value is automatically determined by the electronic device, and this embodiment does not limit this value; for example, the third value can be 2. This third value corresponds to a static background area, and the update rate corresponding to the second weighting coefficient is greater than the update rate corresponding to the third weighting coefficient.

[0079] It should be understood that an update flag of 2 indicates that the slow background is updated at the slowest rate. Since there are static background areas in the slow background image, the slow background image is updated at the slowest update rate. That is, the current background image and the slow background image are weighted and summed using the third weighting coefficient.

[0080] In this case, the update rate of the slow background image is determined according to different update flags. That is, by changing the update rate of the slow background image, the slow background image can be made to almost ignore all transient interference, such as interference from leaves or shadows, so that the slow background image can remain stable in the long term and create a reliable background for determining whether there are leftover objects.

[0081] Optionally, after performing the above steps, the following steps may also be performed.

[0082] In one possible implementation, during the update process, if the number of difference pixels between the slow background image and the current background image is detected to exceed a predetermined proportion of the total number of pixels in the current background image, the current background image is reinitialized to obtain a new fast background image and a new slow background image; and the target number of updates are performed continuously.

[0083] The number of pixels differing between the slow background image and the current background image represents the number of pixels whose values ​​differ by more than 10. The predetermined ratio is automatically determined by the electronic device, and this embodiment does not limit this ratio. For example, the predetermined ratio may be that the number of pixels differing between the slow background image and the current background image exceeds half the total number of pixels in the current background image.

[0084] It should be understood that if the number of pixels differing between the slow background image and the current background image exceeds a predetermined proportion of the total number of pixels in the current background image, it indicates that the current slow background image has changed too much, that is, the slow background image may be disturbed. Therefore, the current background image is re-initialized, that is, the current background image is re-initialized to obtain a new fast background image and a new slow background image.

[0085] Since electronic devices are affected by light when acquiring the current background image, the imaging process requires a buffer time. Therefore, the target number of times the current background image is continuously updated is determined.

[0086] In this implementation, the influence of light on the updated background image is reduced by continuously updating the target number of times the current background image is updated.

[0087] Step 305: Perform difference between the updated fast background image and the current background image to generate a first binary image, and perform difference between the updated slow background image and the current background image to generate a second binary image.

[0088] The first binary image is an image in which the first difference between the fast background image and the current background image is greater than a first threshold; the second binary image is an image in which the second difference between the slow background image and the current background image is greater than a second threshold.

[0089] In some embodiments, a ninth threshold is used to determine whether the first binary image and the second binary image are binary images with significant differences.

[0090] If the pixel difference of the first binary image is greater than the ninth threshold, it is determined that the fast background image differs significantly from the current background image; if the pixel difference of the second binary image is greater than the ninth threshold, it is determined that the slow background image differs significantly from the current background image.

[0091] Step 306: Perform connected component analysis on the first binary graph and the second binary graph to obtain the residual target area in the object residual alarm chain list.

[0092] It should be understood that performing 4-neighborhood connectivity analysis on the first binary image yields a residual target region representation by connecting adjacent pixels in the top, bottom, left, and right directions of pixels with significant differences in the first binary image. Similarly, performing 8-neighborhood connectivity analysis on the second binary image yields a residual target region representation by connecting adjacent pixels in the top, bottom, left, and right directions of pixels with significant differences in the second binary image, and then connecting adjacent pixels in the top, top-left, left, bottom-left, bottom, bottom-right, right, and top-right directions.

[0093] In some embodiments, a first reference binary image is obtained by performing an AND operation on the first binary image and the second binary image, a second reference binary image is obtained by performing one erosion and two dilation operations on the first reference binary image, and a connected component analysis is performed on the second reference binary image to obtain the remaining target region.

[0094] In this embodiment, the first binary image and the second binary image are noisy and not connected, so they cannot be used directly. Therefore, after performing morphological processing on the first reference binary image, the noise is removed from the first reference binary image to obtain the second reference binary image. The second reference binary image is then processed to connect the components, thereby organizing the messy foreground pixels into independent and complete residual target regions.

[0095] Step 307: Determine whether a target object exists in the remaining target area.

[0096] The target object is a suspected abandoned object in the current background image.

[0097] It should be understood that there may be dynamic objects in the area where the target is left behind. These dynamic objects may be non-left-behind objects such as leaves or shadows. In this case, it is necessary to determine whether the dynamic object is the target object.

[0098] In one possible implementation, the number of first pixels of the target pixels in the first binary image of each of the remaining target areas is counted, and the number of second pixels of the target pixels in the second binary image of each of the remaining target areas is counted. The first pixel ratio between the number of first pixels and the total number of pixels in each of the remaining target areas is calculated, and the second pixel ratio between the number of second pixels and the total number of pixels in each of the remaining target areas is calculated. Based on the first pixel ratio and the second pixel ratio, it is determined whether there is a target object in the remaining target area.

[0099] The target pixel is used to represent the target object.

[0100] It should be understood that dynamic objects exist within each residual target region. The actual moving targets within the residual target region will differ significantly from both the fast and slow background images. Newly stationary objects within the residual target region may be absorbed into the background by the fast background image, thus these objects may be considered residual objects. Objects that disturb the background within the residual target region are considered background by the slow background image model and foreground by the fast background image model. Therefore, it is necessary to determine whether a target object exists within the residual target region.

[0101] In this implementation, based on the number of first pixels in the first binary image and the number of second pixels in the second binary image, it is possible to determine the differences between the object in the remaining target area and the fast background image and the slow background image, respectively, thereby determining whether the object in the remaining target area is in a changing state.

[0102] To provide a more detailed explanation of the above embodiments, the following description is divided into several parts.

[0103] The first part explains the counting of the number of first pixels of the target pixels in the first binary image of each legacy target region and the counting of the number of second pixels of the target pixels in the second binary image of each legacy target region.

[0104] In this context, the target pixel is a pixel with a value of 255, indicating that the pixel is white. In the first binary image, the target pixel represents a large difference between the fast-moving background image and the current background image. In the second binary image, the target pixel represents a large difference between the slow-moving background image and the current background image.

[0105] The second part explains the calculation of the first pixel ratio between the first pixel count and the total pixel count of each of the remaining target areas, and the second pixel ratio between the second pixel count and the total pixel count of each of the remaining target areas.

[0106] It should be understood that after obtaining the white pixels in the first and second binary images, it is necessary to distinguish whether the white pixel is the target object.

[0107] In some embodiments, the first pixel count is divided by the total pixel count to obtain the first pixel percentage; the second pixel count is divided by the total pixel count to obtain the second pixel percentage.

[0108] The third part explains how to determine whether a target object exists in the remaining target area based on the proportion of the first pixel and the proportion of the second pixel.

[0109] In one possible implementation, if the proportion of the second pixel is greater than the first preset proportion, it is determined that the object in the remaining target area is tracked and the number of tracking times is counted; if the proportion of the first pixel is greater than the second preset proportion and the number of tracking times is greater than the third threshold, it is determined that there is a target object in the remaining target area; if the number of tracking times is less than or equal to the third threshold, it is determined that there is no target object in the remaining target area.

[0110] The first preset percentage is automatically determined by the electronic device, and this embodiment does not limit this. For example, the first preset percentage is 0.3. The second preset percentage is automatically determined by the electronic device, and this embodiment does not limit this. For example, the second preset percentage is 0.5.

[0111] If the proportion of the second pixel is greater than the first preset proportion, it means that a large part of the target area left in the slow background image is identified as the foreground. If the proportion of the first pixel is greater than the first preset proportion, it means that there is a large change in the fast background image. Then the moving object may be a falling leaf or other continuously moving object. In this case, the moving object is unstable. It is necessary to recount the number of times the soil in the target area is tracked and determine whether the number of times is greater than the third threshold.

[0112] If the number of tracking attempts exceeds the third threshold, it indicates that the moving object has stabilized. In this case, it is determined that there is a target object in the abandoned target area, and the channel for determining whether the target object is an abandoned object is opened.

[0113] If the number of tracking attempts is less than or equal to the third threshold, it indicates that the moving object is still unstable, and the object may be a falling leaf or an object that moves briefly and continuously.

[0114] In some embodiments, objects in the remaining target region are tracked based on the Intersection over Union (IOU), and the number of tracking times (StaticTime) is started when the tracking threshold is greater than the tenth threshold.

[0115] The tenth threshold is a value automatically determined by the electronic device, and this application embodiment does not limit it. For example, the tenth threshold is 0.8.

[0116] In this implementation, interference caused by disturbances in the fast background image and unformed residual target areas in the slow background image are filtered out based on the first pixel ratio and the second pixel ratio, greatly reducing false alarms in the alarm system.

[0117] Step 308: Determine whether the target object is a leftover object based on the boundary information of the target object in the leftover target area and the image parameters of the target object.

[0118] The boundary information includes the boundary coordinates of the target object and the boundary gradient.

[0119] It should be understood that when it is determined that there is a target object in the area of ​​the leftover target, the alarm mechanism will trigger an alarm message for the target object. However, the target object may be interference caused by shadows, so the alarm mechanism may falsely alarm for the shadows. Therefore, it is necessary to determine whether the target object is a leftover object based on the boundary information of the target object in order to eliminate the false alarm mechanism.

[0120] In one possible implementation, the boundary coordinates of the target object are obtained; the boundary gradient of the target object is determined based on the boundary coordinates; it is determined whether the boundary gradient is greater than a fourth threshold; if the boundary gradient is greater than the fourth threshold, the number of alarms for the target object is accumulated; and the target object is determined to be a leftover object based on the image parameters of the target object and the number of alarms.

[0121] The target object's boundaries include four boundaries: top, bottom, left, and right.

[0122] It should be understood that the boundary gradient is used to characterize the drastic change in pixel values. The more drastic the change in the boundary of the target object, the larger the gradient value. A boundary gradient value greater than the fourth threshold indicates that the boundary of the target object truly exists and has a clear object boundary, and the alarm count will be accumulated.

[0123] In this implementation, since the physical boundaries of a real object are clear, while the boundaries of light and shadow are unclear in the image, the existence of a real boundary of the target object can be determined based on the boundary gradient, thereby increasing the reliability of the system alarm.

[0124] To provide a more detailed explanation of the above embodiments, the following description is divided into several parts.

[0125] The first part explains how to obtain the boundary coordinates of the target object.

[0126] In some embodiments, the coordinates of the top left corner of the target object, as well as the width and height of the target object, are obtained, and the boundary coordinates of the target object are calculated based on the top left corner coordinates, width, and height.

[0127] The second part explains how to determine the boundary gradient of the target object based on these boundary coordinates.

[0128] In some embodiments, the boundary gradient of the target object is calculated using the Sobel operator.

[0129] The third part explains how to determine whether the boundary gradient is greater than the fourth threshold.

[0130] It should be understood that the boundary gradient of a real object is usually greater than 50, while the boundary gradient of light and shadow may be less than 20. Therefore, it is necessary to determine whether the boundary gradient is greater than the fourth threshold so as to determine whether the target object is a real object based on the boundary gradient.

[0131] The fourth threshold is automatically determined by the electronic device, and this application embodiment does not limit it.

[0132] The fourth part explains how, when the boundary gradient is greater than the fourth threshold, the number of alarms for the target object is accumulated, and whether the target object is a leftover object is determined based on the image parameters of the target object and the number of alarms.

[0133] It should be understood that if the boundary gradient is greater than the fourth threshold, it means that the boundary is a real and clearly defined object edge. In this case, the alarm mechanism needs to trigger an alarm message for the target object, that is, to accumulate the number of alarms for the target object.

[0134] In one possible implementation, it is determined whether the number of alarms is greater than a fifth threshold; if the number of alarms is less than or equal to the fifth threshold and the number of times the object in the abandoned target area is tracked is greater than a sixth threshold, the target object is determined to be a non-abandoned object; if the number of alarms is greater than the fifth threshold, it is determined whether the target object is an abandoned object based on the degree of change of the image parameters.

[0135] The fifth threshold is automatically determined by the electronic device, and this embodiment does not limit its value. For example, the fifth threshold can be 25. The sixth threshold is automatically determined by the electronic device, and this embodiment does not limit its value. For example, the sixth threshold can be 250. Image parameters include color parameters and brightness parameters.

[0136] If the number of alarms is less than or equal to the fifth threshold and the number of times the object in the abandoned target area is tracked is greater than the sixth threshold, it indicates that the target object is a briefly moving object. Therefore, the target object is a retrieved object, which means that the target object is determined to be a non-abandoned object.

[0137] In some embodiments, image parameters are converted to YUV (Luma Chrominance) format and hue saturation and luminance (HSV); where Y is the luminance parameter, UV is the color parameter, V is the luminance parameter, and HS is the color parameter; a first average color parameter and a first average luminance parameter are calculated within the remaining target area; a second average color parameter and a second average luminance parameter are calculated from the previous frame background image of the current background image; the degree of change in the luminance parameter is obtained based on the luminance difference between the first average luminance parameter and the second average luminance parameter; and the degree of change in the color parameter is obtained based on the color difference between the first average color parameter and the second average color parameter.

[0138] In some embodiments, if the degree of change of the color parameter is greater than or equal to a seventh threshold and the degree of change of the brightness parameter is greater than an eighth threshold, the target object is determined to be a leftover object.

[0139] The seventh threshold and the eighth threshold are automatically determined by the electronic device, and this application embodiment does not limit them.

[0140] If the degree of change of the color parameter is greater than or equal to the seventh threshold, it means that the UV and HS parameters have a large degree of change. If the degree of change of the brightness parameter is greater than the eighth threshold, it means that the V or Y parameter has a large degree of change. In this case, the target object is not a shadow.

[0141] For example, if the degree of change of Y in the YUV parameters is greater than or equal to 50, the degree of change of U is greater than or equal to 3, and the degree of change of V is greater than or equal to 3, and in the HSV parameters, the degree of change of H is greater than or equal to 5, the degree of change of S is greater than or equal to 20, and the degree of change of V is greater than or equal to 35, then the target object is determined to be a leftover object.

[0142] In some embodiments, if the degree of change of the color parameter is less than a seventh threshold and the degree of change of the brightness parameter is greater than an eighth threshold, the target object is determined to be a non-residual object.

[0143] If the change in the color parameter is less than the seventh threshold, it means that the changes in the UV and HS parameters are small. If the change in the brightness parameter is greater than the eighth threshold, it means that the V or Y parameter changes significantly. In this case, the target object is a shadow.

[0144] In this implementation, shadows can make the area of ​​the target object appear to have features that resemble the object. If the shadow is not distinguished, it will lead to a large number of false alarms. Therefore, it is necessary to distinguish the shadow from the real object through image parameters to avoid false alarms caused by the shadow.

[0145] Step 309: If the target object is determined to be a leftover object, update the target object to the object leftover alarm chain list and trigger the alarm information for the target object.

[0146] In cases where the target object is determined to be a leftover object, the target object is added to the object leftover alarm chain list so that the target object can be quickly integrated into the current background image.

[0147] Step 310: If it is determined that the target object is not a leftover object, update the target object to the object leftover alarm chain list, and do not trigger alarm information for the target object.

[0148] It should be understood that, since the target object is not a leftover object, in order to avoid false alarms for the target object, no alarm information will be triggered for the target object, and the target object will be updated to the object leftover alarm chain list.

[0149] This application provides a method for identifying objects left behind during outdoor surveillance. The method initializes a current background image to obtain a fast background image and a slow background image; it then performs a weighted fusion of the fast background image and the current background image, globally updating the fast background image at a first update rate; based on the dynamic object region output by the target detection model and the left-behind target region in the object left-behind alarm list, it sets an update flag for each pixel in the slow background image; the update flag indicates the category to which the pixel belongs; according to the update flag of each pixel, it performs a weighted fusion of the current background image and the slow background image, locally updating the slow background image at different second update rates; the first update rate is greater than the second update rate. A first binary image is generated by subtracting the updated fast background image from the current background image, and a second binary image is generated by subtracting the updated slow background image from the current background image. The first binary image is defined as an image where the first difference between the fast background image and the current background image is greater than a first threshold; the second binary image is defined as an image where the second difference between the slow background image and the current background image is greater than a second threshold. Connectivity analysis is performed on the first and second binary images to obtain the abandoned target region. Based on the boundary information of the target object in the abandoned target region and the image parameters of the target object, it is determined whether the target object is an abandoned object, and the target object is identified as a suspected abandoned object in the current background image. In other words, updating the flag determines whether the background image is slow, and the boundary information of the target object in the abandoned target region determines whether the target object is an abandoned object. This effectively avoids erroneous judgments caused by environmental influences on electronic devices during outdoor monitoring, making the identification of abandoned objects more accurate and providing a better user experience.

[0150] Figure 4 This is a schematic flowchart of an object residue identification method under outdoor monitoring provided in an embodiment of this application.

[0151] For example, such as Figure 4 As shown, taking an electronic device as the executing entity as an example, this application describes a method for identifying objects left behind under outdoor monitoring. The method 400 includes the following steps.

[0152] Step 401: Initialize the current background image to obtain the fast background image and the slow background image.

[0153] It is understood that the specific implementation of step 401 can be found in the relevant descriptions of steps 201 and 301 above, and will not be repeated here.

[0154] Step 402: Perform a weighted fusion of the fast background image and the current background image, and globally update the fast background image at a first update rate.

[0155] It is understood that the specific implementation of step 402 can be found in the relevant descriptions of steps 202 and 302 above, and will not be repeated here.

[0156] Step 403: Perform difference between the fast background image and the current background image to generate a first binary image, and perform difference between the slow background image and the current background image to generate a second binary image.

[0157] It is understood that the specific implementation of step 403 can be found in the relevant descriptions of steps 205 and 305 above, and will not be repeated here.

[0158] Step 404: Perform an AND operation on the first binary image and the second binary image to obtain a first reference binary image. Perform one erosion and two dilation operations on the first reference binary image to obtain a second reference binary image. Perform connected component analysis on the second reference binary image to obtain the remaining target region.

[0159] It is understood that the specific implementation of step 404 can be found in the relevant description of step 306 above, and will not be repeated here.

[0160] Step 405: Count the number of first pixels of the target pixels in the first binary image of each of the remaining target regions, and the number of second pixels of the target pixels in the second binary image of each of the remaining target regions. The target pixels are used to represent the target object. Calculate the ratio of the number of first pixels to the total number of pixels in each of the remaining target regions, and the ratio of the number of second pixels to the total number of pixels in each of the remaining target regions.

[0161] It is understood that the specific implementation of step 405 can be found in the relevant description of step 307 above, and will not be repeated here.

[0162] Step 406: Determine whether there is a target object in the remaining target area based on the first pixel ratio and the second pixel ratio.

[0163] It is understood that the specific implementation of step 406 can be found in the relevant description of step 307 above, and will not be repeated here.

[0164] Step 407: If the proportion of the second pixel is greater than the first preset proportion, determine that the object in the remaining target area is tracked and the number of tracking times is counted; if the proportion of the first pixel is greater than the second preset proportion and the number of tracking times is greater than the third threshold, determine that there is a target object in the remaining target area.

[0165] It is understood that the specific implementation of step 407 can be found in the relevant description of step 307 above, and will not be repeated here.

[0166] Step 408: Determine whether the target object is a leftover object based on the boundary information of the target object in the leftover target area and the degree of change of image parameters.

[0167] It is understood that the specific implementation of step 408 can be found in the relevant description of step 308 above, and will not be repeated here.

[0168] Step 409: If the degree of change of the color parameter is greater than or equal to the seventh threshold and the degree of change of the brightness parameter is greater than the eighth threshold, the target object is determined to be a leftover object, the target object is updated to the object leftover alarm chain list, and an alarm message for the target object is triggered.

[0169] It is understood that the specific implementation of step 409 can be found in the descriptions of steps 308 and 309 above, and will not be repeated here.

[0170] Step 410: Based on the dynamic object region output by the target detection model and the residual target region in the object residual alarm chain list, set update flag bits for each pixel in the slow background image.

[0171] It is understood that the specific implementation of step 410 can be found in the relevant descriptions of steps 203 and 303 above, and will not be repeated here.

[0172] Step 411: Based on the update flag bit of each pixel, the current background image and the slow background image are weighted and fused, and the slow background image is locally updated using different second update rates; the first update rate is greater than the second update rate.

[0173] It is understood that the specific implementation of step 411 can be found in the relevant descriptions of steps 204 and 304 above, and will not be repeated here.

[0174] Step 412: During the update process, if the number of difference pixels between the slow background image and the current background image exceeds a predetermined proportion of the total number of pixels in the current background image, the current background image is reinitialized to obtain a new fast background image and a new slow background image.

[0175] It is understood that the specific implementation of step 412 can be found in the relevant description of step 304 above, and will not be repeated here.

[0176] Figure 5 This is a schematic diagram of the structure of an object residue detection device under outdoor monitoring provided in an embodiment of this application.

[0177] For example, the device 500 includes: Initialization module 501 is used to initialize the current background image and obtain a fast background image and a slow background image. The fast background image is used to represent the image in the current background image with an update rate of a first update rate, and the slow background image is used to represent the image in the current background image with an update rate of a second update rate. The first update rate is greater than the second update rate. The fusion module 502 is used to perform weighted fusion of the fast background image and the current background image, and to globally update the fast background image at a first update rate. The setting module 503 is used to set an update flag for each pixel in the slow background image based on the dynamic object region output by the target detection model and the object residue alarm chain. The update flag is used to indicate the category to which the pixel belongs. The object residue alarm chain is used to store candidate regions. The object residue alarm chain includes candidate regions formed after the alarm is triggered, as well as candidate regions whose update rate is less than the second update rate and cannot confirm whether there is a suspected residue object. The fusion module 502 is further configured to perform weighted fusion of the current background image and the slow background image according to the update flag bit of each pixel, and locally update the slow background image using different second update rates; the first update rate is greater than the second update rate; The difference module 504 is used to perform difference calculations between the updated fast background image and the current background image to generate a first binary image, and to perform difference calculations between the updated slow background image and the current background image to generate a second binary image; wherein, the first binary image is an image in which the first difference value between the fast background image and the current background image is greater than a first threshold; and the second binary image is an image in which the second difference value between the slow background image and the current background image is greater than a second threshold; Analysis module 505 is used to perform connected component analysis on the first binary graph and the second binary graph to obtain the remaining target region; The determination module 506 is used to determine whether the target object is a leftover object based on the boundary information of the target object in the leftover target area and the image parameters of the target object, and the target object is a suspected leftover object in the current background image.

[0178] In one possible implementation, the device 500 includes: The fusion module 502 is specifically used to perform a weighted summation of the current background image and the slow background image using a first weighting coefficient for pixels whose update flag is a first value; The fusion module 502 is specifically used to perform a weighted summation of the current background image and the slow background image using a second weighting coefficient for pixels whose update flag is the second value; The fusion module 502 is specifically used to perform a weighted summation of the current background image and the slow background image using a third weighting coefficient for pixels whose update flag is the third value; Wherein, the first value corresponds to the dynamic object region, the second value corresponds to the leftover target region, and the third value corresponds to the static background region. Furthermore, the update rate corresponding to the first weighting coefficient is greater than the update rate corresponding to the second weighting coefficient, and the update rate corresponding to the second weighting coefficient is greater than the update rate corresponding to the third weighting coefficient.

[0179] In one possible implementation, the device 500 includes: The determining module 506 is further configured to count the number of first pixels of the target pixels in the first binary image of each of the remaining target areas, and the number of second pixels of the target pixels in the second binary image of each of the remaining target areas, wherein the target pixels are used to characterize the target object; The determining module 506 is further configured to calculate the first pixel ratio between the first pixel count and the total pixel count of each of the remaining target regions, and the second pixel ratio between the second pixel count and the total pixel count of each of the remaining target regions; The determining module 506 is also used to determine whether there is a target object in the remaining target area based on the first pixel ratio and the second pixel ratio.

[0180] In one possible implementation, the device 500 includes: The determination module 506 is specifically used to determine, when the proportion of the second pixel is greater than the first preset proportion, to track the object in the remaining target area and count the number of tracking times; The determination module 506 is specifically used to determine that a target object exists in the residual target area when the proportion of the first pixel is greater than the second preset proportion and the number of tracking times is greater than the third threshold. The determination module 506 is specifically used to determine that there is no target object in the remaining target area when the number of tracking attempts is less than or equal to the third threshold.

[0181] In one possible implementation, the device 500 includes: The determination module 506 is specifically used to obtain the boundary coordinates of the target object; The determination module 506 is specifically used to determine the boundary gradient of the target object based on the boundary coordinates; The determination module 506 is specifically used to determine whether the boundary gradient is greater than the fourth threshold. The determination module 506 is specifically used to accumulate the number of alarms for the target object when the boundary gradient is greater than the fourth threshold; and to determine whether the target object is a leftover object based on the image parameters of the target object and the number of alarms.

[0182] In one possible implementation, the device 500 includes: The determination module 506 is specifically used to determine whether the number of alarms exceeds the fifth threshold. The determination module 506 is specifically used to determine that the target object is a non-left-behind object when the number of alarms is less than or equal to the fifth threshold and the number of tracking times of the object in the left-behind target area is greater than the sixth threshold. The determination module 506 is specifically used to determine whether the target object is a leftover object based on the degree of change of the image parameters when the number of alarms exceeds the fifth threshold.

[0183] In one possible implementation, the device 500 includes: The determination module 506 is specifically used to determine that the target object is a non-residual object when the degree of change of the color parameter is less than the seventh threshold and the degree of change of the brightness parameter is greater than the eighth threshold. The determination module 506 is specifically used to determine that the target object is a leftover object when the degree of change of the color parameter is greater than or equal to the seventh threshold and the degree of change of the brightness parameter is greater than the eighth threshold.

[0184] In one possible implementation, the device 500 includes: The determination module 506 is also used to update the target object to the object legacy alarm chain list and trigger alarm information for the target object; The determining module 506 is further configured to, after determining that the target object is a non-residual object, also include: The determination module 506 is also used to update the target object to the object legacy alarm chain list without triggering alarm information for the target object.

[0185] In one possible implementation, the device 500 includes: The initialization module 501 is also used to, during the update process, if it is detected that the number of difference pixels between the slow background image and the current background image exceeds a predetermined proportion of the total number of pixels in the current background image, reinitialize the current background image to obtain a new fast background image and a new slow background image; and continuously update the target number of times.

[0186] Figure 6 This is a schematic diagram of the structure of an electronic device provided in an embodiment of this application.

[0187] For example, such as Figure 6 As shown, the electronic device 600 includes a memory 601 and a processor 602. The memory 601 stores executable program code 603, and the processor 602 is used to call and execute the executable program code 603 to perform an object residue detection method under outdoor monitoring.

[0188] Furthermore, embodiments of this application also protect an apparatus that may include a memory and a processor, wherein the memory stores executable program code, and the processor is used to call and execute the executable program code to perform an object residue detection method under outdoor monitoring provided in embodiments of this application.

[0189] This embodiment can divide the device into functional modules based on the above method example. For example, each module can correspond to a separate function, or two or more functions can be integrated into one processing module. The integrated module can be implemented in hardware. It should be noted that the module division in this embodiment is illustrative and only represents one logical functional division. In actual implementation, there may be other division methods.

[0190] It should be understood that the device provided in this embodiment is used to perform the above-described method for identifying objects left behind under outdoor monitoring, and therefore can achieve the same effect as the above-described implementation method.

[0191] When using integrated units, the device may include a processing module and a storage module. When applied to an electronic device, the processing module can be used to control and manage the operation of the electronic device. The storage module can be used to support the execution of relevant program code by the electronic device.

[0192] The processing module may be a processor or a controller, which can implement or execute various exemplary logic blocks, modules, and circuits shown in conjunction with the disclosure of this application. The processor may also be a combination of functions that implement computing capabilities, such as a combination of one or more microprocessors, a combination of digital signal processing (DSP) and microprocessors, etc., and the storage module may be a memory.

[0193] In addition, the device provided in the embodiments of this application may specifically be a chip, component or module. The chip may include a connected processor and a memory. The memory is used to store instructions. When the processor calls and executes the instructions, the chip can execute the object residue detection method provided in the above embodiments.

[0194] This embodiment also provides a computer-readable storage medium storing computer program code. When the computer program code is run on a computer, the computer executes the above-described related method steps to implement the object residue identification method under outdoor monitoring provided in the above embodiment.

[0195] This embodiment also provides a computer program product. When the computer program product is run on a computer, it causes the computer to perform the above-mentioned related steps to realize the object residue identification method under outdoor monitoring provided in the above embodiment.

[0196] In this embodiment, the device, computer-readable storage medium, computer program product, or chip are all used to execute the corresponding methods provided above. Therefore, the beneficial effects they can achieve can be referred to the beneficial effects in the corresponding methods provided above, and will not be repeated here.

[0197] Through the above description of the embodiments, those skilled in the art will understand that, for the sake of convenience and brevity, only the division of the above functional modules is used as an example. In actual applications, the above functions can be assigned to different functional modules as needed, that is, the internal structure of the device can be divided into different functional modules to complete all or part of the functions described above.

[0198] In the embodiments provided in this application, it should be understood that the disclosed apparatus and methods can be implemented in other ways. For example, the apparatus embodiments described above are merely illustrative; for instance, the division of modules or units is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another device, or some features may be ignored or not executed. Furthermore, the coupling or direct coupling or communication connection shown or discussed may be through some interfaces; the indirect coupling or communication connection between devices or units may be electrical, mechanical, or other forms.

[0199] The above description is merely a specific embodiment of this application, but the scope of protection of this application is not limited thereto. Any variations or substitutions that can be easily conceived by those skilled in the art within the scope of the technology disclosed in this application should be included within the scope of protection of this application. Therefore, the scope of protection of this application should be determined by the scope of the claims.

Claims

1. A method for identifying objects left behind under outdoor surveillance, characterized in that, The method includes: Initialize the current background image to obtain a fast background image and a slow background image. The fast background image is used to represent the image in the current background image with an update rate of a first update rate, and the slow background image is used to represent the image in the current background image with an update rate of a second update rate. The first update rate is greater than the second update rate. The fast background image and the current background image are weighted and fused, and the fast background image is globally updated at a first update rate; Based on the dynamic object region output by the target detection model and the object residue alarm chain, update flag bits are set for each pixel in the slow background image; the update flag bits are used to indicate the category to which the pixel belongs; the object residue alarm chain is used to store candidate regions; the object residue alarm chain includes candidate regions formed after the alarm is triggered, and candidate regions whose update rate is less than the second update rate and cannot confirm whether there is a suspected residue object. Based on the update flag bit of each pixel, the current background image and the slow background image are weighted and fused, and the slow background image is locally updated using different second update rates; A first binary image is generated by subtracting the updated fast background image from the current background image, and a second binary image is generated by subtracting the updated slow background image from the current background image. The first binary image is an image where the first difference between the fast background image and the current background image is greater than a first threshold; the second binary image is an image where the second difference between the slow background image and the current background image is greater than a second threshold. Connectivity analysis is performed on the first binary graph and the second binary graph to obtain the remaining target region; Based on the boundary information of the target object in the abandoned target area and the image parameters of the target object, it is determined whether the target object is an abandoned object, and the target object is a suspected abandoned object in the current background image.

2. The method according to claim 1, characterized in that, The step of weightedly fusing the current background image and the slow background image based on the update flag bit of each pixel includes: For pixels whose update flag is set to the first value, the current background image and the slow background image are weighted and summed using the first weighting coefficient; For pixels whose update flag is set to the second value, the current background image and the slow background image are weighted and summed using the second weighting coefficient; For pixels whose update flag is set to the third value, the current background image and the slow background image are weighted and summed using the third weighting coefficient; Wherein, the first value corresponds to the dynamic object region, the second value corresponds to the leftover target region, and the third value corresponds to the static background region. Furthermore, the update rate corresponding to the first weighting coefficient is greater than the update rate corresponding to the second weighting coefficient, and the update rate corresponding to the second weighting coefficient is greater than the update rate corresponding to the third weighting coefficient.

3. The method according to claim 1, characterized in that, Before determining whether the target object is a leftover object based on the boundary information of the target object in the leftover target area and the image parameters of the target object, the process includes: The number of first pixels of target pixels in the first binary image of each of the remaining target regions is counted, and the number of second pixels of target pixels in the second binary image of each of the remaining target regions is counted. The target pixels are used to represent the target object. Calculate the first pixel ratio between the first pixel count and the total pixel count of each of the remaining target regions, and the second pixel ratio between the second pixel count and the total pixel count of each of the remaining target regions; The presence of a target object in the remaining target area is determined based on the first pixel ratio and the second pixel ratio.

4. The method according to claim 3, characterized in that, The step of determining whether a target object exists in the remaining target area based on the first pixel ratio and the second pixel ratio includes: If the proportion of the second pixel is greater than the first preset proportion, it is determined that the object in the remaining target area will be tracked and the number of tracking times will be counted. If the proportion of the first pixel is greater than the second preset proportion and the number of tracking times is greater than the third threshold, it is determined that there is a target object in the residual target area; If the number of tracking attempts is less than or equal to the third threshold, it is determined that there is no target object in the remaining target area.

5. The method according to claim 1, characterized in that, The step of determining whether a target object is a leftover object based on the boundary information of the target object in the leftover target area and the image parameters of the target object includes: Obtain the boundary coordinates of the target object; The boundary gradient of the target object is determined based on the boundary coordinates; Determine whether the boundary gradient is greater than the fourth threshold; If the boundary gradient is greater than the fourth threshold, the number of alarms for the target object is accumulated; and the target object is determined to be a leftover object based on the image parameters of the target object and the number of alarms.

6. The method according to claim 5, characterized in that, The step of determining whether the target object is a leftover object based on the image parameters of the target object and the number of alarms includes: Determine whether the number of alarms exceeds the fifth threshold; If the number of alarms is less than or equal to the fifth threshold and the number of times the object in the abandoned target area is tracked is greater than the sixth threshold, the target object is determined to be a non-abandoned object. If the number of alarms exceeds the fifth threshold, the degree of change in the image parameters determines whether the target object is a leftover object.

7. The method according to claim 6, characterized in that, The image parameters include brightness parameters and color parameters. Determining whether the target object is a leftover object based on the degree of change in the image parameters includes: If the degree of change of the color parameter is less than the seventh threshold and the degree of change of the brightness parameter is greater than the eighth threshold, the target object is determined to be a non-residual object. If the degree of change of the color parameter is greater than or equal to the seventh threshold and the degree of change of the brightness parameter is greater than the eighth threshold, the target object is determined to be a leftover object.

8. The method according to claim 7, characterized in that, After determining that the target object is a abandoned object, the method further includes: Update the target object to the object legacy alarm chain list and trigger alarm information for the target object; After determining that the target object is a non-relict object, the method further includes: The target object is updated to the object legacy alarm chain list, but no alarm information is triggered for the target object.

9. The method according to claim 1, characterized in that, The method further includes: During the update process, if the number of difference pixels between the slow background image and the current background image exceeds a predetermined proportion of the total number of pixels in the current background image, the current background image is reinitialized to obtain a new fast background image and a new slow background image; and the target number of updates are performed continuously.

10. An electronic device, characterized in that, The electronic device includes: Memory, used to store executable program code; A processor for calling and running the executable program code from the memory, causing the electronic device to perform the method as described in any one of claims 1 to 9.

Images