A method and apparatus for detecting a residual

Abstract

This invention provides a method and apparatus for detecting abandoned objects. First, a background image of a detection area and multiple frames of images to be detected within the detection area are acquired. Each frame of the image to be detected and the background image are sequentially input into a foreground detection model for detection, resulting in the position region of each foreground object in each frame of the image to be detected. The position region is the area within the image to be detected where the bounding rectangle of each foreground object is located. For each foreground object, motion state detection is performed based on its position region to obtain the motion state detection result. Finally, for each foreground object, only if the detection result indicates that the foreground object remains stationary for a preset time is the foreground object determined to be an abandoned object. This method improves the detection accuracy of abandoned objects and avoids the influence of factors such as lighting and camera shake on detection accuracy.

Description

Technical Field

[0001] This invention belongs to the field of video surveillance, and more specifically, relates to a method and apparatus for detecting abandoned objects. Background Technology

[0002] In the field of video surveillance, existing technologies for left-behind object detection can be used in various scenarios such as parks, train stations, and airports to facilitate real-time detection of whether items have been left behind or lost in the monitored area, and to promptly alarm and handle any instances of left-behind objects.

[0003] Chinese patent application number 2022100966074.4 discloses a method, device and storage medium for detecting abandoned objects. It discloses that the presence of a foreground image is initially determined by detecting whether the pixels in the image have changed, and then the foreground object in the foreground image is tracked to finally determine whether the foreground object is an abandoned object. However, this processing method is still affected by lighting, camera shake and noise changes, resulting in low detection accuracy and large computational load.

[0004] In view of this, the present invention is proposed. Summary of the Invention

[0005] The technical problem to be solved by the present invention is to overcome the shortcomings of the prior art and provide a method and device for detecting abandoned objects, which can avoid the influence of factors such as light and camera shake on the detection accuracy and improve the detection accuracy of abandoned objects.

[0006] To solve the above-mentioned technical problems, the basic concept of the technical solution adopted by the present invention is as follows:

[0007] In a first aspect, the present invention provides a method for detecting remnants, comprising:

[0008] Acquire the background image of the detection area and multiple frames of the image to be detected in the detection area;

[0009] Each frame of the image to be detected and the background image are sequentially input into the foreground detection model for detection to obtain the location region of each foreground object in each frame of the image to be detected; the location region is the area of ​​the bounding rectangle of each foreground object in the image to be detected.

[0010] For each foreground object, motion state detection is performed on each foreground object based on its position region to obtain the motion state detection result for each foreground object;

[0011] For each foreground object, if the detection result indicates that the foreground object remains stationary for a preset time, then the foreground object is determined to be a remnant.

[0012] Optionally, each frame of the image to be detected and the background image are sequentially input into the foreground detection model for detection to obtain the location region of the foreground image of each frame of the image to be detected, including:

[0013] For each frame of the image to be detected, the frame of the image to be detected is input into the foreground detection model to obtain a feature map of the image to be detected for each first preset size;

[0014] The background image is input into the foreground detection model to obtain a background image feature map for each of the first preset sizes;

[0015] For each first preset size, a distinguishing feature map of the image to be detected at that first preset size is extracted based on the feature map of the image to be detected at that first preset size and the background image features at that first preset size.

[0016] The total discriminative feature map of the frame to be detected is obtained by fusing the discriminative feature maps of each first preset size;

[0017] The detection information of each foreground object in the frame of the image to be detected is obtained by identifying the total distinguishing feature map.

[0018] Optionally, the detection results include the state of each foreground object at multiple time points; the step of performing motion state detection on each foreground object based on its location region to obtain the motion state detection result for each foreground object includes:

[0019] For each foreground object, calculate the overlap rate of the first location region and the second location region of the foreground object. The first location region is the region of the bounding rectangle of the foreground object in the image to be detected at the current time. The second location region is the region of the bounding rectangle of the foreground object in the image to be detected at the next time.

[0020] For each foreground object, if the overlap rate is greater than or equal to a preset threshold, it is determined that the foreground object is in a stationary state at the current moment.

[0021] For each foreground object, if the overlap rate is less than the preset threshold, it is determined that the foreground object is in motion at the current moment.

[0022] The motion state detection result of each foreground object is generated based on the state of each foreground object at multiple consecutive time points.

[0023] Optionally, for each object to be detected in the foreground image, after determining that the foreground object is a relic if the detection result for each foreground object is that the foreground object remains stationary for a preset time, the method further includes:

[0024] An alarm message is generated so that the security system can respond to the alarm message upon receiving it.

[0025] Optionally, the method further includes:

[0026] After determining that the object to be detected is a relic, the background image of the detection area is updated.

[0027] In a second aspect, the present invention provides a device for detecting remnants, comprising:

[0028] The acquisition module is used to acquire the background image of the detection area and multiple frames of the detection area to be detected;

[0029] The first detection module is used to sequentially input each frame of the image to be detected and the background image into the foreground detection model for detection, and obtain the position region of each foreground object in each frame of the image to be detected; the position region is the region of the bounding rectangle of each foreground object in the image to be detected.

[0030] The second detection module is used to perform motion state detection on each foreground object based on the position area of ​​each foreground object to obtain the motion state detection result of each foreground object.

[0031] The determination module is used to determine that for each foreground object, if the detection result shows that the foreground object is stationary for a preset time, then the foreground object is determined to be a remnant.

[0032] Optionally, the first detection module includes:

[0033] The first processing unit is used to input each frame of the image to be detected into the foreground detection model to obtain a feature map of each first preset size of the image to be detected.

[0034] The second processing unit is used to input the background image into the foreground detection model to obtain a background image feature map of each of the first preset sizes;

[0035] The third processing unit is used to extract a distinguishing feature map of the image to be detected for each first preset size based on the feature map of the image to be detected for that first preset size and the background image features for that first preset size.

[0036] The fourth processing unit is used to fuse the difference feature maps of each first preset size to obtain the total difference feature map of the frame to be detected;

[0037] The detection unit is used to identify the total distinguishing feature map to obtain the detection information of each foreground object in the frame to be detected.

[0038] Optionally, the second detection module includes:

[0039] The calculation unit is used to calculate the overlap rate of a first position region and a second position region of each foreground object, wherein the first position region is the region of the bounding rectangle of the foreground object in the image to be detected at the current time; and the second position region is the region of the bounding rectangle of the foreground object in the image to be detected at the next time.

[0040] The first determining unit is used to determine that, for each foreground object, if the overlap rate is greater than or equal to a preset threshold, the foreground object is in a stationary state at the current moment.

[0041] The second determining unit is used to determine that the foreground object is in motion at the current moment if the overlap rate is less than the preset threshold for each foreground object.

[0042] The detection result generation unit is used to generate the detection result of each foreground object based on the state of each foreground object at multiple consecutive time points.

[0043] Optionally, the device further includes:

[0044] An alarm information generation unit is used to generate alarm information so that the security system can respond to the alarm information after receiving it.

[0045] Optionally, the device further includes:

[0046] An update unit is used to update the background image of the detection area after determining that the object to be detected is a relic.

[0047] This invention provides a method and apparatus for detecting abandoned objects. First, a background image of the detection area and multiple frames of images to be detected in the detection area are acquired. Then, each frame of the image to be detected and the background image are sequentially input into a foreground detection model for detection, resulting in the position region of each foreground object in each frame of the image to be detected. The position region is the area of ​​the bounding rectangle of each foreground object within the image to be detected. For each foreground object, motion state detection is performed based on its position region to obtain the motion state detection result for each foreground object. Finally, for each foreground object, only if the detection result indicates that the foreground object remains stationary for a preset time is the foreground object determined to be an abandoned object.

[0048] By adopting the above technical solution, the present invention has the following beneficial effects compared with the prior art.

[0049] 1. Using a foreground detection model can avoid the impact of factors such as lighting and camera shake on detection accuracy, thus improving the accuracy of object detection. It also has the ability to detect foreground objects of any category and classify objects of a specified category. In addition, it also performs well in detecting small objects of any category.

[0050] 2. The trained foreground detection model can also detect small objects. This detection method can be applied in environments with long-distance monitoring, such as when objects are thrown from high altitudes.

[0051] 3. The designed foreground detection model training method does not require a large amount of manually labeled datasets, yet the trained model still exhibits strong generalization ability.

[0052] The specific embodiments of the present invention will now be described in further detail with reference to the accompanying drawings. Attached Figure Description

[0053] The accompanying drawings, as part of this invention, are used to provide a further understanding of the invention. The illustrative embodiments and descriptions of the invention are used to explain the invention, but do not constitute an undue limitation of the invention. Obviously, the drawings described below are merely some embodiments, and those skilled in the art can obtain other drawings based on these drawings without creative effort. In the drawings:

[0054] Figure 1 This is a schematic flowchart of a method for detecting remnants provided in an embodiment of the present invention;

[0055] Figure 2 This is a schematic diagram of the structure of a residue detection device provided in an embodiment of the present invention.

[0056] It should be noted that these accompanying drawings and textual descriptions are not intended to limit the scope of the invention in any way, but rather to illustrate the concept of the invention to those skilled in the art by referring to specific embodiments. Detailed Implementation

[0057] To make the objectives, technical solutions, and advantages of the embodiments of the present invention clearer, the technical solutions in the embodiments will be clearly and completely described below with reference to the accompanying drawings. The following embodiments are used to illustrate the present invention, but are not intended to limit the scope of the present invention.

[0058] In the description of this invention, it should be noted that the terms "upper", "lower", "front", "rear", "left", "right", "vertical", "inner", "outer", etc., indicate the orientation or positional relationship based on the orientation or positional relationship shown in the accompanying drawings. They are only for the convenience of describing this invention and simplifying the description, and do not indicate or imply that the device or element referred to must have a specific orientation, or be constructed and operated in a specific orientation. Therefore, they should not be construed as limiting this invention.

[0059] In the description of this invention, it should be noted that, unless otherwise explicitly specified and limited, the terms "installation," "connection," and "linking" should be interpreted broadly. For example, they can refer to a fixed connection, a detachable connection, or an integral connection; they can refer to a mechanical connection or an electrical connection; they can refer to a direct connection or an indirect connection through an intermediate medium. Those skilled in the art can understand the specific meaning of the above terms in this invention based on the specific circumstances.

[0060] like Figure 1 As shown, this invention provides a method for detecting remnants, comprising:

[0061] S101. Obtain the background image of the detection area and multiple frames of images to be detected in the detection area;

[0062] S102. Each frame of the image to be detected and the background image are sequentially input into the foreground detection model for detection to obtain the position region of each foreground object in each frame of the image to be detected; the position region is the area of ​​the bounding rectangle of each foreground object in the image to be detected.

[0063] S103. For each foreground object, perform motion state detection on each foreground object according to the position area of ​​each foreground object to obtain the motion state detection result of each foreground object.

[0064] S104. For each foreground object, if the detection result is that the foreground object is stationary for a preset time, then the foreground object is determined to be a remnant.

[0065] In step S101 above, the detection area is the area where the abandoned item is to be detected. This area can be a train station hall, an airport, or a certain area within a residential community. This application does not impose any restrictions.

[0066] The background image is an image composed of a fixed background within the detection area. The background can consist of stationary objects such as roads and buildings. The image to be detected is an image of the detection area captured by a camera.

[0067] Specifically, the server acquires the background image of the detection area and processes the video of the detection area captured by the camera to obtain multiple frames of images to be detected.

[0068] For example, the server obtains a background image of the detection area input manually, or the server selects the Nth frame of the image to be detected from multiple frames of images to be detected as the background image of the detection area. In this invention, the 20th frame of the image to be detected is selected as the background image. N can also be other values, which are not limited in this invention.

[0069] In step S102 above, the process of training the foreground detection model also includes extracting the foreground objects contained in the image to be detected and the detection information of each foreground object. Foreground objects refer to objectively existing objects with shapes, such as vehicles, pedestrians, animals, goods, etc.

[0070] The detection information includes the location information of each foreground object in the image to be detected, the category information of each foreground object, and the confidence information of the detection information. The location information is the location region of the bounding rectangle of the foreground object in the image to be detected; the category information is the category to which the foreground object belongs. The foreground object can belong to any of the categories of vehicle, pedestrian, or commodity. Alternatively, when it is not necessary to classify the foreground, these foregrounds are all classified into the same category; the confidence information is the accuracy information of this detection result.

[0071] Specifically, for each frame of the image to be detected, the image to be detected and the background image are input into the foreground detection model to obtain the detection information of each foreground object in the image to be detected.

[0072] For example, taking two frames of images to be detected, namely image A1 and image A2, and background image B, the detection information of all foreground objects contained in image A1 is obtained by inputting image A1 and background image B into the foreground detection model. Similarly, the detection information of all foreground objects contained in image A2 is obtained by inputting image A2 and background image B into the foreground detection model.

[0073] To gain a more detailed understanding, each frame of the image to be detected and the background image are sequentially input into the foreground detection model for detection, obtaining the detection information of each foreground object in each frame of the image to be detected. Step S102 includes:

[0074] S1021, For each frame of the image to be detected, input the frame of the image to be detected into the foreground detection model to obtain the feature map of the image to be detected for each first preset size;

[0075] S1022, The background image is input into the foreground detection model to obtain a background image feature map of each of the first preset sizes;

[0076] S1023, for each first preset size, extract the difference feature map of the image to be detected for that first preset size based on the feature map of the image to be detected for that first preset size and the feature map of the background image for that first preset size.

[0077] S1024, fuse the difference feature maps of each first preset size to obtain the total difference feature map of the frame to be detected;

[0078] S1025, the detection information of each foreground object in the frame of the image to be detected is obtained by recognizing the total difference feature map.

[0079] In step S1021 above, for each frame of the image to be detected, each frame of the image to be detected is input into the foreground detection model to obtain a feature map of each first preset size of the image to be detected.

[0080] Specifically, taking the image to be detected A1 as an example, the image to be detected A1 is input into the foreground detection model to obtain the image to be detected for each first preset size. This invention mainly involves four first preset sizes. If the first preset sizes are size 1, size 2, size 3 and size 4, the image to be detected A11 with the first preset size of size 1, the image to be detected A12 with the first preset size of size 2, the image to be detected A13 with the first preset size of size 3 and the image to be detected A14 with the first preset size of size 4 are obtained respectively.

[0081] In step S1022 above, the background image of the detection area is input into the foreground detection model to obtain background image feature maps of each of the above first preset sizes. Here, each of the first preset sizes is the same as each of the first preset sizes in step S1021.

[0082] Specifically, taking background image B as an example, the background image is input into the foreground detection model to obtain background image feature maps of each first preset size. Here, the first preset size is also one of four first preset sizes, which are the same as the size of each first preset size in step S1021: size 1, size 2, size 3 and size 4 respectively. The background image B is input into the foreground detection model to obtain background image feature map B1 with the first preset size of size 1, background image feature map B2 with the first preset size of size 2, background image feature map B3 with the first preset size of size 3 and background image feature map B4 with the first preset size of size 4 respectively.

[0083] In step S1023 above, for each first preset size, a distinguishing feature map of the image to be detected of the first preset size is extracted based on the feature map of the image to be detected of the first preset size and the feature map of the background image of the first preset size.

[0084] Specifically, taking size 1 as an example, based on the feature map A11 of the image to be detected and the feature map B1 of the background image, the part that is different between the feature map A11 of the image to be detected and the feature map B1 of the background image is extracted as the distinguishing feature map S1 of the image to be detected in size 1 (that is, the distinguishing feature map of the image to be detected is obtained by removing the background image feature part contained in the feature map of the image to be detected); and so on, finally the distinguishing feature map of the image to be detected in size 2, the distinguishing feature map S3 of the image to be detected in size 3 and the distinguishing feature map S4 of the image to be detected in size 4 are obtained.

[0085] In step S1024 above, the difference feature maps of each first preset size are fused to obtain the total difference feature map of the frame to be detected.

[0086] Specifically, taking the image to be detected A1 as an example, the feature maps of the four sizes of the image to be detected are sampled, stitched, and fused to obtain the total distinguishing feature map of the frame to be detected after fusion.

[0087] In step S1025 above, the location region of each foreground object is obtained by detecting each foreground object contained in the total distinguishing feature map of the image to be detected.

[0088] Specifically, taking the total distinguishing feature map of the image to be detected, A1, as an example, the detection information of three foreground objects is obtained by identifying the total distinguishing feature map. For example, the category of foreground object 1 is plant, the location region of the bounding rectangle of foreground object 1 in the image to be detected, and the confidence level of the detection result is 'a'. In this invention, the location region of each foreground object is obtained after filtering by confidence threshold and maximum suppression.

[0089] The foreground detection model is trained through the following steps:

[0090] Step 1: Obtain a background image training sample set and a test image training sample set. The background image training sample set includes at least one frame of background image training sample, and the test image training sample set includes at least one frame of test image training sample. The test image training sample set includes the test image training sample and the position and category information of manually labeled foreground objects in the test image training sample.

[0091] Step 2: Input the background image training samples and the image training samples to be detected in each frame into the foreground detection model to be trained to obtain a prediction result. Input the position information and category information of the manually labeled foreground objects in the image training samples to be detected as the labeling result into the foreground detection model. Train the foreground detection model by comparing the difference between the prediction result and the labeling result.

[0092] In this invention, the backbone network used in the foreground detection model is the YOLO-v5s backbone network. However, the backbone network can also be the feature extraction backbone network of mainstream object detection networks such as SSD and YOLO. In steps 1 and 2 above, this invention also establishes a database that stores background image training samples and image training samples to be detected for training the foreground detection model. Multiple background image training samples together form a background image training sample set, with at least 3000 background image training samples. Similarly, multiple image training samples to be detected together form an image training sample set, with at least 3000 image training samples to be detected. Furthermore, the database also contains a foreground object sample set, which contains multiple foreground object training samples, each containing segmentation information for the foreground object. The process of generating the image training sample set to be detected is as follows: Taking the generation of one image training sample as an example, firstly, any background image and multiple foreground object training samples are selected. Each foreground object in the aforementioned foreground object training samples is scaled to the size of the background image. The foreground objects are sequentially superimposed onto arbitrary positions in the background image. Simultaneously, based on the binary segmentation information of each foreground object, the bounding rectangle of each foreground object is calculated as its target bounding box. The position of each target bounding box is then transformed into the background image coordinate system. Both the background image and the synthesized image to be detected with foreground objects are randomly scaled and cropped with the same parameters. Different parameters are then applied to the brightness and saturation of both the background image and the synthesized image to be detected with foreground objects. Finally, both the background image and the synthesized image to be detected with foreground objects are rotated around their centers by 0–30 degrees. These operations generate one training sample for the background image and one training sample for the image to be detected. Using this method to generate the training sample sets for the background image and the image to be detected can largely cover situations that may occur in real-world scenes. Furthermore, by randomly transforming and adjusting the brightness, the trained foreground detection model can possess strong illumination invariance and rotation invariance.

[0093] When training the foreground detection model, a background image training sample and a detection image training sample are input into the model. The model outputs the recognition result for each foreground object. This result is then compared with the manually labeled position and category information of each foreground object in the detection image training sample. If the difference is large, the model needs to be adjusted. If the difference is small, the model's training accuracy is improving. When the comparison meets the preset test requirements (i.e., the difference is less than a certain threshold), the training of the foreground detection model ends.

[0094] In step S103 above, the detection result is used to indicate whether the state of the object to be detected has changed. The motion detection result can indicate that the object to be detected is in a stationary state or that the object to be detected is in motion.

[0095] Specifically, for each foreground object, the detection result of each foreground object is obtained by detecting each foreground object based on its location area.

[0096] To gain a more detailed understanding, the detection results include the state of each foreground object at multiple times. For each foreground object, motion state detection is performed on each foreground object based on its position area to obtain the motion state detection result of each foreground object. Step S103 also includes:

[0097] S1031, For each foreground object, calculate the overlap rate of the first position region and the second position region of the foreground object, where the first position region is the region of the bounding rectangle of the foreground object in the image to be detected at the current time; and the second position region is the region of the bounding rectangle of the foreground object in the image to be detected at the next time.

[0098] S1032, for each foreground object, if the overlap rate is greater than or equal to a preset threshold, it is determined that the foreground object is in a stationary state at the current moment;

[0099] S1033, for each foreground object, if the overlap rate is less than the preset threshold, it is determined that the foreground object is in motion at the current moment;

[0100] S1034, generate the detection result of each foreground object based on the state of each foreground object at multiple consecutive time points.

[0101] In step S1031 above, for each foreground object, the overlapping area of ​​the first and second position regions of the foreground object is calculated. The overlap rate of the first and second position regions is obtained by the ratio of the size of the overlapping area to the size of the first position region, or by the ratio of the size of the overlapping area to the size of the second position region. Existing detection networks construct large datasets to detect specified categories such as pedestrians, vehicles, and goods. However, existing detection networks cannot detect foreground objects other than those included in the training samples in the dataset. The foreground detection model in this invention can detect not only the foreground objects involved in the training samples but also foreground objects of any category other than those involved in the training samples. In other words, this invention can essentially detect foreground objects of any category. However, due to differences in scenarios and detection requirements, pre-defined categories can be set in the actual use of foreground detection models. For example, some scenarios only require triggering alarms for people or vehicles that are stationary, so pedestrians or vehicles are set as the pre-defined category. In other scenarios, such as road obstacle detection, it is necessary to detect any type of object left behind except for people and vehicles, so people and vehicles are set as the pre-defined filtering category. Even if people or vehicles are stationary on the road, they will not be misjudged as obstacles. Only objects whose categories match the pre-defined categories are detected, while those that do not match the pre-defined categories, such as pedestrians and vehicles, are filtered out. By setting pre-defined categories, interference from some known categories can be filtered out in advance, reducing the workload of data statistics and calculations.

[0102] For example, taking foreground object A as an example, calculate the size C of the overlapping area of ​​the first position region and the second position region of foreground object A, and the size of the first position region is D. The ratio of C to D is used as the overlap rate of the first position region and the second position region of foreground object A.

[0103] In step S1032 above, for each foreground object, if the overlap rate is greater than or equal to a preset threshold, it is determined that the foreground object is in a stationary state at the current moment.

[0104] For example, in this invention, the preset threshold is 0.85. Taking the foreground object A in the foreground image at the current moment as an example, it is determined whether the ratio of C to D is greater than or equal to the preset threshold. If it is greater than or equal to the preset threshold, it means that the position of the foreground object A in the detection area has not changed in the next moment, and thus it is determined that the foreground object A is in a stationary state at the current moment.

[0105] In step S1032 above, if the value is less than a preset threshold, it indicates that the position of foreground object A in the detection area has changed in the next moment or in the current moment, or that foreground object A did not appear in the previous moment but only appears in the detection area in the current moment. All of these possibilities exist, therefore indicating that the foreground object is not stationary in the current moment, i.e., it is in motion.

[0106] In step S1034 above, the detection result of each foreground object is generated based on the state of each foreground object at multiple consecutive time points.

[0107] In step S104 above, for each foreground object, if the detection result is that the foreground object is stationary for a preset time, then the foreground object is determined to be a remnant.

[0108] Specifically, taking foreground object A as an example, if the foreground object remains stationary for a preset time period T, then foreground object A is determined to be a remnant. This invention also provides the calculation time for the preset time period, according to the formula M = T × fps, where T is the remnant time threshold, fps is the frame rate processed by the algorithm, and M is the number of frames in the image to be detected. In this invention, the image is typically detected every 1-3 seconds, therefore the corresponding frame rate range is... The preset time period can be calculated using this formula.

[0109] Furthermore, for each foreground object, if the detection result indicates that the foreground object remains stationary for a preset time, then the foreground object is determined to be a remnant. Following step S104, the method further includes:

[0110] S105, generate alarm information so that the security system can respond to the alarm information after receiving it.

[0111] Specifically, after determining that the object to be detected is a lost item, the server generates an alarm message and transmits the alarm message to the security system. The security system responds to the alarm message to facilitate the notification of staff for timely handling.

[0112] Since this invention involves long-term monitoring of the detection area, specifically detecting the presence of new objects against a fixed background, the background image of the detection area needs to be periodically updated to avoid the influence of factors such as lighting, noise, and camera shake on the detection. Therefore, to ensure detection accuracy without compromising accuracy, this invention also includes:

[0113] S106, After determining that the object to be detected is a relic, update the background image of the detection area.

[0114] Specifically, after an object is detected and an alarm is generated and the security system is notified, updating the background image of the detection area at this stage will not affect the detection of the object's state. Therefore, the background image of the detection area can be updated at this time. Alternatively, if the position of the object in the detection area has changed relative to the previous moment, and it may have disappeared from the detection area, or if the object has just appeared in the detection area at the current moment, no alarm will be generated. Therefore, updating the background image of the detection area at this time will not affect the detection of the object's motion state.

[0115] like Figure 2 As shown, a device for detecting remnants provided by the present invention includes: an acquisition module 201, a first detection module 202, a second detection module 203, and a determination module 204.

[0116] The acquisition module 201 is used to acquire the background image of the detection area and multiple frames of images to be detected in the detection area;

[0117] The first detection module 202 is used to sequentially input each frame of the image to be detected and the background image into the foreground detection model for detection, so as to obtain the position region of each foreground object in each frame of the image to be detected; the position region is the region of the bounding rectangle of each foreground object in the image to be detected.

[0118] The second detection module 203 is used to perform motion state detection on each foreground object according to the position area of ​​each foreground object to obtain the motion state detection result of each foreground object.

[0119] The determination module 204 is used to determine that for each foreground object, if the detection result is that the foreground object is stationary for a preset time, the foreground object is a remnant.

[0120] Optionally, the first detection module includes:

[0121] The first processing unit is used to input each frame of the image to be detected into the foreground detection model to obtain a feature map of each first preset size of the image to be detected.

[0122] The second processing unit is used to input the background image into the foreground detection model to obtain a background image feature map of each of the first preset sizes;

[0123] The third processing unit is used to extract a difference feature map of the image to be detected for each first preset size based on the feature map of the image to be detected for that first preset size and the background image features for that first preset size.

[0124] The fourth processing unit is used to fuse the difference feature maps of each first preset size to obtain the total difference feature map of the frame to be detected;

[0125] The detection unit is used to identify the total distinguishing feature map to obtain the detection information of each foreground object in the frame to be detected.

[0126] Optionally, the second detection module includes:

[0127] The calculation unit is used to calculate the overlap rate of a first position region and a second position region of each foreground object, wherein the first position region is the region of the bounding rectangle of the foreground object in the image to be detected at the current time; and the second position region is the region of the bounding rectangle of the foreground object in the image to be detected at the next time.

[0128] The first determining unit is used to determine that, for each foreground object, if the overlap rate is greater than or equal to a preset threshold, the foreground object is in a stationary state at the current moment.

[0129] The second determining unit is used to determine that the foreground object is in motion at the current moment if the overlap rate is less than the preset threshold for each foreground object.

[0130] The detection result generation unit is used to generate the detection result of each foreground object based on the state of each foreground object at multiple consecutive time points.

[0131] Optionally, the device further includes:

[0132] An alarm information generation unit is used to generate alarm information so that the security system can respond to the alarm information after receiving it.

[0133] Optionally, the device further includes:

[0134] An update unit is used to update the background image of the detection area after determining that the object to be detected is a relic.

[0135] The above description is merely a preferred embodiment of the present invention and is not intended to limit the present invention in any way. Although the present invention has been disclosed above with reference to preferred embodiments, it is not intended to limit the present invention. Any person skilled in the art can make some modifications or alterations to the above-described technical content to create equivalent embodiments without departing from the scope of the present invention. The implementation schemes in the above embodiments can also be further combined or replaced. Any simple modifications, equivalent changes and alterations made to the above embodiments based on the technical essence of the present invention without departing from the scope of the present invention shall still fall within the scope of the present invention.

Claims

1. A method of detecting a residual material, characterized by, The method comprises the following steps: acquiring a background image of a detection area and a plurality of frames of to-be-detected images of the detection area; sequentially inputting each frame of to-be-detected image and the background image into a foreground detection model for detection to obtain a position area of each foreground object of each frame of to-be-detected image; the position area is an area of a circumscribed rectangular frame of each foreground object in the to-be-detected image; for each foreground object, performing motion state detection on each foreground object according to the position area of each foreground object to obtain a motion state detection result of each foreground object; for each foreground object, if the detection result is that the foreground object is in a static state within a preset time, determining that the foreground object is a left object; wherein, a background image of a detection area can be manually input, or the Nth frame of to-be-detected image is selected as the background image of the detection area from the plurality of frames of to-be-detected images; sequentially inputting each frame of to-be-detected image and the background image into a foreground detection model for detection to obtain a position area of each foreground object of each frame of to-be-detected image, comprising: for each frame of to-be-detected image, inputting each frame of to-be-detected image into the foreground detection model to obtain a to-be-detected image feature map of each first preset size; inputting the background image into the foreground detection model to obtain a background image feature map of each first preset size; for each first preset size, extracting a difference feature map of the to-be-detected image of the first preset size according to the to-be-detected image feature map of the first preset size and the background image feature map of the first preset size, the difference feature map being a foreground difference feature after removing the common part of the to-be-detected image feature map and the background image feature map; fusing the difference feature map of each first preset size to obtain a total difference feature map of the frame of to-be-detected image; identifying the total difference feature map to obtain the position area of each foreground object of the frame of to-be-detected image; wherein, the detection result comprises the state of each foreground object at multiple time points; for each foreground object, generating a motion state detection result of each foreground object according to the state of each foreground object at multiple continuous time points, comprising: calculating an overlap rate of a first position area and a second position area of the foreground object, the first position area being an area of a circumscribed rectangular frame of the foreground object in a to-be-detected image at a current time, the second position area being an area of a circumscribed rectangular frame of the foreground object in a to-be-detected image at a next time, the overlap rate being a ratio of an overlapping area of the two areas to an area of the first position area; if the overlap rate is greater than or equal to a preset threshold, determining that the foreground object is in a static state at the current time; if the overlap rate is less than the preset threshold, determining that the foreground object is in a motion state at the current time.

2. The method of claim 1, wherein, for each foreground object, if the detection result is that the foreground object is in a static state within a preset time, determining that the foreground object is a left object, and then further comprising: generating an alarm information to enable a security system to respond to the alarm information after receiving the alarm information.

3. The method of claim 1, wherein the method further comprises: The method further comprises, for each foreground object, if the detection result is that the foreground object is in a static state within a preset time, determining that the foreground object is a left object. updating the background image of the detection area.

4. A device for detecting a residual substance, characterized by comprising: The method further comprises: an acquisition module, configured to acquire a background image of a detection area and a plurality of frames of to-be-detected images of the detection area; a first detection module, configured to sequentially input each frame of to-be-detected image and the background image into a foreground detection model for detection, to obtain a position area of each foreground object of each frame of to-be-detected image; the position area is an area of an external rectangular frame of each foreground object in the to-be-detected image; a second detection module, configured to, for each foreground object, perform motion state detection on each foreground object according to the position area of each foreground object, to obtain a motion state detection result of each foreground object; a determination module, configured to, for each foreground object, if the detection result is that the foreground object is in a static state within a preset time, determine that the foreground object is a left object; the acquisition module, the first detection module, the second detection module, and the determination module are configured to perform the method for detecting a left object according to any one of claims 1 to 3.

5. A device for detecting a deposit according to claim 4, characterized in that The method further comprises: an alarm information generation unit, configured to generate alarm information, so that a security system responds to the alarm information after receiving the alarm information.

6. The device of claim 4, wherein the device is configured to detect the presence of fecal matter. The method further comprises: an updating unit, configured to, after determining that the to-be-detected object is a left object, update the background image of the detection area.

Images