A residual detection method and device, electronic equipment and storage medium

Abstract

Embodiments of the present application provide a left-behind object detection method and device, electronic equipment and storage medium, which are used to improve the accuracy of left-behind object detection and reduce the false detection probability. The method comprises: performing target detection on collected multiple images; obtaining a first image and a second image according to the detection result; wherein the first image is a first frame image in which a target object is detected in the multiple images, the second image is a first frame image in which the target object is not detected in the multiple images, and the shooting time of the first image is earlier than the shooting time of the second image; performing difference on the image pixel matrix of the first image and the second image to obtain a first difference image; and determining a region in which the pixel value is greater than a first preset threshold in the first difference image as a left-behind object region.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of video monitoring, and in particular to a left object detection method and device, electronic equipment and a storage medium. BACKGROUND

[0002] Detecting left objects in a monitoring scene is a key task to ensure public safety. For example, debris accumulated at a fire exit or a safety exit can affect the evacuation of personnel in an emergency. Currently, there are mainly two kinds of left object detection methods. One is left object detection based on background modeling, and the other is specific left object detection based on deep learning.

[0003] Left object detection based on background modeling mainly includes three steps: background modeling, stationary object detection and object classification. In the background modeling stage, the algorithm processes the monitoring video stream by using a Gaussian mixture model and a visual background extraction method to obtain a background image of the monitoring scene. In the stationary object detection stage, the algorithm obtains the position of stationary objects in the monitoring area by using a double background model and a foreground region tracking method. In the object classification stage, the algorithm excludes the interference of stationary people and other non-left object objects by using a feature comparison method to obtain the positions of various types of left objects in the monitoring scene.

[0004] Left object detection based on deep learning is to detect objects in a monitoring scene by using a deep learning-based object detection algorithm, and then determine whether the object is a left object by judging the distance between the object and the object owner.

[0005] However, the left object detection method based on background modeling is very sensitive to light changes and is easily affected by light changes, thereby making false judgments on left objects. For example, when a strong light suddenly appears in the monitoring scene, the model is likely to consider it as a left object, thereby falsely issuing a left object alarm. The left object detection method based on deep learning is less affected by light, but needs to be trained on the objects to be detected in advance. The untrained object types cannot be detected, and only specific types of left objects that have been trained can be detected, which can easily lead to missed detection, thereby reducing the accuracy of left object detection. SUMMARY

[0006] The embodiments of the present application provide a left object detection method, device, electronic equipment and storage medium, which can improve the accuracy of left object detection and reduce the false detection probability.

[0007] In a first aspect, a left object detection processing method is provided, and the method comprises:

[0008] performing target detection on the collected multiple frames of images;

[0009] acquire a first image and a second image according to the detection result; wherein the first image is a first frame image in which the target object is detected in the plurality of frame images, and the second image is a first frame image in which the target object is not detected in the plurality of frame images, and a shooting time of the first image is earlier than a shooting time of the second image;

[0010] perform difference on image pixel matrices of the first image and the second image to obtain a first difference image;

[0011] determine a region in which a pixel value is greater than a first preset threshold in the first difference image as a residual object region.

[0012] Optionally, the determining of the region in which the pixel value is greater than the first preset threshold in the first difference image as the residual object region comprises:

[0013] perform filtering and binarization processing on the first difference image to obtain a first binarization image;

[0014] determine a region in which a pixel value is greater than an average pixel value of the first binarization image in the first binarization image as the residual object region.

[0015] Optionally, the determining of the region in which the pixel value is greater than the first preset threshold in the first difference image as the residual object region further comprises:

[0016] mark a region in which the target object is located in the first difference image;

[0017] determine whether there is another region in the first difference image except the marked region; wherein a pixel value of the another region is greater than the first preset threshold;

[0018] if it is determined that the another region exists, determine that the another region is the residual object region.

[0019] Optionally, the determining of the region in which the pixel value is greater than the first preset threshold in the first difference image as the residual object region further comprises:

[0020] determine whether an object in a region in which the pixel value is greater than the first preset threshold in the first difference image is stationary;

[0021] if it is determined that the object in the region in which the pixel value is greater than the first preset threshold is stationary, determine that the region in which the pixel value is greater than the first preset threshold is the residual object region.

[0022] Optionally, the determining of whether the object in the region in which the pixel value is greater than the first preset threshold in the first difference image is stationary comprises:

[0023] acquiring a third image; wherein the third image is continuous with the second image and is taken later than the second image;

[0024] differencing pixel matrices of the third image and the first image to obtain a second difference image;

[0025] filtering and binarizing the second difference image to obtain a second binarized image;

[0026] determining whether a first region with a pixel value greater than an average pixel value of the second binarized image exists in the second binarized image;

[0027] if the first region exists in the second binarized image, acquiring first region coordinates corresponding to the first region and second region coordinates corresponding to a region with a pixel value greater than a first preset threshold in the first difference image, respectively;

[0028] determining whether a displacement between the first region coordinates and the second region coordinates is less than a second preset threshold;

[0029] if the displacement between the first region coordinates and the second region coordinates is less than the second preset threshold, determining that an object in the region with the pixel value greater than the first preset threshold in the first difference image is static.

[0030] In a second aspect, a device for detecting a left object is provided, and the device comprises:

[0031] a detection module configured to detect a target in a plurality of acquired images;

[0032] a processing module configured to acquire a first image and a second image according to a detection result, wherein the first image is a first frame image in which the target object is detected in the plurality of images, the second image is a first frame image in which the target object is not detected in the plurality of images, and a taking time of the first image is earlier than a taking time of the second image;

[0033] the processing module is further configured to difference image pixel matrices of the first image and the second image to obtain a first difference image;

[0034] the processing module is further configured to determine a region with a pixel value greater than a first preset threshold in the first difference image as a left object region.

[0035] Optionally, the processing module is specifically configured to:

[0036] filter and binarize the first difference image to obtain a first binarized image;

[0037] determine a region in the first binarization image in which a pixel value is greater than an average pixel value of the first binarization image as the residual region.

[0038] Optionally, the processing module is specifically configured to:

[0039] mark a region in the first difference image in which the target object is located;

[0040] determine whether there is another region in the first difference image other than the marked region; wherein a pixel value of the another region is greater than the first preset threshold value;

[0041] if it is determined that the another region exists, determine that the another region is the residual region.

[0042] Optionally, the processing module is specifically configured to:

[0043] determine whether an object in a region in the first difference image in which a pixel value is greater than a first preset threshold value is stationary;

[0044] if it is determined that the object in the region in which the pixel value is greater than the first preset threshold value is stationary, determine that the region in which the pixel value is greater than the first preset threshold value is the residual region.

[0045] Optionally, the processing module is specifically configured to:

[0046] obtain a third image; wherein the third image is continuous with the second image and is taken later than the second image;

[0047] obtain a second difference image by performing a difference operation on pixel matrices of the third image and the first image;

[0048] perform filtering and binarization processing on the second difference image to obtain a second binarization image;

[0049] determine whether there is a first region in the second binarization image in which a pixel value is greater than an average pixel value of the second binarization image;

[0050] if the first region exists in the second binarization image, obtain first region coordinates corresponding to the first region and second region coordinates corresponding to a region in the first difference image in which a pixel value is greater than a first preset threshold value, respectively;

[0051] determine whether a displacement between the first region coordinates and the second region coordinates is less than a second preset threshold value;

[0052] if the displacement between the first region coordinates and the second region coordinates is less than the second preset threshold value, determine that an object in a region in the first difference image in which a pixel value is greater than a first preset threshold value is stationary.

[0053] In a third aspect, an electronic device is provided, and the electronic device comprises:

[0054] a memory configured to store program instructions;

[0055] a processor configured to invoke the program instructions stored in the memory and perform the steps of the method of any one of the first aspect according to the obtained program instructions.

[0056] In a fourth aspect, a computer readable storage medium is provided, and the computer readable storage medium stores computer executable instructions for causing a computer to perform the steps of the method of any one of the first aspect.

[0057] In a fifth aspect, a computer program product containing instructions which, when the computer program product is executed on a computer, cause the computer to carry out the legacy object detection method described in the various possible implementation manners.

[0058] In the embodiments of the present application, target detection is performed on the images collected by the camera, when a target object is detected, the first frame image in which the target object appears is obtained as the first image, and the continuous collection of multiple frames of images continues to be detected, when the target object leaves the shooting range of the camera (i.e. the target object does not exist in the collected images), the first frame image in which the target object leaves the camera range is obtained as the second image, the image pixel matrix of the first image and the second image is subtracted to obtain the first difference image, and the area in which the image pixel value in the first difference image is greater than the first preset threshold value is determined as the legacy object area. That is, the present application separates the background and the legacy object in the camera monitoring scene by image processing on the images when the target object appears and disappears, and realizes the detection of the legacy object generated after the target object passes, so that the influence of the light change can be effectively avoided, and the detection of various object types and object shapes can be realized, the accuracy of the legacy object detection is effectively improved, and the false detection probability is reduced. BRIEF DESCRIPTION OF DRAWINGS

[0059] In order to more clearly illustrate the technical solutions of the embodiments of the present application or the prior art, the drawings needed in the embodiment description will be briefly introduced. Obviously, the drawings in the following description are only some embodiments of the present application.

[0060] Figure 1 A flowchart of a legacy object detection method provided by the embodiments of the present application;

[0061] Figure 2 A structural block diagram of a legacy object detection device provided by the embodiments of the present application;

[0062] Figure 3 Fig. 1 is a structural schematic diagram of a computer device in an embodiment of the present application. DETAILED DESCRIPTION

[0063] In order to make the objects, technical solutions and advantages of the present application clearer, the technical solutions in the embodiments of the present application will be described clearly and completely below with reference to the drawings in the embodiments of the present application. Obviously, the described embodiments are only a part of the embodiments of the present application, but not all the embodiments. Based on the embodiments in the present application, all other embodiments obtained by those skilled in the art without creative work fall within the scope of protection of the present application. The embodiments in the present application and the features in the embodiments can be combined with each other arbitrarily without conflict. Moreover, although the logical order is shown in the flowchart, in some cases, the steps shown or described can be performed in an order different from that here.

[0064] The terms "first" and "second" in the specification and claims of the present application and the above drawings are used to distinguish different objects, but not to describe a specific order. In addition, the term "comprising" and any variations thereof are intended to cover the non-exclusive protection. For example, a process, method, system, product or device including a series of steps or units is not limited to the listed steps or units, but can optionally include steps or units not listed, or can optionally include other steps or units inherent to the process, method, product or device. "Multiple" in the present application can mean at least two, for example, can be two, three or more, and the embodiments of the present application are not limited.

[0065] In addition, the term "and / or" herein is only a description of the association relationship between the associated objects, which means that there can be three relationships, for example, A and / or B can mean that there are three cases of A alone, A and B together, and B alone. In addition, the character " / " herein generally represents an "or" relationship between the associated objects without special description.

[0066] The application scenarios to which the technical solutions of the embodiments of the present application can be applied will be briefly introduced below. It should be noted that the application scenarios introduced below are only used to illustrate the embodiments of the present application, but not limited. In the specific implementation, the technical solutions provided by the embodiments of the present application can be flexibly applied according to actual needs.

[0067] In the embodiments of the present application, the provided residual detection method and residual detection device can be applied to a video monitoring system, which includes an image acquisition system and a data analysis system. The residual detection method provided by the present application can be executed by the image acquisition system in the video monitoring system, or the image acquisition system sends images to the data analysis system, which executes the residual detection method. The image acquisition system provides a shooting device, such as a camera, a monitoring device, or other devices that can shoot images, for acquiring images of a monitoring area.

[0068] The residual detection method provided by the embodiments of the present application will be described below in conjunction with the accompanying drawings of the specification. Referring to FIG. 1, Figure 1 The flow of the residual detection method in the embodiments of the present application is described as follows:

[0069] Step 101: performing target detection on the acquired multiple frames of images;

[0070] The target detection is performed on the images acquired by the camera by a target detection model to determine whether a target object needs to be detected in the images. The target object can be a person, an animal, or a vehicle, or other objects that can move and have been trained in advance. In the embodiments of the present application, the target detection on the acquired multiple frames of images can be performed on multiple frames of images shot by the camera to determine whether a target object exists, or the real-time detection can be performed on the images shot by the camera to determine whether a target object exists in the currently acquired images. The manner of target detection is not limited in the embodiments of the present application.

[0071] Step 102: acquiring a first image and a second image according to the detection result;

[0072] As described above, the target detection on the acquired multiple frames of images can include two ways, and therefore, the way of acquiring a first image and a second image according to the detection result includes the following two ways:

[0073] The first way corresponds to the above-mentioned target detection on the multiple frames of images shot by the camera. At this time, a first frame of image in which a target object is detected is acquired from the multiple frames of images as the first image, and a first frame of image in which a target object is not detected is acquired from the multiple frames of images as the second image, wherein the shooting time of the first image is earlier than that of the second image, and the multiple frames of images are multiple frames of images continuously shot by the camera.

[0074] The second way corresponds to the above-mentioned real-time detection on the images shot by the camera. At this time, when a target object appears, the image currently shot by the camera is acquired as the first image, and when the target object leaves the shooting range of the camera, the image currently shot by the camera is acquired as the second image.

[0075] Step 103: difference the image pixel matrix of the first image and the second image to obtain a first difference image;

[0076] In the embodiment of the present application, the pixel value of the corresponding pixel points of the first image and the second image is differentiated to obtain a changed pixel value (i.e. a pixel difference value), and the first difference image is formed by the pixel difference value.

[0077] Step 104: determine the region with a pixel value greater than a first preset threshold in the first difference image as a residual object region.

[0078] After the image pixel matrix of the first image and the second image is differentiated, the changed place in the obtained first difference image will form a pixel difference value, i.e. the region where the residual object is located, the region where the light changes, and the region where other minor changes occur will form a pixel difference value, and the pixel difference value in the region where the residual object is located is larger, and the pixel difference value formed by the light change and other minor changes is smaller. At this time, in order to avoid the difference caused by the light change and other minor changes, the region in the first difference image can be threshold filtered, so that the residual object region can be obtained. The way of determining the residual object region includes but is not limited to the following three ways:

[0079] A, the first difference image can be filtered and denoised to eliminate discrete noise, and then the image after eliminating noise is binarized to obtain a gray image (a first binarized image) of the first difference image, and the region with a pixel value greater than the average pixel value of the gray image is determined as the residual object region.

[0080] B, after obtaining the first difference image, the region where the target object is located can be marked, and it is judged whether there is another region with a pixel value greater than the first preset threshold in the first difference image except the marked region, if there is, the other region is determined as the residual object region.

[0081] In the specific implementation process, by marking the region where the target object is located, it can be effectively avoided that the finally determined residual object region is actually the region where the target object is located, thereby reducing the false detection probability.

[0082] C, when it is determined that the first difference image includes a region with a pixel value greater than the first preset threshold, it can also be judged whether the object in the region is stationary, if the object in the region is stationary, the region is determined as the residual object region.

[0083] Specifically, a third image later than the second image in shooting time is acquired, wherein the third image and the second image can be consecutive images or images at a certain time interval, then the pixel matrix of the third image and the first image is subtracted to obtain a second difference image, the second difference image is filtered and binarized to obtain a second binarized image, wherein the filtering and binarization of the second difference image are the same as in the manner A, and it is determined whether the second binarized image has a first region with a pixel value greater than the average pixel value of the second binarized image, if yes, the first region coordinates corresponding to the first region and the second region coordinates corresponding to the region in the first difference image with a pixel value greater than the first preset threshold are acquired respectively, then it is judged whether the displacement between the first region coordinates and the second region coordinates is less than the second preset threshold, if yes, it is determined that the object in the region in the first difference image with a pixel value greater than the first preset threshold is stationary.

[0084] In the specific implementation process, by determining whether the object in the region with a pixel value higher than the first preset threshold is stationary, the moving object can be effectively avoided to be determined as the left object, thereby improving the accuracy of left object detection.

[0085] For example, a pedestrian pulls an empty baby carriage, and there is a certain distance between the baby carriage and the pedestrian, at this time, the region where the baby carriage is located exists in the left object region determined according to the second image and the first image, but the baby carriage is a moving object, therefore, the baby carriage needs to be avoided to be determined as the left object, at this time, the region where the baby carriage is located can be excluded by acquiring the third image.

[0086] The above manners A, B or C can be used alone or in combination. When the above three manners are used in combination, the specific implementation process can include:

[0087] Manner 1, after the image pixel matrix of the first image and the second image is subtracted, the first difference image is filtered and binarized to obtain a first binarized image, then the region where the target object is located in the first binarized image is marked, it is determined whether there is another region in the first binarized image with a pixel value greater than the average pixel value of the first binarized image except the marked region, if yes, it is judged whether the object in the other region is stationary, if yes, the other region is determined as the left object region.

[0088] In a manner 2, after the image pixel matrix of the first image and the second image is subtracted, a region where the target object is located in the first difference image is marked (for example, the position where the target object is located in the first difference image is blackened), and then the marked difference image is filtered and binarized to obtain a first binarized image, and it is determined whether there is another region whose pixel value is greater than the average pixel value of the first binarized image, if there is, it is determined whether the object in the other region is stationary, if it is stationary, the other region is determined as the left object region.

[0089] In the specific implementation process, by marking the region where the target object is located, it can be effectively avoided that the finally determined left object region is actually the region where the target object is located, thereby reducing the false detection probability, and by determining whether the object in the region whose pixel value is higher than the first preset threshold is stationary, it can be effectively avoided that the moving object is determined as the left object, thereby further improving the accuracy of left object detection.

[0090] In a possible implementation, when it is determined that there is a left object in the camera shooting range, an automatic alarm system is triggered to alarm, so that the staff can clean the left object in time to avoid affecting the evacuation of personnel in an emergency.

[0091] In some other embodiments, a fourth image between the first image and the second image can also be acquired, the image pixel matrix of the fourth image and the second image is subtracted to obtain a third difference image, the third difference image is filtered and binarized to obtain a third binarized image, then the region where the target object is located in the third binarized image is marked, it is determined whether there is a second region in the third binarized image, wherein the image pixel value of the second region is higher than the average pixel value of the third binarized image, and the second region is different from the left object region determined in the foregoing, if there is, it is determined that the region where the target object is located in the first image is also a left object region. In this way, it can be effectively avoided that the position of the left object is exactly the position where the target object sits in the first image, thereby further improving the accuracy of left object detection.

[0092] In the specific implementation process, the left object detection method provided by the embodiments of the present application can detect various types of left object bodies left by the target object passing through the video monitoring scene, without the need for prior setting and training of the left object.

[0093] Based on the same inventive concept, the embodiments of the present application provide a left object detection device which can realize the functions corresponding to the foregoing left object detection method. The left object detection device can be a hardware structure, a software module, or a hardware structure plus a software module. The left object detection device can be realized by a chip system, which can be composed of a chip or can contain a chip and other discrete devices. Please refer toFigure 2 As shown, the residual detection device includes a detection module 201 and a processing module 202. Wherein:

[0094] The detection module 201 is configured to perform target detection on the collected multiple frames of images.

[0095] The processing module 202 is configured to obtain a first image and a second image according to the detection result; wherein the first image is a first frame of image in which the target object is detected in the multiple frames of images, and the second image is a first frame of image in which the target object is not detected in the multiple frames of images, and the shooting time of the first image is earlier than the shooting time of the second image.

[0096] The processing module 202 is further configured to perform difference operation on the image pixel matrix of the first image and the second image to obtain a first difference image.

[0097] The processing module 202 is further configured to determine a region in which the pixel value is greater than a first preset threshold in the first difference image as a residual region.

[0098] In a possible implementation, the processing module 202 is specifically configured to:

[0099] perform filtering and binarization processing on the first difference image to obtain a first binarization image;

[0100] determine a region in which the pixel value is greater than the average pixel value of the first binarization image in the first binarization image as the residual region.

[0101] In a possible implementation, the processing module 202 is specifically configured to:

[0102] mark a region in which the target object is located in the first difference image;

[0103] determine whether there is another region in the first difference image except the marked region; wherein the pixel value of the other region is greater than the first preset threshold;

[0104] if it is determined that the other region exists, the other region is determined as the residual region.

[0105] In a possible implementation, the processing module 202 is specifically configured to:

[0106] determine whether the object in a region in which the pixel value is greater than the first preset threshold in the first difference image is stationary;

[0107] if it is determined that the object in the region in which the pixel value is greater than the first preset threshold is stationary, the region in which the pixel value is greater than the first preset threshold is determined as the residual region.

[0108] In one possible implementation, the processing module 202 is specifically used for:

[0109] Acquire a third image; wherein the third image is consecutive to the second image and was captured later than the second image;

[0110] The second difference image is obtained by subtracting the pixel matrix of the third image from that of the first image;

[0111] The second difference image is filtered and binarized to obtain a second binarized image;

[0112] Determine whether there exists a first region in the second binarized image whose pixel value is greater than the average pixel value of the second binarized image;

[0113] If the first region exists in the second binarized image, then the coordinates of the first region corresponding to the first region and the coordinates of the second region corresponding to the region in the first difference image whose pixel value is greater than the first preset threshold are obtained respectively.

[0114] Determine whether the displacement between the coordinates of the first region and the coordinates of the second region is less than a second preset threshold.

[0115] If the displacement between the coordinates of the first region and the coordinates of the second region is less than the second preset threshold, then it is determined that the object in the region where the pixel value in the first difference image is greater than the first preset threshold is stationary.

[0116] All relevant content of each step involved in the aforementioned embodiments of the residue detection method can be referenced to the functional description of the corresponding functional module of the residue detection device in the embodiments of this application, and will not be repeated here.

[0117] The module division in this embodiment is illustrative and represents only one logical functional division. In actual implementation, other division methods may be used. Furthermore, the functional modules in each embodiment of this application can be integrated into a single processor, exist as separate physical entities, or be integrated into a single module. The integrated modules described above can be implemented in hardware or as software functional modules.

[0118] Based on the same inventive concept, embodiments of this application provide an electronic device. Please refer to... Figure 3 As shown, the electronic device includes at least one processor 301 and a memory 302 connected to the at least one processor. In this embodiment, the specific connection medium between the processor 301 and the memory 302 is not limited. Figure 3 Taking the connection between processor 301 and memory 302 via bus 300 as an example, bus 300 in...Figure 3 The connections between other components shown in the figure are shown as lines that represent some of the possible connections between the devices. Many actual systems can include numerous devices interconnected with each other in actual implementations of the described functionality. Figure 3 Only one of each type of bus is shown for each bus bus, but any number of buses can be present in the system.

[0119] In the embodiments of the present application, the memory 302 stores instructions executable by the at least one processor 301, and the at least one processor 301 can execute the steps included in the foregoing residual detection method by executing the instructions stored in the memory 302.

[0120] The processor 301 is the control center of the electronic device, and can connect various parts of the electronic device through various interfaces and lines. The processor 301 can monitor the entire electronic device by running or executing instructions stored in the memory 302 and calling data stored in the memory 302, and process data, thereby monitoring the entire electronic device. Optionally, the processor 301 can include one or more processing units. The processor 301 can integrate an application processor and a modem processor. The application processor mainly processes operating systems and application programs, and the modem processor mainly processes wireless communication. It can be understood that the modem processor can also not be integrated into the processor 301. In some embodiments, the processor 301 and the memory 302 can be implemented on the same chip, and in some embodiments, they can also be implemented on separate chips.

[0121] The processor 301 can be a general-purpose processor, such as a central processing unit (CPU), a digital signal processor, an application-specific integrated circuit, a field programmable gate array, or other programmable logic device, a discrete gate or transistor logic device, a discrete hardware component, and can implement or execute the methods, steps, and logic block diagrams disclosed in the embodiments of the present application. The general-purpose processor can be a microprocessor or any conventional processor. The steps of the residual detection method disclosed in conjunction with the embodiments of the present application can be directly embodied as execution completed by a hardware processor, or executed by a combination of hardware and software modules in the processor.

[0122] The memory 302, as a non-volatile computer readable storage medium, can be used to store non-volatile software programs, non-volatile computer executable programs and modules. The memory 302 can include at least one type of storage medium, for example, can include flash memory, hard disk, multimedia card, card type memory, random access memory (RAM), static random access memory (SRAM), programmable read-only memory (PROM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), magnetic storage, magnetic disk, optical disk, etc. The memory 302 is any other medium capable of carrying or storing desired program code in the form of instructions or data structures and capable of being accessed by a computer, but is not limited thereto. The memory 302 in the embodiments of the present application can also be a circuit or any other device capable of realizing a storage function, used to store program instructions and / or data.

[0123] By designing and programming the processor 301, the code corresponding to the abandoned object detection method introduced in the foregoing embodiments can be fixed in the chip, so that the chip can execute the steps of the foregoing abandoned object detection method when running. How to design and program the processor 301 is a technology known to those skilled in the art, and will not be described here.

[0124] Based on the same inventive concept, the embodiments of the present application also provide a computer readable storage medium storing computer instructions, when the computer instructions run on a computer, the computer instructions make the computer execute the steps of the abandoned object detection method as described above.

[0125] In some possible implementations, various aspects of the abandoned object detection method provided by the present application can also be implemented in the form of a program product, which includes program code for causing the detection device to execute the steps of the abandoned object detection method according to various exemplary embodiments of the present application described above in the specification when the program product runs on the electronic device.

[0126] Those skilled in the art will appreciate that embodiments of the present application can be devised for a variety of other systems which are currently developed or later developed. Therefore, the present application is intended to cover all such modifications and variations of this application that are within the scope of the appended claims and their equivalents. It is intended that each element of claim 1 and 2 is independent of one another. No element of claim 1 and 2, or any other claim, is implied to depend on any other element or limitation of claim 1 and 2 or any other claim except where expressly recited in that claim.

[0127] The present application is described in reference to the flowchart and / or block diagrams of the method, apparatus (system) and computer program product according to this application. It will be understood that each block of the flowchart and / or block diagrams, and combinations of blocks in the flowchart and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general purpose computer, special purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, create means for implementing the functions specified in the flowchart and / or block diagram block or blocks. Figure 1 Figure 1

[0128] These computer program instructions can also be stored in a computer- readable memory that can direct a computer or other programmable data processing apparatus to function in a particular manner, such that the instructions stored in the computer-readable memory produce an article of manufacture including instructions which implement the function specified in the flowchart and / or block diagram block or blocks. Figure 1 Figure 1

[0129] These computer program instructions can also be loaded onto a computer or other programmable data processing apparatus to cause a series of operational steps to be performed on the computer or other programmable apparatus to produce a computer implemented process such that the instructions which execute on the computer or other programmable apparatus provide steps for implementing the functions specified in the flowchart and / or block diagram block or blocks. Figure 1 Figure 1

[0130] Obviously, numerous modifications and variations of the present application are possible in light of the above teachings. It is therefore to be understood that within the scope of the appended claims and their equivalents, the application can be practiced otherwise than as specifically described.​​​​​​

Claims

1. A method of detecting a residual material, characterized by, The method comprises: target detection is performed on the collected multiple frames of images; a first image and a second image are obtained according to the detection result; the first image is a first frame of image in which a target object is detected in the multiple frames of images, and the second image is a first frame of image in which the target object is not detected in the multiple frames of images; the shooting time of the first image is earlier than the shooting time of the second image; a first difference image is obtained by performing difference on image pixel matrices of the first image and the second image; a region in which the target object is located in the first difference image is marked; other regions in which pixel values are greater than a first preset threshold except the marked region in the first difference image are determined as residual object regions.

2. The method of claim 1, wherein, The determination of the other regions in which the pixel values are greater than the first preset threshold except the marked region in the first difference image as the residual object regions comprises: filtering and binarization processing are performed on the first difference image to obtain a first binarization image; regions in which pixel values are greater than an average pixel value of the first binarization image except the marked region in the first binarization image are determined as the residual object regions.

3. The method of claim 1, wherein, The determination of the other regions in which the pixel values are greater than the first preset threshold except the marked region in the first difference image as the residual object regions further comprises: determination of whether objects in the other regions are stationary; if it is determined that the objects in the other regions are stationary, the other regions are determined as the residual object regions.

4. The method of claim 3, wherein, The determination of whether the objects in the other regions are stationary comprises: a third image is obtained; the third image is continuous with the second image and is shot later than the second image; a second difference image is obtained by performing difference on pixel matrices of the third image and the first image; filtering and binarization processing are performed on the second difference image to obtain a second binarization image; determination of whether a first region in which a pixel value is greater than an average pixel value of the second binarization image exists in the second binarization image; if the first region exists in the second binarization image, first region coordinates corresponding to the first region and second region coordinates corresponding to the other regions are respectively obtained; determination of whether a displacement between the first region coordinates and the second region coordinates is less than a second preset threshold; if the displacement between the first region coordinates and the second region coordinates is less than the second preset threshold, it is determined that the objects in the other regions are stationary.

5. A residue detection device, characterized by, The device comprises: a detection module configured to perform target detection on the collected multiple frames of images; a processing module configured to obtain a first image and a second image according to the detection result; the first image is a first frame of image in which a target object is detected in the multiple frames of images, and the second image is a first frame of image in which the target object is not detected in the multiple frames of images; the shooting time of the first image is earlier than the shooting time of the second image; the processing module is further configured to obtain a first difference image by performing difference on image pixel matrices of the first image and the second image; The processing module is further configured to mark a region in which the target object is located in the first difference image, and determine other regions in which pixel values are greater than a first preset threshold in the first difference image except the marked region as the residual region.

6. The apparatus of claim 5, wherein, The processing module is specifically configured to: perform filtering and binarization processing on the first difference image to obtain a first binarization image; determine a region in which a pixel value is greater than an average pixel value of the first binarization image except the marked region in the first binarization image as the residual region.

7. An electronic device, comprising: comprise: a memory configured to store program instructions; a processor configured to invoke the program instructions stored in the memory, and perform steps included in the method according to any one of claims 1-4 according to the obtained program instructions.

8. A computer-readable storage medium, characterized in that, The computer readable storage medium stores a computer program, and the computer program comprises program instructions. When the program instructions are executed by a computer, the computer is caused to perform the method according to any one of claims 1-4.

Images