Method and apparatus for detecting residual, electronic device, and computer-readable storage medium

Abstract

The application discloses a residual object detection method and device, electronic equipment and a computer readable storage medium. The method comprises the following steps: acquiring a current frame image and a background frame image corresponding to the current frame image, wherein the background frame image corresponding to the current frame image is obtained based on a preset background frame updating strategy; determining a foreground image corresponding to the current frame image according to the current frame image and the background frame image; performing residual object detection on the foreground image by using a preset object detection model to obtain a residual object detection result, wherein the preset object detection model is obtained based on a non-residual object image; and performing verification on the residual object detection result by using a preset verification strategy to obtain a verified residual object detection result in the case that the residual object detection result is a non-residual object image. The application realizes efficient detection of various types of residual objects and easily deformed objects, and the detection result is verified twice, thereby reducing the missed detection rate of residual objects and improving the accuracy of residual object detection.

Description

Technical Field

[0001] The present application relates to the field of target detection technology, and in particular to a method, device, electronic device, and computer-readable storage medium for detecting remains. Background Art

[0002] Object detection is an important branch of object detection and is particularly important in certain application scenarios. For example, in a snowy environment, if unwanted objects such as helmets and ski gear are left on the slopes of a ski resort, it could endanger the normal competition, recreational activities, and even the lives of athletes and ski enthusiasts. Therefore, detecting objects left on the slopes is essential.

[0003] In this regard, a detection solution provided in the prior art is to use deep learning algorithms such as YOLO (You Only Look Once) and SSD (Single Shot MultiBox Detector) algorithms to detect relics in the scene, and to issue an alarm if learned relics are detected. However, this solution can only identify the types of relics that the model has learned, and cannot detect and judge the newly added types of relics. In addition, the recognition effect for objects that are easily deformed, such as clothes, is very poor, and this type of relics is often also very common. In addition, there are too many types of relics, such as helmets, skis, ski poles, mineral water bottles, stones, clothes, etc. In response to this situation, the commonly used deep learning method can no longer adapt flexibly because there are too many types of samples and the learning cost is too high.

[0004] Another existing detection solution uses traditional machine learning methods to extract the foreground area of ​​an image. It then employs methods such as SVM (Support Vector Machines) and HOG (Histogram of Oriented Gradients) to distinguish objects in the foreground area and trigger an alarm if they are left behind. However, the algorithms used in this solution often cannot compare with deep learning algorithms in terms of processing speed and accuracy, and their implementation is complex, making them unable to meet detection requirements in terms of performance. Summary of the Invention

[0005] The embodiments of the present application provide a method, device, electronic device, and computer-readable storage medium for detecting relics to improve the accuracy and efficiency of relics detection.

[0006] The embodiments of this application adopt the following technical solutions:

[0007] In a first aspect, an embodiment of the present application provides a method for detecting residues, the method comprising:

[0008] Acquire a current frame image and a background frame image corresponding to the current frame image, wherein the background frame image corresponding to the current frame image is updated based on a preset background frame update strategy;

[0009] Determining a foreground image corresponding to the current frame image according to the current frame image and a background frame image corresponding to the current frame image;

[0010] Performing a legacy detection on a foreground image corresponding to the current frame image using a preset object detection model to obtain a legacy detection result, wherein the preset object detection model is trained based on non-legacy images;

[0011] In a case where the relic detection result indicates that the foreground image is a non-relic image, the relic detection result is verified using a preset verification strategy to obtain the verified relic detection result.

[0012] Optionally, acquiring the current frame image and a background frame image corresponding to the current frame image includes:

[0013] Get the background frame image corresponding to the previous frame image;

[0014] Determine the background area in the previous frame image;

[0015] The background region in the previous frame image is updated using a preset background frame update strategy to obtain a background frame image corresponding to the current frame image.

[0016] Optionally, determining the foreground image corresponding to the current frame image according to the current frame image and the background frame image corresponding to the current frame image includes:

[0017] Determine the absolute value of the grayscale value difference between the pixel points in the current frame image and the pixel points in the background frame image;

[0018] Determining a foreground pixel in the current frame image according to an absolute value of a grayscale value difference between a pixel in the current frame image and a pixel in the background frame image;

[0019] The foreground pixels in the current frame image are processed using a preset image processing strategy to obtain a foreground image corresponding to the current frame image.

[0020] Optionally, the preset object detection model can be trained in the following manner: obtaining an image to be trained, wherein the image to be trained contains labeling information of non-legacy objects; preprocessing the image to be trained to obtain a preprocessed image to be trained; using the preprocessed image to be trained to train the object detection model to obtain a trained object detection model; using a preset optimizer to accelerate the trained object detection model to obtain the preset object detection model.

[0021] Optionally, verifying the residue detection result by using a preset verification strategy to obtain the verified residue detection result includes:

[0022] determining whether the type of the non-legacy object in the legacy detection result is a dynamic target;

[0023] If so, multiple frames of historical frame images are obtained for detection to obtain historical frame detection results;

[0024] The relic detection result is verified according to the historical frame detection result to obtain the verified relic detection result.

[0025] Optionally, verifying the remnant detection result according to the historical frame detection result to obtain the verified remnant detection result includes:

[0026] If the historical frame detection result shows that the type of the non-legacy in the legacy detection result is a dynamic target, directly outputting the legacy detection result;

[0027] If the historical frame detection result shows that the type of the non-legacy in the legacy detection result is a static target, the legacy detection result is corrected and then output.

[0028] Optionally, verifying the residue detection result by using a preset verification strategy to obtain the verified residue detection result includes:

[0029] Determining an image scene type corresponding to the foreground image;

[0030] Determining target distribution characteristics corresponding to the foreground image according to the image scene type;

[0031] The residue detection result is verified according to the target distribution characteristics corresponding to the foreground image to obtain the verified residue detection result.

[0032] In a second aspect, an embodiment of the present application further provides a device for detecting residual objects, wherein the device comprises:

[0033] an acquiring unit, configured to acquire a current frame image and a background frame image corresponding to the current frame image, wherein the background frame image corresponding to the current frame image is updated based on a preset background frame update strategy;

[0034] a determining unit, configured to determine a foreground image corresponding to the current frame image based on the current frame image and a background frame image corresponding to the current frame image;

[0035] a relic detection unit, configured to perform relic detection on the foreground image corresponding to the current frame image using a preset object detection model, to obtain a relic detection result, wherein the preset object detection model is trained based on non-relic images;

[0036] The verification unit is used to verify the relic detection result using a preset verification strategy when the relic detection result shows that the foreground image is a non-relic image, so as to obtain the verified relic detection result.

[0037] In a third aspect, an embodiment of the present application further provides an electronic device, including:

[0038] processor; and

[0039] A memory arranged to store computer executable instructions, which when executed cause the processor to perform any of the aforementioned methods.

[0040] In a fourth aspect, an embodiment of the present application further provides a computer-readable storage medium, which stores one or more programs. When the one or more programs are executed by an electronic device including multiple applications, the electronic device executes any of the aforementioned methods.

[0041] At least one of the above technical solutions adopted in the embodiments of the present application can achieve the following beneficial effects: the method for detecting relics in the embodiments of the present application first obtains the current frame image and the background frame image corresponding to the current frame image, the background frame image corresponding to the current frame image is updated based on a preset background frame update strategy; then, based on the current frame image and the background frame image corresponding to the current frame image, the foreground image corresponding to the current frame image is determined; then, a preset object detection model is used to perform relics detection on the foreground image corresponding to the current frame image to obtain a relics detection result, the preset object detection model is trained based on non-relics images; finally, when the relics detection result shows that the foreground image is a non-relics image, the relics detection result is verified using a preset verification strategy to obtain a verified relics detection result. The method for detecting relics in the embodiments of the present application adopts a method that combines foreground image segmentation based on machine learning and foreground image detection based on deep learning to detect relics, thereby achieving efficient detection of multiple types of relics and easily deformed objects, and performing secondary verification on the detection results, thereby reducing the missed detection rate of relics and improving the accuracy of relics detection. BRIEF DESCRIPTION OF THE DRAWINGS

[0042] The drawings described herein are used to provide a further understanding of the present application and constitute a part of the present application. The illustrative embodiments of the present application and their descriptions are used to explain the present application and do not constitute an improper limitation on the present application. In the drawings:

[0043] Figure 1 Schematic diagram of a flow chart of a method for detecting residues in an embodiment of the present application;

[0044] Figure 2 This is a structural diagram of a residue detection device in an embodiment of the present application;

[0045] Figure 3 This is a structural diagram of an electronic device in an embodiment of the present application. DETAILED DESCRIPTION

[0046] To make the purpose, technical solutions, and advantages of this application more clear, the technical solutions of this application will be clearly and completely described below in conjunction with the specific embodiments of this application and the corresponding drawings. Obviously, the embodiments described are only part of the embodiments of this application, not all of them. Based on the embodiments in this application, all other embodiments obtained by ordinary technicians in this field without making creative efforts are within the scope of protection of this application.

[0047] The following describes in detail the technical solutions provided by various embodiments of the present application in conjunction with the accompanying drawings.

[0048] The present application provides a method for detecting residues. Figure 1As shown, a flow chart of a method for detecting residues in an embodiment of the present application is provided, wherein the method comprises at least the following steps S110 to S140:

[0049] Step S110 , obtaining a current frame image and a background frame image corresponding to the current frame image, wherein the background frame image corresponding to the current frame image is updated based on a preset background frame update strategy.

[0050] When performing debris detection in the embodiment of the present application, it is necessary to first obtain the current frame image captured by the camera in a specific scene. The specific scene here can be flexibly set according to actual needs. For example, it can be a ski resort, an airport runway, or a highway, or any scene where there is a need for debris detection.

[0051] In addition, it is also necessary to obtain the background frame image corresponding to the current frame image. The background frame image here can be updated through a pre-set background frame update strategy. The image information it represents is the background information in the image area captured by the camera, which is the basis for subsequent segmentation of foreground targets.

[0052] Step S120 : determining a foreground image corresponding to the current frame image according to the current frame image and the background frame image corresponding to the current frame image.

[0053] Since the current frame image may contain both foreground information and background information, after obtaining the current frame image and the background frame image corresponding to the current frame, the embodiment of the present application can compare the two, thereby segmenting the foreground area from the current frame image, that is, the image area that needs to be detected in focus, and obtaining the foreground image.

[0054] Step S130 , performing a legacy detection on the foreground image corresponding to the current frame image using a preset object detection model to obtain a legacy detection result, wherein the preset object detection model is trained based on non-legacy images.

[0055] Since there are many types of relics in actual scenes, and the detection of some relics is easily affected by the deformation of objects, in order to improve the detection accuracy, the object detection model trained in the embodiment of the present application is mainly used to detect non-relic objects in the image. Although these non-relic objects are also foreground targets in the image, they are not objects left behind or lost in the scene, such as people, animals, and vehicles.

[0056] Based on this, the embodiment of the present application can use the above-mentioned preset object detection model to perform relic detection on the foreground image obtained in the above steps. The preset object detection model can output the probability that the foreground image is a non-relic image. The larger the probability value, the less likely the foreground target contained in the foreground image is to be a relic. The smaller the probability value, the more likely the foreground target contained in the foreground image is to be a relic. Therefore, a probability value threshold can be used to determine whether the foreground image is a relic image, thereby obtaining a relic detection result.

[0057] Step S140 , when the residue detection result indicates that the foreground image is a non-residue image, the residue detection result is verified using a preset verification strategy to obtain the verified residue detection result.

[0058] The above-mentioned trained preset object detection model can accurately detect whether the image contains relics to a large extent, and can meet the needs of relic detection in most scenarios. However, due to considerations of training cost and training efficiency, the detection accuracy of the above-mentioned preset object detection model is difficult to reach 100%. Therefore, the embodiment of the present application can verify the relic detection results, thereby further improving the accuracy of the relic detection results.

[0059] Specifically, when the preset object detection model detects that the foreground image is a relic image, the detection result of the model can be directly trusted. However, when the preset object detection model detects that the foreground image is a non-relic image, certain verification strategies need to be adopted to further verify whether the relic detection result is reliable. For example, when the characteristics of the relic are very close to those of the non-relic, the preset object detection model may mistakenly identify the relic as a non-relic, resulting in missed detection. Therefore, the above-mentioned verification link can further reduce the missed detection rate of the model, thereby improving the accuracy of relic detection.

[0060] The relics detection method of the embodiment of the present application adopts a method that combines foreground image segmentation based on machine learning and foreground image detection based on deep learning to detect relics, thereby achieving efficient detection of various types of relics and easily deformed objects, and performing secondary verification of the detection results, thereby reducing the missed detection rate of relics and improving the accuracy of relic detection.

[0061] In one embodiment of the present application, obtaining the current frame image and the background frame image corresponding to the current frame image includes: obtaining the background frame image corresponding to the previous frame image; determining the background area in the previous frame image; and updating the background area in the previous frame image using a preset background frame update strategy to obtain the background frame image corresponding to the current frame image.

[0062] When obtaining the background frame image corresponding to the current frame image, the embodiment of the present application can first obtain the background frame image corresponding to the previous frame image. Since the purpose of updating the background frame is to restore the background frame image that does not contain any foreground targets as much as possible, when updating the background frame image, it is not necessary to update the foreground area in the image, only the background area needs to be updated.

[0063] Specifically, for the background frame image corresponding to the previous frame image, a sliding average method can be used to smoothly update the background frame image, thereby minimizing the influence of light and other factors. For example, the following method can be used:

[0064] bkframe'=(1-a)*bkframe+a*curframe

[0065] Among them, bkframe is the background frame image before updating, bkframe' is the background frame image after updating, curframe is the current frame image, and a is the learning rate.

[0066] Of course, those skilled in the art can also flexibly set other background frame update strategies to update the background frame image according to actual needs, which are not listed here one by one.

[0067] In one embodiment of the present application, determining the foreground image corresponding to the current frame image based on the current frame image and the background frame image corresponding to the current frame image includes: determining the absolute value of the grayscale value difference between the pixel points in the current frame image and the pixel points in the background frame image; determining the foreground pixel points in the current frame image based on the absolute value of the grayscale value difference between the pixel points in the current frame image and the pixel points in the background frame image; and processing the foreground pixel points in the current frame image using a preset image processing strategy to obtain the foreground image corresponding to the current frame image.

[0068] In the embodiment of the present application, when determining the foreground image corresponding to the current frame image, the current frame image and the background frame image may be first grayscale processed to obtain the corresponding current frame grayscale image and background frame grayscale image. Then, a grayscale value difference operation is performed between the current frame grayscale image and the corresponding background frame grayscale image to determine the foreground pixel point in the current frame grayscale image. For example, the following method may be used:

[0069] |It(x,y)-Bt(x,y)|;

[0070] Where It(x,y) is the grayscale value of any pixel in the current grayscale image, and Bt(x,y) is the grayscale value of any pixel in the background grayscale image. For any pixel in the current grayscale image, if |It(x,y)-Bt(x,y)|>T, the pixel is considered a foreground pixel; otherwise, it is considered a background pixel.

[0071] After the above steps, the foreground area in the current frame image can be determined, and then a series of morphological operations such as corrosion, expansion, opening and closing operations can be performed on the foreground area to obtain a connected foreground image.

[0072] Since the connected foreground image obtained after the above processing is equivalent to a local area cut out from the current frame image, and its shape is often irregular, the embodiment of the present application can further determine the circumscribed rectangle corresponding to the connected foreground image, and then expand it to a certain range to obtain the final foreground image, thereby meeting the detection requirements of the subsequent preset object detection model.

[0073] In addition, it should be noted that the above embodiment first updates the background frame image and then determines the foreground image corresponding to the current frame image. In actual application, it is also possible to first subtract the grayscale values ​​of the current frame image from the background frame image corresponding to the previous frame image to determine the foreground area in the current frame image, and then update the background based on the background area in the current frame image, because the difference between two adjacent background frame images is very small.

[0074] In one embodiment of the present application, the preset object detection model can be trained in the following manner: obtaining an image to be trained, wherein the image to be trained contains labeling information of non-legacy objects; preprocessing the image to be trained to obtain a preprocessed image to be trained; using the preprocessed image to be trained to train the object detection model to obtain a trained object detection model; using a preset optimizer to accelerate the trained object detection model to obtain the preset object detection model.

[0075] Specifically, the embodiment of the present application can use the ResNet50 network to train the preset object detection model. First, the image to be trained is obtained. Here, data sets such as CIFAR-10 can be used as training samples, and image data in actual detection scenarios, such as image data from the Winter Olympics, are mixed. For a total of 10 categories of common non-legacy objects such as people, dogs, and cars, 20,000 images of each category are obtained, of which 19,000 are used for training and the other 1,000 are used for testing. Therefore, a total of 190,000 training samples and 10,000 test samples are obtained. Then, the training images are preprocessed. For example, the training images can be scaled to a size of 32*32 to improve the efficiency of model training.

[0076] The pre-processed training images are then subjected to feature extraction and recognition using a pre-defined ResNet-50 network. This results in detection results for the training images. A defined loss function is then used to calculate the loss value for the training images, which is then used to update the network parameters. Training ends when the model's detection accuracy meets preset requirements, such as 97% accuracy and 96% recall, resulting in a trained object detection model. Finally, the model is accelerated using the TensorRT optimizer, creating an acceleration engine for subsequent inference. TensorRT is a high-performance deep learning inference optimizer that provides low-latency, high-throughput deployment and inference for deep learning applications.

[0077] Of course, the above training process is only a schematic description of the embodiment of the present application. Those skilled in the art can flexibly define the network structure, loss function and optimizer according to actual needs, and no specific limitations are made here.

[0078] In one embodiment of the present application, the relics detection result is verified using a preset verification strategy to obtain the verified relics detection result, including: determining whether the type of non-relics in the relics detection result is a dynamic target; if so, obtaining multiple frames of historical frame images for detection to obtain historical frame detection results; verifying the relics detection result according to the historical frame detection results to obtain the verified relics detection result.

[0079] The types of non-legacy objects defined in the embodiments of the present application may include dynamic targets in the image, such as people, dogs, etc., while relics are often static targets. Based on this, the relics detection results can be further verified.

[0080] Specifically, first determine whether the type of non-legacy detected by the model is a dynamic target such as a person or dog. If it is a dynamic target, then the feature changes such as the position of the dynamic target in multiple frames should be relatively obvious. Therefore, a certain number of historical frame images can be obtained to detect non-legacy objects, and the historical frame detection results can be obtained. Then, the historical frame detection results of these multiple frames can be compared to determine whether the detected non-legacy objects are actually moving in the multiple frames. This can verify whether the legacy detection results output by the model are accurate.

[0081] In one embodiment of the present application, the relic detection result is verified based on the historical frame detection result to obtain the verified relic detection result, including: if the historical frame detection result shows that the type of non-relic in the relic detection result is a dynamic target, then the relic detection result is directly output; if the historical frame detection result shows that the type of non-relic in the relic detection result is a static target, then the relic detection result is corrected and then output.

[0082] If, after comparing the detection results of multiple historical frames, it is found that a non-legacy object of the dynamic target type has not moved between the multiple frames, this may indicate that the model has misdetected a legacy object, that is, it has detected a non-legacy object. In this case, it is necessary to correct the model's legacy detection results and issue a legacy alarm. Conversely, if, after comparing the detection results of multiple historical frames, it is found that a non-legacy object of the dynamic target type has moved between the multiple frames, this proves that the model's detection results are accurate, and the legacy detection results can be directly output.

[0083] The verification strategy of the above embodiment can reduce the possibility of model omission to a certain extent, ensure that all types of relics are detected to the greatest extent, and further ensure the safety of the environment in certain special scenarios.

[0084] In one embodiment of the present application, the relic detection result is verified using a preset verification strategy to obtain the verified relic detection result, which includes: determining the image scene type corresponding to the foreground image; determining the target distribution characteristics corresponding to the foreground image based on the image scene type; and verifying the relic detection result based on the target distribution characteristics corresponding to the foreground image to obtain the verified relic detection result.

[0085] In actual application, the target distribution characteristics of different scenes are different, and the target distribution characteristics of the same scene in different time periods, different seasons and different scene areas may also be different. Based on this, the embodiment of the present application can adopt certain verification strategies based on the characteristics of different scenes to further verify the detection results of the model, improve the accuracy of the detection results, and at the same time, it can also improve the universality of the preset object detection model for the detection of relics in different application scenarios to a certain extent, and make up for the detection errors caused by scene differences.

[0086] Specifically, embodiments of the present application can pre-analyze the target distribution characteristics of different image scene types across different time periods, seasons, and scene areas. For example, under normal circumstances, the clearway area of ​​an airport runway must be free of obstacles other than runway lights. Another example is that during off-hours at a ski resort or on temporarily closed sections of a highway, moving objects such as people and vehicles are generally prohibited.

[0087] Based on this, the embodiment of the present application can determine the image scene type corresponding to the currently obtained foreground image, such as a ski resort, an airport runway, or a highway scene, and then determine the image acquisition time period and acquisition area location corresponding to the currently obtained foreground image, and then combine the target distribution characteristics corresponding to the above-mentioned image scene type in the image acquisition time period and acquisition area location for analysis. If the residue detection result meets the target distribution characteristics, then it proves that the detection result of the model is accurate. If the residue detection result does not meet the target distribution characteristics, it means that the detection result of the model is inaccurate, and a residue alarm or other processing is required.

[0088] The present application also provides a residue detection device 200, such as Figure 2 As shown, a schematic structural diagram of a residue detection device in an embodiment of the present application is provided. The device 200 includes: an acquisition unit 210, a determination unit 220, a residue detection unit 230, and a verification unit 240, wherein:

[0089] An acquiring unit 210 is configured to acquire a current frame image and a background frame image corresponding to the current frame image, wherein the background frame image corresponding to the current frame image is updated based on a preset background frame update strategy;

[0090] A determining unit 220, configured to determine a foreground image corresponding to the current frame image based on the current frame image and a background frame image corresponding to the current frame image;

[0091] A remnant detection unit 230 is configured to perform remnant detection on the foreground image corresponding to the current frame image using a preset object detection model, wherein the preset object detection model is trained based on non-remnant images to obtain a remnant detection result;

[0092] The verification unit 240 is configured to verify the relic detection result using a preset verification strategy when the relic detection result indicates that the foreground image is a non-relic image, to obtain the verified relic detection result.

[0093] In one embodiment of the present application, the acquisition unit 210 is specifically used to: acquire a background frame image corresponding to a previous frame image; determine the background area in the previous frame image; and update the background area in the previous frame image using a preset background frame update strategy to obtain a background frame image corresponding to the current frame image.

[0094] In one embodiment of the present application, the determination unit 220 is specifically used to: determine the absolute value of the grayscale value difference between the pixel points in the current frame image and the pixel points in the background frame image; determine the foreground pixel points in the current frame image based on the absolute value of the grayscale value difference between the pixel points in the current frame image and the pixel points in the background frame image; and use a preset image processing strategy to process the foreground pixel points in the current frame image to obtain a foreground image corresponding to the current frame image.

[0095] In one embodiment of the present application, the preset object detection model can be trained in the following manner: obtaining an image to be trained, wherein the image to be trained contains labeling information of non-legacy objects; preprocessing the image to be trained to obtain a preprocessed image to be trained; using the preprocessed image to be trained to train the object detection model to obtain a trained object detection model; using a preset optimizer to accelerate the trained object detection model to obtain the preset object detection model.

[0096] In one embodiment of the present application, the verification unit 240 is specifically used to: determine whether the type of non-relic in the relic detection result is a dynamic target; if so, obtain multiple frames of historical frame images for detection to obtain historical frame detection results; verify the relic detection result based on the historical frame detection result to obtain the verified relic detection result.

[0097] In one embodiment of the present application, the verification unit 240 is specifically used to: if the historical frame detection result shows that the type of non-legacy in the relic detection result is a dynamic target, then directly output the relic detection result; if the historical frame detection result shows that the type of non-legacy in the relic detection result is a static target, then correct the relic detection result and then output it.

[0098] In one embodiment of the present application, the verification unit 240 is specifically used to: determine the image scene type corresponding to the foreground image; determine the target distribution characteristics corresponding to the foreground image according to the image scene type; verify the relic detection result according to the target distribution characteristics corresponding to the foreground image to obtain the verified relic detection result.

[0099] It can be understood that the above-mentioned relic detection device can implement the various steps of the relic detection method performed by the clearing server provided in the above-mentioned embodiment. The relevant explanations about the relic detection method are applicable to the relic detection device and will not be repeated here.

[0100] Figure 3 This is a schematic diagram of the structure of an electronic device according to an embodiment of the present application. Figure 3 At the hardware level, the electronic device includes a processor and, optionally, an internal bus, a network interface, and memory. The memory may include internal memory, such as high-speed random-access memory (RAM), or non-volatile memory, such as at least one disk drive. Of course, the electronic device may also include other hardware required for its services.

[0101] The processor, network interface, and memory can be interconnected via an internal bus, which can be an ISA (Industry Standard Architecture) bus, a PCI (Peripheral Component Interconnect) bus, or an EISA (Extended Industry Standard Architecture) bus. The bus can be divided into an address bus, a data bus, a control bus, etc. For ease of representation, Figure 3 Only one bidirectional arrow is used in the diagram, but this does not mean that there is only one bus or one type of bus.

[0102] The memory is used to store programs. Specifically, the program may include program code, which includes computer operating instructions. The memory may include internal memory and non-volatile memory, and provides instructions and data to the processor.

[0103] The processor reads the corresponding computer program from the non-volatile memory into the internal memory and then runs it, forming a residue detection device at the logical level. The processor executes the program stored in the memory and is specifically used to perform the following operations:

[0104] Acquire a current frame image and a background frame image corresponding to the current frame image, wherein the background frame image corresponding to the current frame image is updated based on a preset background frame update strategy;

[0105] Determining a foreground image corresponding to the current frame image according to the current frame image and a background frame image corresponding to the current frame image;

[0106] Performing a legacy detection on a foreground image corresponding to the current frame image using a preset object detection model to obtain a legacy detection result, wherein the preset object detection model is trained based on non-legacy images;

[0107] In a case where the relic detection result indicates that the foreground image is a non-relic image, the relic detection result is verified using a preset verification strategy to obtain the verified relic detection result.

[0108] The above application Figure 1 The method performed by the residue detection device disclosed in the illustrated embodiment can be applied to a processor or implemented by a processor. The processor may be an integrated circuit chip with signal processing capabilities. During implementation, each step of the above method can be completed by an integrated logic circuit of hardware in the processor or by instructions in the form of software. The above processor can be a general-purpose processor, including a central processing unit (CPU), a network processor (NP), etc.; it can also be a digital signal processor (DSP), an application-specific integrated circuit (ASIC), a field-programmable gate array (FPGA) or other programmable logic devices, discrete gates or transistor logic devices, discrete hardware components. The various methods, steps and logic block diagrams disclosed in the embodiments of the present application can be implemented or executed. The general-purpose processor can be a microprocessor or the processor can also be any conventional processor, etc. The steps of the method disclosed in conjunction with the embodiments of the present application can be directly embodied as being executed by a hardware decoding processor, or executed by a combination of hardware and software modules in the decoding processor. The software module can be located in a storage medium well-known in the art, such as random access memory, flash memory, read-only memory, programmable read-only memory, electrically erasable programmable memory, registers, etc. The storage medium is located in the memory, and the processor reads the information in the memory and, in conjunction with its hardware, completes the steps of the above method.

[0109] The electronic device may also perform Figure 1 The method executed by the residue detection device in the Figure 1 The functions of the illustrated embodiment will not be described in detail in the embodiments of the present application.

[0110] The embodiment of the present application also provides a computer-readable storage medium, which stores one or more programs, wherein the one or more programs include instructions, which, when executed by an electronic device including multiple application programs, can enable the electronic device to execute Figure 1 The method performed by the residue detection device in the illustrated embodiment is specifically used to perform:

[0111] Acquire a current frame image and a background frame image corresponding to the current frame image, wherein the background frame image corresponding to the current frame image is updated based on a preset background frame update strategy;

[0112] Determining a foreground image corresponding to the current frame image according to the current frame image and a background frame image corresponding to the current frame image;

[0113] Performing a legacy detection on a foreground image corresponding to the current frame image using a preset object detection model to obtain a legacy detection result, wherein the preset object detection model is trained based on non-legacy images;

[0114] In a case where the relic detection result indicates that the foreground image is a non-relic image, the relic detection result is verified using a preset verification strategy to obtain the verified relic detection result.

[0115] It will be understood by those skilled in the art that embodiments of the present invention may be provided as methods, systems, or computer program products. Thus, the present invention may take the form of an entirely hardware embodiment, an entirely software embodiment, or an embodiment combining software and hardware. Furthermore, the present invention may take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to magnetic disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0116] The present invention is described with reference to flowcharts and / or block diagrams of methods, devices (systems), and computer program products according to embodiments of the present invention. It should be understood that each process and / or block in the flowcharts and / or block diagrams, as well as combinations of processes and / or blocks in the flowcharts and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to produce a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the processes in the flowcharts and / or block diagrams. Figure 1 a process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0117] These computer program instructions may also be stored in a computer readable memory that can direct a computer or other programmable data processing device to work in a specific manner, so that the instructions stored in the computer readable memory produce an article of manufacture comprising an instruction device, which implements the process Figure 1 a process or multiple processes and / or boxes Figure 1 The function specified in one or more boxes.

[0118] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operational steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing the instructions executed on the computer or other programmable device for implementing the process. Figure 1 a process or multiple processes and / or boxes Figure 1 A step that specifies a function in one or more boxes.

[0119] In a typical configuration, a computing device includes one or more processors (CPUs), input / output interfaces, network interfaces, and memory.

[0120] Memory may include non-permanent storage in a computer-readable medium, random access memory (RAM) and / or non-volatile memory in the form of read-only memory (ROM) or flash RAM. Memory is an example of a computer-readable medium.

[0121] Computer-readable media includes permanent and non-permanent, removable and non-removable media that can be implemented by any method or technology to store information. The information can be computer-readable instructions, data structures, program modules or other data. Examples of computer storage media include, but are not limited to, phase change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technology, compact disc read-only memory (CD-ROM), digital versatile disc (DVD) or other optical storage, magnetic cassettes, magnetic tape, magnetic disk storage or other magnetic storage devices or any other non-transmission media that can be used to store information that can be accessed by a computing device. As defined herein, computer-readable media does not include transitory computer-readable media (transitory media), such as modulated data signals and carrier waves.

[0122] It should also be noted that the terms "comprises," "includes," or any other variations thereof are intended to encompass non-exclusive inclusion, such that a process, method, commodity, or apparatus that includes a series of elements includes not only those elements but also other elements not explicitly listed, or includes elements inherent to such process, method, commodity, or apparatus. In the absence of further limitations, an element defined by the phrase "comprises a ..." does not exclude the presence of other identical elements in the process, method, commodity, or apparatus that includes the element.

[0123] Those skilled in the art will appreciate that the embodiments of the present application may be provided as methods, systems, or computer program products. Therefore, the present application may take the form of a complete hardware embodiment, a complete software embodiment, or an embodiment combining software and hardware. Furthermore, the present application may take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to magnetic disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0124] The foregoing is merely an embodiment of the present application and is not intended to limit the present application. For those skilled in the art, the present application may have various changes and variations. Any modifications, equivalent replacements, improvements, etc. made within the spirit and principles of the present application should all be included within the scope of the claims of the present application.

Claims

1. A method for detecting residues, wherein: The method comprises: Acquire a current frame image and a background frame image corresponding to the current frame image, wherein the background frame image corresponding to the current frame image is updated based on a preset background frame update strategy; Determining a foreground image corresponding to the current frame image according to the current frame image and a background frame image corresponding to the current frame image; Performing a legacy detection on a foreground image corresponding to the current frame image using a preset object detection model to obtain a legacy detection result, wherein the preset object detection model is trained based on non-legacy images; When the relic detection result indicates that the foreground image is a non-relic image, verifying the relic detection result using a preset verification strategy to obtain the verified relic detection result; The verification of the residue detection result by using a preset verification strategy to obtain the verified residue detection result includes: Determining an image scene type corresponding to the foreground image; Determining target distribution characteristics corresponding to the foreground image according to the image scene type; Verifying the residue detection result according to the target distribution characteristics corresponding to the foreground image to obtain the verified residue detection result; The verifying the residue detection result by using a preset verification strategy to obtain the verified residue detection result further includes: The target distribution characteristics of different image scene types in different time periods, different seasons and different scene areas are statistically analyzed in advance.

2. The method according to claim 1, wherein: The acquiring of the current frame image and the background frame image corresponding to the current frame image comprises: Get the background frame image corresponding to the previous frame image; Determine the background area in the previous frame image; The background region in the previous frame image is updated using a preset background frame update strategy to obtain a background frame image corresponding to the current frame image.

3. The method according to claim 1, wherein: The determining, based on the current frame image and the background frame image corresponding to the current frame image, of a foreground image corresponding to the current frame image includes: Determine the absolute value of the grayscale value difference between the pixel points in the current frame image and the pixel points in the background frame image; Determining a foreground pixel in the current frame image according to an absolute value of a grayscale value difference between a pixel in the current frame image and a pixel in the background frame image; The foreground pixels in the current frame image are processed using a preset image processing strategy to obtain a foreground image corresponding to the current frame image.

4. The method according to claim 1, wherein: The preset object detection model is trained in the following way: Acquire an image to be trained, wherein the image to be trained includes labeling information of non-legacy objects; Preprocessing the image to be trained to obtain a preprocessed image to be trained; Training the object detection model using the preprocessed image to be trained to obtain a trained object detection model; The trained object detection model is accelerated using a preset optimizer to obtain the preset object detection model.

5. The method according to claim 1, wherein: The verification of the residue detection result by using a preset verification strategy to obtain the verified residue detection result includes: determining whether the type of the non-legacy object in the legacy detection result is a dynamic target; If so, multiple frames of historical frame images are obtained for detection to obtain historical frame detection results; The relic detection result is verified according to the historical frame detection result to obtain the verified relic detection result.

6. The method according to claim 5, wherein: The verifying the remnant detection result according to the historical frame detection result to obtain the verified remnant detection result includes: If the historical frame detection result shows that the type of the non-legacy in the legacy detection result is a dynamic target, directly outputting the legacy detection result; If the historical frame detection result shows that the type of the non-legacy in the legacy detection result is a static target, the legacy detection result is corrected and then output.

7. A device for detecting residues, wherein: The device comprises: an acquiring unit, configured to acquire a current frame image and a background frame image corresponding to the current frame image, wherein the background frame image corresponding to the current frame image is updated based on a preset background frame update strategy; a determining unit, configured to determine a foreground image corresponding to the current frame image based on the current frame image and a background frame image corresponding to the current frame image; a relic detection unit, configured to perform relic detection on the foreground image corresponding to the current frame image using a preset object detection model, to obtain a relic detection result, wherein the preset object detection model is trained based on non-relic images; a verification unit, configured to verify the relic detection result using a preset verification strategy to obtain the verified relic detection result when the relic detection result indicates that the foreground image is a non-relic image; The verification unit is specifically used for: Determining an image scene type corresponding to the foreground image; Determining target distribution characteristics corresponding to the foreground image according to the image scene type; Verifying the residue detection result according to the target distribution characteristics corresponding to the foreground image to obtain the verified residue detection result; The verification unit is further configured to: The target distribution characteristics of different image scene types in different time periods, different seasons and different scene areas are statistically analyzed in advance.

8. An electronic device comprising: processor; as well as A memory arranged to store computer-executable instructions, which, when executed, cause the processor to perform the method of any one of claims 1 to 6.

9. A computer-readable storage medium storing one or more programs, wherein when the one or more programs are executed by an electronic device including a plurality of application programs, the electronic device executes the method according to any one of claims 1 to 6.

Images