Vehicle Debris Detection Method, Device, Electronic Device, and Storage Medium

Abstract

The present application relates to a vehicle throwing object detection method, device, electronic device, and storage medium. Among them, the vehicle throwing object detection method includes: obtaining a reference background image and a to-be-processed image frame of a target shooting scene; updating the reference background image based on the non-interested area of the to-be-processed image frame to obtain a target background image, where the non-interested area includes other areas in the to-be-processed image frame except the to-be-processed vehicle area and the to-be-processed object area; determining a target object area in the target background image that has the same position information as the to-be-processed object area; determining whether the to-be-processed object area contains throwing objects based on the feature similarity between the target object area and the to-be-processed object area. Through the present application, the problem of low detection rate of throwing objects caused by the throwing objects being blocked when the vehicle speed is fast and there are serious mutual blockages between vehicles in a high-speed scenario is avoided, thereby improving the detection rate of throwing objects.

Description

Technical Field

[0001] The present application relates to the field of image processing, and particularly to a method and apparatus for detecting vehicle litter, an electronic device, and a storage medium. Background Art

[0002] With the development of society and the improvement of living standards, the number of vehicles in traffic scenarios such as urban roads and highways has gradually increased. Therefore, the probability of litter appearing in high-speed scenarios has also gradually increased. Vehicles on highways travel at relatively high speeds, and the appearance of litter will pose serious safety hazards.

[0003] In order to reduce the danger of litter, in related technologies, some detect vehicles through object detection. For example, in the case of detecting a target vehicle from a target image frame, a reference image frame before the target image frame is determined from a target video; the reference image frame includes the target vehicle and is the image frame with the fewest other target objects except the target vehicle. Finally, based on the target image frame and the reference image frame, it is determined whether there is litter falling on the target vehicle, so as to identify litter such as stones. However, through the above method, in the case of high vehicle speeds and severe mutual occlusion of vehicles in high-speed scenarios, detecting litter only in the area around the vehicle is very likely to fail to detect the litter when it appears.

[0004] In view of the problem of low detection rate of litter in related technologies, no effective solution has been proposed yet. Summary of the Invention

[0005] In this embodiment, a method and apparatus for detecting vehicle litter, an electronic device, and a storage medium are provided to solve the problem of low detection rate of litter in related technologies.

[0006] In a first aspect, in this embodiment, a method for detecting vehicle litter is provided, including:

[0007] Obtaining a reference background image and a to-be-processed image frame of a target shooting scene;

[0008] Updating the reference background image based on a non-interested area of the to-be-processed image frame to obtain a target background image, where the non-interested area includes other areas in the to-be-processed image frame except the to-be-processed vehicle area and the to-be-processed object area;

[0009] Determining a target object area in the target background image that has the same position information as the to-be-processed object area;

[0010] Based on the feature similarity between the target object area and the to-be-processed object area, determining whether the to-be-processed object area contains litter.

[0011] In some of these embodiments, updating the reference background image based on the non - interested region of the image frame to be processed to obtain a target background image includes:

[0012] Determine the pixel values of the non - interested region;

[0013] Replace the pixel values in the reference background image corresponding to the position information of the non - interested region with the pixel values of the non - interested region to obtain the target background image.

[0014] In some of these embodiments, the reference background image includes: a historical scatter region; updating the reference background image based on the non - interested region of the image frame to be processed to obtain a target background image includes:

[0015] Update the region of the reference background image other than the historical scatter region based on the non - interested region of the image frame to be processed to obtain the target background image.

[0016] In some of these embodiments, determining whether the region of the object to be processed contains scatter based on the feature similarity between the target object region and the region of the object to be processed includes:

[0017] Input the image corresponding to the region of the object to be processed into the trained feature extraction model to obtain the feature of the object to be processed in the region of the object to be processed, and input the image corresponding to the target object region into the trained feature extraction model to obtain the feature of the target object in the target object region;

[0018] Determine the feature similarity between the target object feature and the feature of the object to be processed;

[0019] In the case where the feature similarity is less than a preset feature similarity, it is determined that the region of the object to be processed contains scatter.

[0020] In some of these embodiments, in the case where the feature similarity is less than a preset feature similarity, determining that the region of the object to be processed contains scatter includes:

[0021] Determine the cosine distance between the feature of the object to be processed and the target object feature;

[0022] In the case where the cosine distance is greater than a preset distance, it is determined that the feature similarity is less than the preset feature similarity, and it is determined that the region of the object to be processed contains scatter.

[0023] In some of these embodiments, after determining that the object region to be processed contains scattered objects when the feature similarity is less than a preset feature similarity, the method further includes:

[0024] Obtain historical scattered object information, where the historical scattered object information includes: scattered object feature information and scattered object position information;

[0025] According to the historical scattered object position information, detect whether there is a target scattered object whose position information is the same as that of the scattered objects in the object region to be processed;

[0026] When it is detected that there is a target scattered object whose position information is the same as that of the scattered objects in the object region to be processed, detect whether the feature information of the scattered objects in the object region to be processed is the same as the target scattered object feature information;

[0027] When it is detected that the feature information of the scattered objects in the object region to be processed is the same as the target scattered object feature information, merge the scattered objects in the object region to be processed into the target scattered object information and delete the scattered object information of the object region to be processed.

[0028] In some of these embodiments, when it is detected that there is no target scattered object whose position information is the same as that of the scattered objects in the object region to be processed, or when it is detected that the feature information of the scattered objects in the object region to be processed is different from the target scattered object feature information, the method further includes:

[0029] Take the scattered objects in the object region to be processed as new scattered objects, and obtain the information of the new scattered objects and save it into the historical scattered object information.

[0030] In some of these embodiments, the historical scattered object information further includes: scattered object loss time. When it is detected that the feature information of the scattered objects in the object region to be processed is the same as the target scattered object feature information, after merging the scattered objects in the object region to be processed into the target scattered object information and deleting the scattered object information of the object region to be processed, the method further includes:

[0031] According to the historical scattered object information, determine whether there is a first scattered object whose scattered object loss time is greater than a first preset time;

[0032] When it is determined that there is a first scattered object whose scattered object loss time is greater than a first preset time, delete the information corresponding to the first scattered object.

[0033] In some of these embodiments, the historical spill information further includes: the spill residence time. When it is detected that the characteristic information of the spill in the area of the object to be processed is the same as the target spill characteristic information, after merging the spill in the area of the object to be processed into the target spill information and deleting the spill information in the area of the object to be processed, the method further includes:

[0034] According to the historical spill information, determine whether there is a second spill with a spill residence time greater than a second preset time;

[0035] When it is determined that there is a second spill with a spill residence time greater than the second preset time, report the information corresponding to the second spill.

[0036] In a second aspect, in this embodiment, a vehicle spill detection device is provided, including:

[0037] A first acquisition module, configured to acquire a reference background image and an image frame to be processed of a target shooting scene;

[0038] An update module, configured to update the reference background image based on the non - interested area of the image frame to be processed to obtain a target background image, where the non - interested area includes other areas in the image frame to be processed except the area of the vehicle to be processed and the area of the object to be processed;

[0039] A first determination module, configured to determine a target object area in the target background image that has the same position information as the area of the object to be processed;

[0040] A second determination module, configured to determine whether the area of the object to be processed contains a spill based on the feature similarity between the target object area and the area of the object to be processed.

[0041] In a third aspect, in this embodiment, an electronic device is provided, including a memory, a processor, and a computer program stored on the memory and executable on the processor. When the processor executes the computer program, the vehicle spill detection method described in the first aspect above is implemented.

[0042] In a fourth aspect, in this embodiment, a storage medium is provided, on which a computer program is stored. When the program is executed by a processor, the vehicle spill detection method described in the first aspect above is implemented.

[0043] Compared with the related art, in the vehicle litter detection method, device, electronic device, and storage medium provided in this embodiment, by obtaining a reference background image and a to-be-processed image frame of a target shooting scene; updating the reference background image based on the non-interested regions of the to-be-processed image frame to obtain a target background image, where the non-interested regions include other regions in the to-be-processed image frame except the to-be-processed vehicle region and the to-be-processed object region; determining a target object region in the target background image that has the same position information as the to-be-processed object region; and determining whether the to-be-processed object region contains litter based on the feature similarity between the target object region and the to-be-processed object region, the problem of low litter detection rate caused by litter being blocked in the case of high-speed scenarios where vehicle speeds are relatively fast and there are serious mutual blockages between vehicles is avoided, thereby improving the litter detection rate.

[0044] Details of one or more embodiments of the present application are set forth in the following drawings and description to make other features, objects, and advantages of the present application more concise and understandable. BRIEF DESCRIPTION OF THE DRAWINGS

[0045] The drawings described herein are used to provide a further understanding of the present application and form a part of the present application. The illustrative embodiments and descriptions thereof of the present application are used to explain the present application and do not constitute an improper limitation of the present application. In the drawings:

[0046] Figure 1 is a hardware structure block diagram of a terminal of the vehicle litter detection method in this embodiment;

[0047] Figure 2 is a flowchart of the vehicle litter detection method in this embodiment;

[0048] Figure 3 is a schematic diagram of a to-be-processed frame image captured in a target shooting scene in this embodiment;

[0049] Figure 4 is the preferred process of the vehicle litter detection method in this embodiment Figure 1 ;

[0050] Figure 5 is the preferred process of the vehicle litter detection method in this embodiment Figure 2 ;

[0051] Figure 6 is the preferred process of the vehicle litter detection method in this embodiment Figure 3 ;

[0052] Figure 7 is a structure block diagram of the vehicle litter detection device in this embodiment. DETAILED DESCRIPTION OF THE EMBODIMENTS

[0053] To better understand the purpose, technical solution, and advantages of this application, the following describes and explains this application in conjunction with the accompanying drawings and embodiments.

[0054] Unless otherwise defined, the technical terms or scientific terms involved in this application shall have the general meaning understood by those with ordinary skills in the technical field to which this application belongs. In this application, words such as "a", "one", "a kind of", "the", "these", etc. do not indicate a limitation in quantity, and they can be singular or plural. The terms "include", "comprise", "have" and any variants thereof involved in this application are intended to cover non-exclusive inclusion; for example, a process, method, system, product, or device that includes a series of steps or modules (units) is not limited to the listed steps or modules (units), but may include unlisted steps or modules (units), or may include other steps or modules (units) inherent in these processes, methods, products, or devices. The terms "connected", "coupled", etc. involved in this application do not limit to physical or mechanical connections, but may include electrical connections, whether directly or indirectly connected. The "plurality" involved in this application refers to two or more. "And / or" describes the association relationship of associated objects, indicating that three relationships can exist. For example, "A and / or B" can represent: A exists alone, A and B exist simultaneously, and B exists alone. Usually, the character " / " indicates that the objects associated before and after are in an "or" relationship. The terms "first", "second", "third", etc. involved in this application only distinguish similar objects and do not represent a specific sorting for the objects.

[0055] The method embodiment provided in this embodiment can be executed on a terminal, a computer, or a similar computing device. For example, running on a terminal, Figure 1 is the hardware structure block diagram of the terminal of the vehicle throwing object detection method in this embodiment. As Figure 1 shown, the terminal may include one or more ( Figure 1 only one is shown in the figure) processors 102 and a memory 104 for storing data. Among them, the processor 102 may include, but is not limited to, a processing device such as a microprocessor MCU or a programmable logic device FPGA. The above terminal may also include a transmission device 106 for communication functions and an input / output device 108. Those of ordinary skill in the art can understand that Figure 1 the structure shown is only schematic and does not limit the structure of the above terminal. For example, the terminal may also include more or fewer components than Figure 1 shown in the figure, or have a different configuration from Figure 1 shown.

[0056] The memory 104 can be used to store computer programs, such as software programs and modules of application software, such as the computer program corresponding to the vehicle litter detection method in this embodiment. The processor 102 executes various functional applications and data processing by running the computer program stored in the memory 104, that is, the above method is implemented. The memory 104 may include a high-speed random access memory, and may also include a non-volatile memory, such as one or more magnetic storage devices, flash memory, or other non-volatile solid-state memories. In some instances, the memory 104 may further include a memory remotely disposed relative to the processor 102, and these remote memories can be connected to the terminal through a network. Examples of the above network include but are not limited to the Internet, enterprise intranet, local area network, mobile communication network, and combinations thereof.

[0057] The transmission device 106 is used to receive or send data via a network. The above network includes a wireless network provided by a communication provider of the terminal. In one instance, the transmission device 106 includes a network adapter (Network Interface Controller, abbreviated as NIC), which can be connected to other network devices through a base station and thus communicate with the Internet. In one instance, the transmission device 106 can be a radio frequency (RF) module, which is used to communicate with the Internet wirelessly.

[0058] In this embodiment, a vehicle litter detection method is provided. Figure 2 It is a flowchart of the vehicle litter detection method in this embodiment, as Figure 2 shown, and the process includes the following steps:

[0059] Step S201, obtain a reference background image and an image frame to be processed of the target shooting scene.

[0060] In this step, the reference background image can be the first frame image captured in the target shooting scene, or the previous frame image of the image frame to be processed. In the case where the image frame to be processed is the first frame image, the reference background image is itself.

[0061] Step S202, update the reference background image based on the non-interested region of the image frame to be processed to obtain a target background image, where the non-interested region includes other regions in the image frame to be processed except the vehicle region to be processed and the object region to be processed.

[0062] In this step, the extraction of the non - interested regions in the image frame to be processed can be achieved through a neural network model in some related technologies. This neural network model is trained to have a certain recognition accuracy, or it can also be other methods that can achieve the extraction of non - interested regions. Secondly, based on some classification models, the non - interested regions are classified to obtain the vehicle region to be processed, the object region to be processed, and other regions.

[0063] It should be noted that updating the reference background image based on the non - interested regions of the image frame to be processed only updates the regions in the reference background image corresponding to the position information of the non - interested regions.

[0064] Next, an embodiment in an application scenario is used to describe and illustrate the extraction of non - interested regions in the image frame to be processed by a neural network model.

[0065] In this embodiment, taking the deep - learning object - detection algorithm as an example of the neural network model, the deep - learning object - detection algorithm is used to obtain the detection information of motor vehicles (equivalent to the vehicle region to be processed in the above - mentioned embodiment) and foreground object targets (equivalent to the object region to be processed in the above - mentioned embodiment) from the target shooting scene region of the image frame to be processed, as Figure 3 shown. Among them, the foreground object represents a movable object with three - dimensional information, and the opposite background object represents an immovable object that exists fixedly in the environment (for example, bridges, roads, railings, etc.). The detection information includes the position information of each target, that is, Figure 3 the coordinate box in, for the convenience of description, the detection information of motor vehicles and foreground object targets can be respectively called the motor - vehicle box and the object box.

[0066] Step S203: Determine the target object region in the target background image that has the same position information as the object region to be processed.

[0067] In this step, the image sizes of the target shooting scenes are basically the same. Therefore, the method for determining the target object region in the target background image that has the same position information as the object region to be processed can be to find the region in the corresponding target background image that has the same position information as this object region to be processed, and this same region is the target object region.

[0068] It should be noted that the position information can be the position of the rectangular box representing the object in the image.

[0069] Step S204: Based on the feature similarity between the target object region and the object region to be processed, determine whether the object region to be processed contains scattered objects.

[0070] Based on the above steps S201 to S204, by updating the reference background image based on the non-interested regions of the image frame to be processed, a target background image is obtained. Then, the target object region in the target background image that has the same position information as the object region to be processed is determined. Next, based on the feature similarity between the target object region and the object region to be processed, the method of determining whether the object region to be processed contains scattered objects realizes the detection of scattered objects in the target shooting scene. Instead of taking the vehicle as the detection main body and only detecting the area around the vehicle, it avoids the problem that scattered objects are blocked when the vehicle speed is fast and there are serious mutual occlusions among vehicles in the high-speed scene, resulting in a low detection rate of scattered objects, thereby improving the detection rate of scattered objects.

[0071] In some of these embodiments, updating the reference background image based on the non-interested regions of the image frame to be processed to obtain the target background image includes: determining the pixel values of the non-interested regions; replacing the pixel values in the reference background image corresponding to the position information of the non-interested regions with the pixel values of the non-interested regions to obtain the target background image.

[0072] In this embodiment, by replacing the pixel values in the reference background image corresponding to the position information of the non-interested regions according to the pixel values of the non-interested regions, the update of the reference background image is realized.

[0073] In some of these embodiments, the reference background image includes: a historical scattered object region; updating the reference background image based on the non-interested regions of the image frame to be processed to obtain the target background image includes: updating the region in the reference background image except the historical scattered object region based on the non-interested regions of the image frame to be processed to obtain the target background image.

[0074] In this embodiment, by updating the region in the reference background image except the historical scattered object region based on the non-interested regions of the image frame to be processed to obtain the target background image, the influence of historical scattered objects on the determination of scattered objects in the image frame to be processed can be avoided, thereby further improving the accuracy of scattered objects in the image frame to be processed.

[0075] In some of these embodiments, determining whether the object region to be processed contains scattered objects based on the feature similarity between the target object region and the object region to be processed includes: inputting the image corresponding to the object region to be processed into the trained feature extraction model to obtain the feature of the object to be processed in the object region to be processed, and inputting the image corresponding to the target object region into the trained feature extraction model to obtain the feature of the target object in the target object region; determining the feature similarity between the feature of the target object and the feature of the object to be processed; and determining that the object region to be processed contains scattered objects when the feature similarity is less than the preset feature similarity.

[0076] In this embodiment, by using the trained feature model to obtain the features of the object to be processed in the object region to be processed and the features of the target object in the target object region, the accuracy of the object features can be improved, so as to improve the detection rate of scattered objects subsequently; and the feature similarity between the target object features and the object features to be processed is determined. Then, in the case where the feature similarity is less than the preset feature similarity, the method of determining that the object region to be processed contains scattered objects realizes the determination of the detection of scattered objects.

[0077] In some of these embodiments, in the case where the feature similarity is less than the preset feature similarity, determining that the object region to be processed contains scattered objects includes: determining the cosine distance between the object features to be processed and the target object features; in the case where the cosine distance is greater than the preset distance, it is determined that the feature similarity is less than the preset feature similarity, and it is determined that the object region to be processed contains scattered objects.

[0078] In this embodiment, by calculating the cosine distance of the object features of two objects, and in the case where the cosine distance is greater than the preset distance, it is determined that the feature similarity is less than the preset feature similarity, and the method of determining that the object region to be processed contains scattered objects realizes the determination of whether the object region to be processed contains scattered objects.

[0079] The method for determining the similarity of two objects based on the cosine distance of the object features of the two objects is described and illustrated through the following steps:

[0080] Step 1, use the deep learning object classification model (equivalent to the trained and complete feature extraction model in the above embodiment),

[0081] It should be noted that the deep learning object classification model is obtained by training an open-source ImageNet data to obtain a classifier, and the backbone model of the classifier is used as a feature extraction tool, hereinafter simply referred to as a feature extractor.

[0082] Step 2, based on the coordinates of the object box Box1 in the object region to be processed, extract the image of the object region to be processed of the object in the image frame to be processed, and send the image of the object region to be processed into the feature extractor to generate feature F1.

[0083] Step 3, based on the object box Box1 in the object region to be processed, extract the image of the corresponding target object region of the background image, and send the image of the target object region into the feature extractor to generate feature F2.

[0084] Step 4, calculate the cosine distance between F1 and F2, Dis12 = 1 - cosine(F1, F2), and take dis1 as the feature threshold (equivalent to the preset distance in the above embodiment). If Dis12 >= dis1, it is determined that the similarity between the feature of the object to be processed in the object area to be processed and the feature of the target object in the target object area is less than the preset similarity, and the object in the object area to be processed is judged as a scattered object. If Dis12 < dis1, it is determined that the similarity between the feature of the object to be processed in the object area to be processed and the feature of the target object in the target object area is greater than the preset similarity, and the object in the object area to be processed is determined as an object in a non - interested area.

[0085] It should be noted that in this embodiment, the scattered object is associated with five pieces of information: the position box of the scattered object, the existence time, the loss time, the image feature, and the alarm state. Among them, the position box is set as Box corresponding to Box, the state time ST = 1 frame, the loss time LT = 0 frame, the image feature F = F1, and the alarm state AS = 0, as shown in Table 1.

[0086] Table 1 Scattered Object Information Table

[0087] Object Position Box Survival Time ST Loss Time LT Image Feature F Alarm Status AS Scattered Objects Box1 1 0 F1 0

[0088] In some of these embodiments, after determining that the object area to be processed contains scattered objects when the feature similarity is less than the preset feature similarity, historical scattered object information can also be obtained. The historical scattered object information includes: scattered object feature information and scattered object position information; according to the historical scattered object position information, detect whether there is a target scattered object whose position information is the same as that of the scattered object in the object area to be processed; when it is detected that there is a target scattered object whose position information is the same as that of the scattered object in the object area to be processed, detect whether the feature information of the scattered object in the object area to be processed is the same as the target scattered object feature information; when it is detected that the feature information of the scattered object in the object area to be processed is the same as the target scattered object feature information, merge the scattered object in the object area to be processed into the target scattered object information and delete the scattered object information in the object area to be processed.

[0089] In this embodiment, by comparing the historical scattered object information with the scattered object information of the frame to be processed, and when it is detected that the feature information of the scattered object in the frame of the image to be processed is the same as the target scattered object feature information, merging the scattered object in the frame of the image to be processed into the target scattered object information and deleting the scattered object information in the frame of the image to be processed, the merging of the scattered object in the frame of the image to be processed and the historical scattered object is realized, and accurate reporting of the scattered object in the subsequent frame to be processed can be achieved, thereby improving the detection accuracy of the scattered object in the frame to be processed.

[0090] In some of these embodiments, in the case where it is detected that there is no target spill consistent with the position information of the spill in the image frame to be processed, or, in the case where the feature information of the spill in the image frame to be processed is different from the feature information of the target spill, the spill in the image frame to be processed can also be used as a new spill, and the information of the new spill can be obtained and saved into the historical spill information.

[0091] In this embodiment, by using the spill in the image frame to be processed as a new spill and obtaining the information of the new spill and saving it into the historical spill information, it is convenient to uniformly report the spills in the target shooting scene and convenient for the staff to track and view the spills subsequently.

[0092] In some of these embodiments, the historical spill information further includes: the spill loss time. In the case where the feature information of the spill in the image frame to be processed is the same as the feature information of the target spill, after merging the spill in the image frame to be processed into the target spill information and deleting the spill information of the image frame to be processed, it can also be determined according to the historical spill information whether there is a first spill whose spill loss time is greater than a first preset time; in the case where it is determined that there is a first spill whose spill loss time is greater than the first preset time, the information corresponding to the first spill is deleted.

[0093] In this embodiment, by deleting the first spill whose spill loss time is greater than the first preset time, it is possible to avoid the influence of the lost objects in the shooting scene on the reported spill information and further improve the accuracy of spill detection.

[0094] In some of these embodiments, the historical spill information further includes: the spill stay time. In the case where the feature information of the spill in the image frame to be processed is the same as the feature information of the target spill, after merging the spill in the image frame to be processed into the target spill information and deleting the spill information of the image frame to be processed, it can also be determined according to the historical spill information whether there is a second spill whose spill stay time is greater than a second preset time; in the case where it is determined that there is a second spill whose spill stay time is greater than the second preset time, the information corresponding to the second spill is reported.

[0095] In this embodiment, by reporting the information corresponding to the second spill in the case where it is determined that there is a second spill whose spill stay time is greater than the second preset time, it is possible to inform the relevant staff to process the spill, avoid causing traffic safety accidents, and improve traffic safety.

[0096] The following describes and illustrates this embodiment through preferred embodiments.

[0097] Figure 4 This is the preferred process of the vehicle litter detection method of this embodiment. Figure 1 ,like Figure 4 As shown, the vehicle scattered object detection method includes the following steps:

[0098] Step S401, obtaining a reference background image and an image frame to be processed of a target shooting scene.

[0099] Step S402, determining a non-interest region and an interest region of the image frame to be processed.

[0100] In this step, the non-interest region and the interest region can be obtained from the image by using a deep learning target detection algorithm, wherein the non-interest region includes other regions in the image frame to be processed except the vehicle region to be processed and the object region to be processed.

[0101] Step S403, obtaining historical scattered object information.

[0102] In this step, the historical scattered object information may include the scattered object information that existed in the image frame under the target shooting scene, and the historical scattered object information may include: scattered object feature information, scattered object location information, scattered object loss time, scattered object residence time, and scattered object area.

[0103] Step S404: based on the non-interested area of the image frame to be processed, the area other than the historical scattered object area in the reference background image is updated to obtain a target background image.

[0104] In this step, based on step S404, the following steps may be included:

[0105] Step 41, determining a historical scattered object area according to historical scattered object information, and setting the historical scattered object area as a first ignored area.

[0106] Step 42: Set the non-interested region as the second ignored region.

[0107] Step 43, based on the first ignored area and the second ignored area, determine the non-ignored area in the image frame to be processed, and use the RGB pixel values of the non-ignored area of the image frame to be processed to replace the RGB pixel values of the area corresponding to the position information of the non-ignored area of the reference background image, wherein the area in the reference background image corresponding to the position information of the first ignored area and the second ignored area still retains the original RGB pixel values of the reference background image, thereby realizing the update of the background image.

[0108] Step S405: Determine the feature similarity between the target object features and the object features to be processed. If the feature similarity is less than the preset feature similarity, execute Step S406; if the feature similarity is greater than the preset feature similarity, end.

[0109] Step S406: When the feature similarity is less than the preset feature similarity, determine the object in the object area to be processed as the scattered object of the image frame to be processed.

[0110] Step S407: According to the historical scattered object position information, detect whether there is a target scattered object whose position information is consistent with that of the scattered object of the image frame to be processed. If a target scattered object whose position information is consistent with that of the scattered object of the image frame to be processed is detected, execute Step S408; if no target scattered object whose position information is consistent with that of the scattered object of the image frame to be processed is detected, end.

[0111] Step S408: Detect whether the feature information of the scattered object of the image frame to be processed is the same as the target scattered object feature information. If the feature information of the scattered object of the image frame to be processed is the same as the target scattered object feature information, execute Step S409; if the feature information of the scattered object of the image frame to be processed is different from the target scattered object feature information, execute Step 410.

[0112] Step S409: Merge the scattered object of the image frame to be processed into the target scattered object information and delete the scattered object information of the image frame to be processed.

[0113] It should be noted that when deleting the scattered object information of the image frame to be processed, the information corresponding to the historical scattered object corresponding to the scattered object information of the image frame to be processed in the target scattered object information will be updated, such as the scattered object position information, existence time, loss time, and object features.

[0114] Step S410: Take the scattered object of the image frame to be processed as a new scattered object and obtain the information of the new scattered object and save it into the historical scattered object information.

[0115] As Figure 5 shown, after Step S409 and Step S410, the following steps may further be included:

[0116] Step S411: According to the historical scattered object information in Step S409 or Step S410, determine whether there is a first scattered object whose loss time is greater than the first preset time. If it is determined that there is a first scattered object whose loss time is greater than the first preset time, execute Step S412; if it is determined that there is a first scattered object whose loss time is less than the first preset time, end.

[0117] Step S412: Delete the information corresponding to the first scattered object.

[0118] As Figure 6 shown, after steps S409 and S410, the following steps may further be included:

[0119] Step S413: According to the historical scattered object information in step S409 or step S410, determine whether there is a second scattered object whose residence time is greater than a second preset time. In the case where it is determined that there is a second scattered object whose residence time is greater than the second preset time, execute step S414. In the case where it is determined that there is a second scattered object whose residence time is less than the second preset time, end.

[0120] Step S414: Report the information corresponding to the second scattered object.

[0121] It should be noted that steps S411 and S413 can be carried out simultaneously after step S410 or after step S409 without a specific order.

[0122] In this embodiment, a vehicle scattered object detection device is further provided. This device is used to implement the above-mentioned embodiment and preferred implementation manners, and those that have been described will not be repeated here. The following terms such as "module", "unit", "sub-unit", etc. can be a combination of software and / or hardware that can achieve a predetermined function. Although the devices described in the following embodiments are preferably implemented in software, implementation in hardware, or a combination of software and hardware is also possible and contemplated.

[0123] Figure 7 is the structural block diagram of the vehicle scattered object detection device in this embodiment. As Figure 7 shown, this device includes:

[0124] A first acquisition module 71, configured to acquire a reference background image and a to-be-processed image frame of a target shooting scene;

[0125] An update module 72, coupled to the first acquisition module 71, configured to update the reference background image based on the non-interested region of the to-be-processed image frame to obtain a target background image, where the non-interested region includes other regions in the to-be-processed image frame except the to-be-processed vehicle region and the to-be-processed object region;

[0126] A first determination module 73, coupled to the update module 72, configured to determine a target object region in the target background image that has the same position information as the to-be-processed object region;

[0127] A second determination module 74, coupled to the first determination module 73, configured to determine whether the to-be-processed object region contains a scattered object based on the feature similarity between the target object region and the to-be-processed object region.

[0128] In some of these embodiments, the update module 72 includes: a first determination unit configured to determine pixel values of a non-interested region; and a replacement unit configured to replace pixel values in the reference background map corresponding to the position information of the non-interested region with the pixel values of the non-interested region to obtain a target background map.

[0129] In some of these embodiments, the reference background map includes: a historical scattering region; and the update module 72 includes: an update unit configured to update a region in the reference background map other than the historical scattering region based on the non-interested region of the image frame to be processed to obtain a target background map.

[0130] In some of these embodiments, the second determination module 74 includes: an input unit configured to input an image corresponding to the object region to be processed into a trained feature extraction model to obtain a feature of the object to be processed in the object region to be processed, and input an image corresponding to the target object region into the trained feature extraction model to obtain a feature of the target object in the target object region; a second determination unit configured to determine a feature similarity between the feature of the target object and the feature of the object to be processed; and a processing unit configured to determine that the object region to be processed contains scattered objects when the feature similarity is less than a preset feature similarity.

[0131] In some of these embodiments, the processing unit includes: a determination subunit configured to determine a cosine distance between the feature of the object to be processed and the feature of the target object; and a processing subunit configured to determine that the feature similarity is less than the preset feature similarity and determine that the object region to be processed contains scattered objects when the cosine distance is greater than a preset distance.

[0132] In some of these embodiments, the apparatus further includes: a second acquisition module configured to acquire historical scattering information, where the historical scattering information includes: scattering feature information and scattering position information; a first detection module configured to detect whether there is a target scattered object whose position information is consistent with the position information of the scattered object in the object region to be processed according to the historical scattering position information; a second detection module configured to detect whether the feature information of the scattered object in the object region to be processed is the same as the feature information of the target scattered object when it is detected that there is a target scattered object whose position information is consistent with the position information of the scattered object in the object region to be processed; and a merging module configured to merge the scattered object in the object region to be processed into the target scattered object information and delete the scattered object information of the object region to be processed when it is detected that the feature information of the scattered object in the object region to be processed is the same as the feature information of the target scattered object.

[0133] In some of these embodiments, the device further includes: a storage module, configured to, when it is detected that there is no target spill consistent with the position information of the spill in the area of the object to be processed, or when it is detected that the feature information of the spill in the area of the object to be processed is different from the feature information of the target spill, use the spill in the area of the object to be processed as a new spill, and obtain the information of the new spill and save it to the historical spill information.

[0134] In some of these embodiments, the historical spill information further includes: the spill loss time, and the device further includes: a first judgment module, configured to judge, according to the historical spill information, whether there is a first spill whose spill loss time is greater than a first preset time; a deletion module, configured to, when it is judged that there is a first spill whose spill loss time is greater than a first preset time, delete the information corresponding to the first spill.

[0135] In some of these embodiments, the historical spill information further includes: the spill residence time, and the device further includes: a second judgment module, configured to judge, according to the historical spill information, whether there is a second spill whose spill residence time is greater than a second preset time; a reporting module, configured to, when it is judged that there is a second spill whose spill residence time is greater than a second preset time, report the information corresponding to the second spill.

[0136] It should be noted that the above-mentioned respective modules can be functional modules or program modules, and can be implemented either by software or by hardware. For the modules implemented by hardware, the above-mentioned respective modules can be located in the same processor; or the above-mentioned respective modules can also be located in different processors in any combined form.

[0137] In this embodiment, an electronic device is further provided, including a memory and a processor, where a computer program is stored in the memory, and the processor is configured to run the computer program to execute the steps in any one of the above method embodiments.

[0138] Optionally, the above-mentioned electronic device may further include a transmission device and an input / output device, where the transmission device is connected to the above-mentioned processor, and the input / output device is connected to the above-mentioned processor.

[0139] Optionally, in this embodiment, the above-mentioned processor may be configured to execute the following steps through a computer program:

[0140] Step S1, obtain a reference background image and an image frame to be processed of a target shooting scene;

[0141] Step S2, update the reference background image based on the non - region of interest of the image frame to be processed to obtain a target background image, where the non - region of interest includes other regions in the image frame to be processed except the vehicle region to be processed and the object region to be processed;

[0142] Step S3, determine the target object region in the target background image that has the same position information as the object region to be processed;

[0143] Step S4, based on the feature similarity between the target object region and the object region to be processed, determine whether the object region to be processed contains spilled objects.

[0144] It should be noted that the specific examples in this embodiment can refer to the examples described in the above - mentioned embodiments and optional implementation manners, and will not be elaborated in this embodiment.

[0145] In addition, in combination with the vehicle spill detection method provided in the above - mentioned embodiments, a storage medium can also be provided in this embodiment to implement it. A computer program is stored on the storage medium; when the computer program is executed by a processor, any one of the vehicle spill detection methods in the above - mentioned embodiments is implemented.

[0146] It should be understood that the specific embodiments described here are only used to explain this application, rather than to limit it. According to the embodiments provided in the present application, all other embodiments obtained by those of ordinary skill in the art without creative work fall within the protection scope of the present application.

[0147] Obviously, the drawings are only some examples or embodiments of the present application. For those of ordinary skill in the art, the present application can also be applied to other similar situations based on these drawings without creative work. In addition, it can be understood that although the work done during the development process may be complex and time - consuming, for those of ordinary skill in the art, certain design, manufacturing, or production changes based on the technical content disclosed in the present application are only conventional technical means and should not be regarded as insufficient disclosure of the present application.

[0148] The term "embodiment" in the present application means that the specific features, structures, or characteristics described in combination with the embodiment can be included in at least one embodiment of the present application. The phrase appears in various positions in the specification and does not necessarily mean the same embodiment, nor does it mean being independent or alternative to other embodiments and mutually exclusive. Those of ordinary skill in the art can clearly or implicitly understand that the embodiments described in the present application can be combined with other embodiments without conflict.

[0149] The above-described embodiments merely represent several implementation manners of the present application. The description thereof is relatively specific and detailed, but it should not be construed as a limitation on the scope of patent protection. It should be noted that for those of ordinary skill in the art, without departing from the concept of the present application, several modifications and improvements can still be made, and these all fall within the protection scope of the present application. Therefore, the protection scope of the present application shall be subject to the appended claims.

Claims

1. A vehicle throwing object detection method, characterized in that, Including: Obtain a reference background image and a to-be-processed image frame of a target shooting scene; Update the reference background image based on the non-interested region of the to-be-processed image frame to obtain a target background image, where the non-interested region includes other regions in the to-be-processed image frame except the to-be-processed vehicle region and the to-be-processed object region; Determine a target object region in the target background image that has the same position information as the to-be-processed object region; Based on the feature similarity between the target object region and the to-be-processed object region, determine whether the to-be-processed object region contains scattered objects, including: Input the image corresponding to the to-be-processed object region into the trained feature extraction model to obtain the to-be-processed object features in the to-be-processed object region, and input the image corresponding to the target object region into the trained feature extraction model to obtain the target object features in the target object region; Determine the feature similarity between the target object features and the to-be-processed object features; In the case where the feature similarity is less than a preset feature similarity, determine that the to-be-processed object region contains scattered objects.

2. The vehicle debris detection method according to claim 1, wherein Updating the reference background image based on the non-interested region of the to-be-processed image frame to obtain a target background image includes: Determine the pixel values of the non-interested region; Replace the pixel values in the reference background image corresponding to the position information of the non-interested region with the pixel values of the non-interested region to obtain the target background image.

3. The vehicle debris detection method according to claim 1, wherein The reference background image includes: a historical scattered object region; updating the reference background image based on the non-interested region of the to-be-processed image frame to obtain a target background image includes: Update the region in the reference background image except the historical scattered object region based on the non-interested region of the to-be-processed image frame to obtain the target background image.

4. The vehicle throw detection method according to claim 1, characterized in that In the case where the feature similarity is less than a preset feature similarity, determining that the to-be-processed object region contains scattered objects includes: Determine the cosine distance between the to-be-processed object features and the target object features; In the case where the cosine distance is greater than a preset distance, determine that the feature similarity is less than the preset feature similarity, and determine that the to-be-processed object region contains scattered objects.

5. The vehicle throw detection method according to claim 1, characterized in that, After determining that the to-be-processed object region contains scattered objects in the case where the feature similarity is less than a preset feature similarity, the method further includes: Obtain historical scattered object information, where the historical scattered object information includes: scattered object feature information and scattered object position information; According to the historical scattered object position information, detect whether there is a target scattered object whose position information is consistent with the position information of the scattered object in the to-be-processed object region; In the case where a target scattered object whose position information is consistent with the position information of the scattered object in the to-be-processed object region is detected, detect whether the feature information of the scattered object in the to-be-processed object region is the same as the target scattered object feature information; When it is detected that the characteristic information of the scattered objects in the area of the object to be processed is the same as the characteristic information of the target scattered objects, merge the scattered objects in the area of the object to be processed into the target scattered object information, and delete the scattered object information in the area of the object to be processed.

6. The vehicle throwing object detection method according to claim 5, wherein When it is detected that there is no target scattered object whose position information is consistent with the scattered objects in the area of the object to be processed, or when it is detected that the characteristic information of the scattered objects in the area of the object to be processed is different from the characteristic information of the target scattered objects, the method further includes: Take the scattered objects in the area of the object to be processed as new scattered objects, and obtain the information of the new scattered objects and save it into the historical scattered object information.

7. The vehicle debris detection method according to claim 6, wherein The historical scattered object information further includes: the scattered object loss time. When it is detected that the characteristic information of the scattered objects in the area of the object to be processed is the same as the characteristic information of the target scattered objects, after merging the scattered objects in the area of the object to be processed into the target scattered object information and deleting the scattered object information in the area of the object to be processed, the method further includes: According to the historical scattered object information, determine whether there is a first scattered object whose scattered object loss time is greater than a first preset time; When it is determined that there is a first scattered object whose scattered object loss time is greater than the first preset time, delete the information corresponding to the first scattered object.

8. The vehicle throwing object detection method according to claim 6, wherein, The historical scattered object information further includes: the scattered object stay time. When it is detected that the characteristic information of the scattered objects in the area of the object to be processed is the same as the characteristic information of the target scattered objects, after merging the scattered objects in the area of the object to be processed into the target scattered object information and deleting the scattered object information in the area of the object to be processed, the method further includes: According to the historical scattered object information, determine whether there is a second scattered object whose scattered object stay time is greater than a second preset time; When it is determined that there is a second scattered object whose scattered object stay time is greater than the second preset time, report the information corresponding to the second scattered object.

9. A vehicle throwable detection device, characterized in that, Including: A first acquisition module, configured to acquire a reference background image and a to-be-processed image frame of a target shooting scene; An update module, configured to update the reference background image based on the non-interested area of the to-be-processed image frame to obtain a target background image, where the non-interested area includes other areas in the to-be-processed image frame except the to-be-processed vehicle area and the to-be-processed object area; A first determination module, configured to determine a target object area in the target background image that has the same position information as the to-be-processed object area; A second determination module, configured to determine whether the to-be-processed object area contains scattered objects based on the feature similarity between the target object area and the to-be-processed object area, including: Input the image corresponding to the to-be-processed object area into the trained feature extraction model to obtain the to-be-processed object features in the to-be-processed object area, and input the image corresponding to the target object area into the trained feature extraction model to obtain the target object features in the target object area; Determine the feature similarity between the target object features and the object features to be processed; In the case where the feature similarity is less than a preset feature similarity, determine that the object area to be processed contains scattered objects.

10. An electronic device, comprising a memory and a processor, characterized in that, A computer program is stored in the memory, and the processor is configured to run the computer program to execute the vehicle scattered object detection method according to any one of claims 1 to 8.

11. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by the processor, the steps of the vehicle scattered object detection method according to any one of claims 1 to 8 are implemented.

Images