Method, device and equipment for identifying road debris

Abstract

The application discloses a road litter identification method, device and equipment, and is applied to the technical field of identification, and comprises the following steps: acquiring a first group of scene pictures and a second group of scene pictures of a preset road scene, wherein the first group of scene pictures and the second group of scene pictures both comprise a first scene picture, and the shooting time of the first scene picture is earlier than that of other scene pictures in the first group of scene pictures and the second group of scene pictures; processing the first group of scene pictures and the second group of scene pictures by a pre-trained litter identification model to obtain a first identification result picture and a second identification result picture representing picture changes; and determining whether there is litter in the preset road scene based on the position relationship between the change area in the first identification result picture and the change area in the second identification result picture. The application solves the technical problem of incorrect detection of litter.

Description

TECHNICAL FIELD

[0001] The present application belongs to the technical field of intelligent transportation, and particularly relates to a road litter identification method, device and equipment. BACKGROUND

[0002] During driving, passengers often throw waste from the window, which is of various types, such as beverage bottles, fruit peels, pop cans, lighters, and so on. In addition, truck owners do not take effective protective measures, resulting in scattered goods, mainly building materials and iron materials. These road litter constantly endangers the driving safety of vehicles on the road and easily causes traffic accidents.

[0003] The prior art identifies road litter through a target detection model. However, due to the various types of objects on the road and the difficulty of data collection, the current target detection model is highly dependent on the number and quality of data samples, resulting in incorrect and missed detection of road litter by the target detection model trained by samples, and low confidence of the detection result. SUMMARY

[0004] The embodiments of the present application provide a road litter identification method, device and equipment, which solve the technical problem of low confidence of the detection result.

[0005] In a first aspect, the embodiments of the present application provide a road litter identification method, comprising: obtaining a first group of scene pictures and a second group of scene pictures of a preset road scene, wherein the first group of scene pictures and the second group of scene pictures each include a first scene picture, and the shooting time of the first scene picture is earlier than that of other scene pictures in the first group of scene pictures and the second group of scene pictures; processing the first group of scene pictures and the second group of scene pictures by a pre-trained litter identification model to obtain a first identification result picture and a second identification result picture representing picture changes; and determining whether there is litter in the preset road scene based on the positional relationship between the changed area in the first identification result picture and the changed area in the second identification result picture.

[0006] In combination with the first aspect of the present application, in some embodiments, the determination of whether there is litter in the preset road scene based on the positional relationship between the changed area in the first identification result picture and the changed area in the second identification result picture comprises: if there is an overlapping area between the changed area in the first identification result picture and the changed area in the second identification result picture, it is determined that there is static litter in the preset road scene; and if there is a non-overlapping area between the changed area in the first identification result picture and the changed area in the second identification result picture, it is determined whether there is moving litter in the preset road scene based on the non-overlapping area.

[0007] In combination with the first aspect of the present application, in some embodiments, the method further comprises a step of pre-training the original difference discrimination model, which comprises: obtaining a sample picture set, the sample picture set comprising a plurality of groups of sample pictures, each group of sample pictures comprising a first sample picture of a same road scene in a non-debris state, a second sample picture in a debris state, and a mask picture of the second sample picture; training the original difference discrimination model based on the sample picture set to obtain the debris recognition model.

[0008] In combination with the first aspect of the present application, in some embodiments, before determining that the overlapping region contains static debris, the method comprises: comparing pixel values of the first recognition result picture and the second recognition result picture; and determining whether the changed region in the first recognition result picture and the changed region in the second recognition result picture have an overlapping region based on the comparison result.

[0009] In combination with the first aspect of the present application, in some embodiments, after determining that the overlapping region contains static debris, the method comprises: adding a bounding rectangle frame to the overlapping region; obtaining a center point coordinate of the bounding rectangle frame; and determining the position information of the debris according to the center point coordinate.

[0010] In combination with the first aspect of the present application, in some embodiments, if there is no overlapping region between the changed region in the first recognition result picture and the changed region in the second recognition result picture, determining whether the preset road scene contains moving debris based on the non-overlapping region comprises: if there are M target changed regions without overlapping regions between the changed region in the first recognition result picture and the changed region in the second recognition result picture, M being a positive integer; taking each target changed region belonging to the first recognition result picture in the M target changed regions as a current target changed region, respectively, determining a distance between the current target changed region and each target changed region belonging to the second recognition result picture in the M target changed regions to obtain at least one distance parameter; determining at least one group of to-be-judged regions according to the at least one distance parameter; and for each group of to-be-judged regions, if a graphic difference between the two target changed regions corresponding to the group of to-be-judged regions is less than a preset difference threshold, determining that the preset road scene corresponding to the to-be-judged region of the second recognition result picture contains moving debris.

[0011] With reference to the first aspect of the present application, in some embodiments, after determining that the to-be-judged region of the second identification result picture corresponds to a preset road scene with moving litter, the method further comprises: obtaining a third set of scene pictures of the preset road scene, the third set of scene pictures comprising a first scene picture and a second scene picture; processing the third set of scene pictures by using the litter identification model to obtain a third identification result picture representing picture changes; re-determining whether the preset road scene has moving litter based on the first identification result picture and the third identification result picture, and outputting a determination result that the preset road scene has moving litter if the preset road scene has moving litter.

[0012] With reference to the first aspect of the present application, in some embodiments, determining at least one set of to-be-judged regions according to the at least one distance parameter comprises: determining a set of to-be-judged regions corresponding to a minimum distance parameter from the at least one distance parameter, or determining each to-be-judged region smaller than a preset distance threshold from the at least one distance parameter.

[0013] In a second aspect, an embodiment of the present application provides a road litter identification device, comprising: a scene picture acquisition unit configured to acquire a first set of scene pictures and a second set of scene pictures of a preset road scene, wherein the first set of scene pictures and the second set of scene pictures both comprise a first scene picture, and the first scene picture is taken earlier than other scene pictures in the first set of scene pictures and the second set of scene pictures; a result picture acquisition unit configured to process the first set of scene pictures and the second set of scene pictures by using a pre-trained litter identification model to obtain a first identification result picture and a second identification result picture representing picture changes; and a determination unit configured to determine whether the preset road scene has litter based on a positional relationship between a changed region in the first identification result picture and a changed region in the second identification result picture.

[0014] In a third aspect, an embodiment of the present application provides an electronic device, comprising a memory, a processor, and a computer program stored in the memory and capable of running on the processor, wherein the processor implements the method according to any one of claims 1-8 when executing the computer program.

[0015] The one or more technical solutions provided by the embodiments of the present application at least achieve the following technical effects or advantages:

[0016] The road litter identification method provided by the embodiment of the present application comprises the following steps: acquiring a first group of scene pictures and a second group of scene pictures of a preset road scene, wherein the first group of scene pictures and the second group of scene pictures each comprise a first scene picture, and the shooting time of the first scene picture is earlier than that of other scene pictures in the first group of scene pictures and the second group of scene pictures; processing the first group of scene pictures and the second group of scene pictures by using a pre-trained litter identification model to obtain a first identification result picture and a second identification result picture representing picture changes; and determining whether there is litter in the preset road scene based on the positional relationship between the change area in the first identification result picture and the change area in the second identification result picture. In the above technical solution, the litter identification model can distinguish the change area of each of the two scene pictures, and the first group of scene pictures and the second group of scene pictures are combined to determine whether there is litter. By optimizing the litter identification algorithm on the application layer, the dependence of the litter identification model on the training sample pictures is reduced, and the detection problem caused by insufficient sample pictures is avoided, thereby solving the technical problems of litter mis-detection and omission, and improving the confidence of the detection result. BRIEF DESCRIPTION OF DRAWINGS

[0017] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the drawings needed in the embodiment description will be briefly introduced. Obviously, the drawings in the following description are some embodiments of the present application, and other drawings can be obtained by those skilled in the art without creative labor.

[0018] Figure 1 The flow chart of the road litter identification method in the embodiment of the present application;

[0019] Figure 2 The specific schematic diagram of the scene picture of the preset road scene with static litter in the embodiment of the present application;

[0020] Figure 3 The specific schematic diagram of the group of sample pictures for training the litter identification model in the embodiment of the present application;

[0021] Figure 4 The specific schematic diagram of the scene picture of the preset road scene with static litter and moving litter in the embodiment of the present application;

[0022] Figure 5 The specific schematic diagram of the scene picture of the preset road scene with static litter and window object in the embodiment of the present application;

[0023] Figure 6 The function module diagram of the road litter identification device in the embodiment of the present application;

[0024] Figure 7 Fig. 1 is a schematic diagram of an electronic device according to an embodiment of the present application. DETAILED DESCRIPTION

[0025] The technical solutions in the embodiments of the present application will be clearly and completely described below with reference to the drawings in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, rather than all the embodiments of the present application. Based on the embodiments in the present application, all other embodiments obtained by a person of ordinary skill in the art without creative work fall within the protection scope of the present application.

[0026] In the present application, the description such as “first”, “second” and the like is only for the purpose of description, and cannot be understood as indicating or implying the relative importance of the indicated technical features or implicitly indicating the number of the indicated technical features. Therefore, the features defined with “first”, “second” can explicitly or implicitly include at least one of the features. In addition, the technical solutions of various embodiments can be combined with each other, but it must be based on the fact that a person of ordinary skill in the art can realize it. When the combination of technical solutions appears contradictory or unachievable, it should be considered that the combination of technical solutions does not exist, and is not within the protection scope required by the present application.

[0027] The embodiment of the present application provides a road litter identification method, applied to an electronic device capable of data interaction with a sensing device. The electronic device can be an RSU (Road Side Unit). The RSU is a device installed on the roadside in an ETC system, which uses DSRC (Dedicated Short Range Communication) technology to communicate with an OBU (On Board Unit) to realize functions such as road litter identification.

[0028] Reference Figure 1 As shown in the figure, the road litter identification method comprises the following steps:

[0029] S101: Obtain a first group of scene pictures and a second group of scene pictures of a preset road scene. The first group of scene pictures and the second group of scene pictures each include a first scene picture, and the shooting time of the first scene picture is earlier than that of other scene pictures in the first group of scene pictures and the second group of scene pictures.

[0030] It should be noted that three frames of pictures taken at different times of a preset road scene are obtained. The first group of scene pictures can refer to the first scene picture and the next frame of scene picture adjacent to the first scene picture. The second group of scene pictures can refer to the first scene picture and a frame of scene picture N frames apart after the first scene picture. N is a positive integer.

[0031] Specifically, taking the shooting interval time between adjacent scene pictures as thirty-one seconds, the first scene picture as the first frame, another scene picture in the first group of scene pictures as the second frame, and another scene picture in the second group of scene pictures as the third frame, the fourth frame, or the fifth frame as examples.

[0032] S102: The first group of scene pictures and the second group of scene pictures are respectively processed by the pre-trained litter identification model, and the first identification result picture and the second identification result picture representing picture changes are obtained.

[0033] Reference Figure 2 As shown, the first group of scene pictures is processed by the litter identification model to identify the change area between the two frames of scene pictures in the first group of scene pictures, and the first identification result picture is obtained. The second group of scene pictures is processed by the litter identification model to identify the change area between the two frames of scene pictures in the second group of scene pictures, and the second identification result picture is obtained. Specifically, the first identification result picture and the second identification result picture are both binary pictures, and both represent the litter changes between the two frames of pictures.

[0034] The litter identification model can be obtained by training the original difference discrimination model, where the original difference discrimination model can be SNUnet (Siam-Nested UNet, change detection network), or a similar discrimination model can be used instead.

[0035] Specifically, a sample picture set is obtained, where the sample picture set includes multiple groups of sample pictures, and referring to Figure 3 As shown, each group of sample pictures includes a first sample picture of a road scene in a litter-free state, a second sample picture with litter, and a mask picture of the second sample picture; the original difference discrimination model is trained based on the sample picture set to obtain the litter identification model.

[0036] A historical scene picture set of multiple road scenes is obtained; each frame of historical scene picture in the historical scene picture set is classified according to whether there is litter, and two types of scene pictures are obtained, where the first type of sample picture refers to each first sample picture in a litter-free state; the second type of sample picture refers to each second sample picture in a litter state.

[0037] It can be understood that the generation process of the mask picture of each second sample picture includes: labeling the litter area of each second sample picture; and performing binary processing on each second sample picture after labeling the litter area to generate the mask picture of the second sample picture.

[0038] Specifically, the labeling of the litter area can be manual labeling, specifically, the litter area can be circled or a dot is drawn on the litter area, so as to accurately distinguish the litter from the pedestrians and vehicles, avoid the litter recognition model trained to identify the vehicles and pedestrians as litter, and improve the reliability of the litter recognition model.

[0039] In the specific implementation process, the scene pictures with litter are generally less, resulting in insufficient number of negative sample pictures, a plurality of first sample pictures can be randomly selected from the first sample pictures, and litter can be added to each selected first sample picture by retouching to obtain the second sample picture corresponding to the first sample picture in a certain road scene, so as to solve the problem of insufficient number of negative sample pictures.

[0040] S103: determining whether there is litter in the preset road scene based on the position relationship between the changed area in the first recognition result picture and the changed area in the second recognition result picture.

[0041] In the embodiment of the application, the changed area in the first recognition result picture and the second recognition result picture is an area where litter may exist, as shown in Figure 2 It can be seen that the area a and the area b in the first recognition result picture are changed areas; the area c and the area d in the second recognition result picture are changed areas.

[0042] It should be noted that in S103, based on the position relationship between the changed area in the first recognition result picture and the changed area in the second recognition result picture, it is determined whether there is litter in the preset road scene, including the following two cases A1 and A2:

[0043] A1: if there is an overlapping area between the changed area in the first recognition result picture and the changed area in the second recognition result picture, it is determined that there is static litter in the preset road scene.

[0044] In A1, the first recognition result picture and the second recognition result picture are compared in pixel value; and it is determined whether the changed area in the first recognition result picture and the changed area in the second recognition result picture have an overlapping area based on the pixel value comparison result.

[0045] It can be understood that there are many implementation manners for determining whether the changed area in the first recognition result picture and the changed area in the second recognition result picture have an overlapping area based on the pixel value comparison result, and two examples are given as follows:

[0046] Implementation manner one:

[0047] Taking the white pixel value being 255 and the black pixel value being 0 in the first recognition result picture and the second recognition result picture as an example, the pixel value of each pixel point in the first recognition result picture is obtained, and the pixel value of each pixel point in the second recognition result picture is obtained. Whether the pixel values of the pixel points corresponding to the same position in the first recognition result picture and the second recognition result picture are both 255 is compared. After the comparison of the pixel values of each pixel point in the first recognition result picture and the second recognition result picture is completed, the pixel point set formed by each pixel point whose pixel value in the first recognition result picture and the second recognition result picture is 255 is taken as the overlapping area. Figure 2 For example, the area a of the first recognition result picture and the area d of the second recognition result picture are the overlapping areas in the changed area.

[0048] Embodiment II:

[0049] The changed area in the first recognition result picture is obtained, and a bounding rectangle is added to each changed area in the first recognition result picture. The changed area in the second recognition result picture is obtained, and a bounding rectangle is added to each changed area in the second recognition result picture. Each bounding rectangle in the first recognition result picture is compared with each bounding rectangle in the second recognition result picture in turn. If a group of bounding rectangles formed by any one bounding rectangle in the first recognition result picture and any one bounding rectangle in the second recognition result picture satisfies that the position coordinate deviation is less than the preset coordinate threshold and the size deviation is less than the preset size deviation threshold, it is determined that the changed area corresponding to the group of bounding rectangles is the overlapping area.

[0050] Through the above pixel value comparison, the similarities and differences between the changed area in the first recognition result picture and the changed area in the second recognition result picture are accurately determined, so that it can be determined whether there is an overlapping area, and further, it can be determined whether there is a stationary litter.

[0051] After it is determined that the overlapping area exists the stationary litter, the position information of the litter can also be determined based on the bounding rectangle of the overlapping area. Specifically, a bounding rectangle is added to the overlapping area, the center point coordinates of the bounding rectangle are obtained, and the position information of the stationary litter is determined according to the center point coordinates.

[0052] The center point coordinates of the bounding rectangle are directly taken as the center point coordinates corresponding to the overlapping area, and the position information of the stationary litter is directly represented by the center point coordinates corresponding to the overlapping area, so that the position information of the litter can be more accurately determined.

[0053] It should be noted that, since the litter in the actual scene can be immobile or mobile, through the above steps S101-S103, only the overlapping area between the change area in the first recognition result picture and the change area in the second recognition result picture is determined, that is, only the immobile litter is recognized, but due to the mobile litter, there is actually no corresponding overlapping area between the change area in the first recognition result picture and the change area in the second recognition result picture, so that the mobile litter cannot be recognized by determining whether there is an overlapping area between the change area in the first recognition result picture and the change area in the second recognition result picture.

[0054] Reference Figure 4 , the area a and the area d are overlapping areas, so there is the same litter, and the area e and the area f also have the same litter, but are not overlapping areas. It can be understood that the area a and the area d correspond to the immobile litter, and the area e and the area f correspond to the mobile litter, so it is necessary to continue to determine whether there is mobile litter in the preset road scene according to the change area in the first recognition result picture and the change area in the second recognition result picture except the overlapping area.

[0055] Therefore, in addition to the first case A1, a second case A2 is also included: if there is a non-overlapping area between the change area in the first recognition result picture and the change area in the second recognition result picture, it is determined whether there is mobile litter in the preset road scene based on the non-overlapping area.

[0056] Specifically, the following steps S1031-S1034 are further included:

[0057] S1031: Determine whether there are M target change areas without overlapping areas between the change area in the first recognition result picture and the change area in the second recognition result picture, M being a positive integer; if there are, execute step S1032, and if there are not, determine that the preset road scene corresponding to the to-be-determined area of the second recognition result picture does not have mobile litter.

[0058] Specifically, the change area in which the change area in the first recognition result picture and the change area in the second recognition result picture overlap is filtered out to obtain M target change areas.

[0059] Reference Figure 4 For example, the area a and the area d are filtered out to obtain 4 target change areas, and the 4 target change areas include the area b and the area e of the first recognition result picture, and the area c and the area f of the second recognition result picture.

[0060] S1032: Determine distances between each target change region belonging to the first recognition result picture and each target change region belonging to the second recognition result picture in the M target change regions, as a current target change region, to obtain at least one distance parameter.

[0061] Specifically, the distance parameter can be an Euclidean distance, as shown in the example of Figure 4 As shown in the example, take region b and region e in the first recognition result picture as the current target change region respectively, calculate the Euclidean distance between region b and region c in the second recognition result picture, the Euclidean distance between region b and region f in the second recognition result picture, the Euclidean distance between region e and region c in the second recognition result picture, and the Euclidean distance between region e and region f in the second recognition result picture respectively, to obtain four distance values correspondingly.

[0062] S1033: Determine at least one group of to-be-judged regions according to the at least one distance parameter.

[0063] It should be noted that when determining the at least one group of to-be-judged regions according to the at least one distance parameter, the specific implementation can be that a group of to-be-judged regions corresponding to the smallest distance parameter is determined from the at least one distance parameter, or each to-be-judged region smaller than a preset distance threshold can be determined from the at least one distance parameter. In addition, the group of change regions with smaller distance is selected as a group of to-be-judged regions because the range of movement of the moving litter in a short time interval of several frames is smaller.

[0064] Specifically, a group of to-be-judged regions represents two change regions that can be the same litter, and correspondingly includes one change region in the first recognition result picture and one change region in the second recognition result picture.

[0065] Reference is made to Figure 4 Taking the Euclidean distance as an example, each Euclidean distance obtained in step S1032 is compared with a preset distance threshold respectively to obtain each Euclidean distance smaller than the preset distance threshold. For example, assuming that the Euclidean distance between region e and region f is smaller than the preset distance threshold, region e and region f are taken as a group of to-be-judged regions, and assuming that the Euclidean distance between region e and region c is not smaller than the preset distance threshold, region e and region f will not be taken as a group of to-be-judged regions.

[0066] S1034: For each group of to-be-judgment regions, if the graphic difference between the two target change regions corresponding to the group of to-be-judgment regions is less than the preset difference threshold, it is determined that the preset road scene corresponding to the to-be-judgment region of the second recognition result picture exists a moving litter; if the graphic difference between the two target change regions corresponding to the group of to-be-judgment regions is not less than the preset difference threshold, it is determined that the preset road scene corresponding to the to-be-judgment region of the second recognition result picture does not exist a moving litter.

[0067] It should be noted that the graphic difference between the two target change regions includes the difference in graphic size and / or the difference in graphic shape. Correspondingly, the preset difference threshold includes a size difference threshold and / or a shape difference threshold.

[0068] Hereinafter, taking the graphic difference including the difference in graphic size and the difference in graphic shape as an example, reference is made to Figure 4 For example, add an outer rectangle frame to region e, and add an outer rectangle frame to region f; compare the size and shape of the two outer rectangle frames; if the size difference between the outer rectangle frame of region e and the outer rectangle frame of region f is less than the size difference threshold, and the shape difference is less than the shape difference threshold, it is determined that the preset road scene exists a moving litter at the position corresponding to region f. Since the change region is an irregular shape, the size difference is compared by adding an outer rectangle frame, so as to facilitate the judgment of the graphic difference of the change region.

[0069] Since the above steps S1031-S1034 combine distance judgment and graphic difference judgment, distance judgment can distinguish between a group of change regions with far and near distances, avoiding regarding a group of change regions with far distances as a group of to-be-judgment regions, and graphic difference judgment can distinguish between the size difference or shape difference of a group of to-be-judgment regions, avoiding identifying a group of to-be-judgment regions with large graphic difference as a change region with litter. Therefore, the above steps make the way of determining whether there is a moving litter more accurate and reliable, so that the litter recognition model can not only recognize static litter, but also recognize moving litter, so as to improve the type of litter recognition, and further improve the accuracy of litter recognition.

[0070] It can be understood that after determining that the preset road scene corresponding to the to-be-judgment region of the second recognition result picture exists a moving litter, there will also be false positives, such as a window object occasionally extending out and then retracting in the car window, which will be identified as a moving litter based on steps S1031-S1034. However, the window object is moving, but it is not actually a litter. Therefore, after determining that the preset road scene corresponding to the to-be-judgment region of the second recognition result picture exists a moving litter, it is necessary to determine whether the moving litter is a retractable window object.

[0071] To address the issue of misidentifying objects outside the window as moving debris, after determining that there is moving debris in the preset road scene corresponding to the area to be judged in the second recognition result image, the following steps S1035~S1037 may also be included:

[0072] S1035: Obtain the third set of scene images of the preset road scene, which includes the first scene image and the second scene image;

[0073] It should be noted that the second scene image is taken at a time significantly later than the first and second set of scene images. The interval between the second scene image and the first scene image is relatively long and can be set according to actual needs. For example, the first scene image is the first frame, and the second scene image is the 50th frame.

[0074] S1036: The third set of scene images is processed by the litter recognition model to obtain the third recognition result image representing the changes in the images.

[0075] For details, please refer to Figure 5 As shown, the debris recognition model processes the third set of scene images, identifies the changing areas of the second scene image relative to the first scene image, and obtains the third recognition result image. In specific implementation, the specific processing of obtaining the third recognition result image based on the third set of scene images is similar to the specific processing of obtaining the first recognition result image based on the first set of scene images; therefore, step S102 can be referred to.

[0076] S1037: Based on the first recognition result image and the third recognition result image, re-determine whether there is moving debris in the preset road scene. If there is, output the determination result that there is moving debris in the preset road scene. If there is no debris, output the determination result that there is no moving debris in the preset road scene.

[0077] Step S1037 is the same as or similar to the above steps S1031 to S1034. You can refer to steps S1031 to S1034.

[0078] In the case that the window object is occasionally extended out of the window and then retracted, such as a mobile phone of the user in the vehicle, the object is retracted immediately after being extended out of the window for a short time, and in this case, the object is not present in the first scene image and the second scene image, but the object is present in the other frame scene images between the first scene image and the second scene image, which leads to the determination that the preset road scene has the moving litter. Therefore, steps S1035-S1037 combine the first recognition result image and the third recognition result image, and the accurate determination of the window object that is occasionally extended out of the window and then retracted is achieved by analyzing the change region of the scene image with a long interval time, so that after the error determination that the preset road scene has the moving litter corresponding to the to-be-determined region of the second recognition result image, the litter recognition model can automatically correct the error and output the determination result that the preset road scene does not have the moving litter, and thus the accuracy of litter recognition is improved.

[0079] It should be noted that the re-determination of whether the preset road scene has the moving litter based on the first recognition result image and the third recognition result image in step S1037 can include the following steps:

[0080] Step 1: Determine whether the change region in the first recognition result image and the change region in the third recognition result image have N target change regions without overlapping regions, N being a positive integer, if yes, perform step 2, and if no, output the determination result that the preset road scene does not have the moving litter.

[0081] Specifically, the change region in the first recognition result image and the change region in the third recognition result image are filtered to obtain N target change regions. For example, the region k and the region a are filtered to obtain 3 target change regions, and the 3 target change regions include the region g of the first recognition result image and the region n and the region m of the third recognition result image. Figure 5 For example, the region k and the region a are filtered to obtain 3 target change regions, and the 3 target change regions include the region g of the first recognition result image and the region n and the region m of the third recognition result image.

[0082] Step 2: Take each target change region belonging to the first recognition result image in the N target change regions as a current target change region, determine the distance between the current target change region and each target change region belonging to the third recognition result image in the N target change regions, and obtain at least one distance parameter.

[0083] Specifically, the distance parameter can be the Euclidean distance. For example, the Euclidean distance between the region g and the region n of the third recognition result image and the Euclidean distance between the region g and the region m of the third recognition result image are calculated by taking the region g of the first recognition result image as the current target change region. Figure 5 For example, the Euclidean distance between the region g and the region n of the third recognition result image and the Euclidean distance between the region g and the region m of the third recognition result image are calculated by taking the region g of the first recognition result image as the current target change region.

[0084] Step 3: Determine at least one group of to-be-judged areas according to at least one distance parameter.

[0085] Step 4: For each group of to-be-judged areas, if the graphic difference between the two target change areas corresponding to the group of to-be-judged areas is less than a preset difference threshold, it is determined that the preset road scene corresponding to the to-be-judged area of the third recognition result picture exists a moving litter; if the graphic difference between the two target change areas corresponding to the group of to-be-judged areas is not less than the preset difference threshold, it is determined that the preset road scene corresponding to the to-be-judged area of the third recognition result picture does not exist a moving litter.

[0086] Through the above steps 1-4, after occasionally extending the window object outside the window to be wrongly judged as a moving litter, automatic error correction is performed, and thus the accuracy of litter recognition is improved.

[0087] Based on the same inventive concept, referring to Figure 6 The embodiment of the present application provides a road litter recognition device 10, which comprises: a scene picture acquisition unit 110, which is used for acquiring a first group of scene pictures and a second group of scene pictures of a same preset road scene, the first group of scene pictures and the second group of scene pictures both comprise a first scene picture, and the shooting time of the first scene picture is earlier than that of other scene pictures in the first group of scene pictures and the second group of scene pictures; a result picture acquisition unit 120, which is used for processing the first group of scene pictures and the second group of scene pictures through a pre-trained litter recognition model to obtain a first recognition result picture and a second recognition result picture representing picture changes; and a judgment unit 130, which is used for determining whether there is a litter in the preset road scene based on the positional relationship between the change area in the first recognition result picture and the change area in the second recognition result picture.

[0088] It can be understood that the judgment unit 130 is specifically used for: if there is an overlapping area between the change area in the first recognition result picture and the change area in the second recognition result picture, it is determined that there is a static litter in the preset road scene; and if there is a non-overlapping area between the change area in the first recognition result picture and the change area in the second recognition result picture, it is determined whether there is a moving litter in the preset road scene based on the non-overlapping area.

[0089] It can be understood that the road debris recognition device 10 further comprises a model training unit 140, the model training unit 140 comprising: a sample picture acquisition subunit 1410, configured to acquire a sample picture set, the sample picture set comprising a plurality of groups of sample pictures, each group of sample pictures comprising a first sample picture of a same road scene in a non-debris state, a second sample picture in a debris state, and a mask picture of the second sample picture; and a training subunit 1420, configured to train the original difference discrimination model based on the sample picture set to obtain the debris recognition model.

[0090] It can be understood that the sample picture acquisition subunit 1410 comprises: a labeling module 14110, configured to label a debris region of each second sample picture; and a mask picture generation module 14120, configured to perform binaryzation processing on each labeled second sample picture to generate a mask picture of the second sample picture.

[0091] It can be understood that the determination unit 130 further comprises: a pixel value comparison subunit 1310, configured to compare the pixel values of the first recognition result picture and the second recognition result picture; and an overlap recognition subunit 1320, configured to determine whether the changed regions in the first recognition result picture and the changed regions in the second recognition result picture have an overlapping region based on the pixel value comparison result.

[0092] It can be understood that the road debris recognition device 10 further comprises a positioning unit 150, the positioning unit 150 comprising: a bounding box adding subunit 1510, configured to add a bounding rectangle bounding box to the overlapping region; a coordinate acquisition subunit 1520, configured to acquire a center point coordinate of the bounding rectangle bounding box; and a position determination subunit 1530, configured to determine the position information of the debris according to the center point coordinate.

[0093] It can be understood that the determination unit 130 further comprises: a distance calculation subunit 1310, configured to, if the changed regions in the first recognition result picture and the changed regions in the second recognition result picture have M target changed regions without an overlapping region, M being a positive integer; determine, as a current target changed region, each target changed region belonging to the first recognition result picture in the M target changed regions, determine a distance between the current target changed region and each target changed region belonging to the second recognition result picture in the M target changed regions, and obtain at least one distance parameter; a region determination subunit 1320, configured to determine at least one group of to-be-judged regions according to the at least one distance parameter; and a graphical difference judgment subunit 1330, configured to, for each group of to-be-judged regions, if a graphical difference between two target changed regions corresponding to the group of to-be-judged regions is less than a preset difference threshold, determine that a preset road scene corresponding to a to-be-judged region of the second recognition result picture has a moving debris.

[0094] It can be understood that the scene picture acquisition unit 110 is further configured to acquire a third set of scene pictures of the preset road scene, the third set of scene pictures comprising the first scene picture and the second scene picture; the result picture acquisition unit 120 is further configured to obtain a third identification result picture representing picture changes by processing the third set of scene pictures through the litter identification model; and the determination unit 130 is further configured to re-determine whether the preset road scene has the moving litter based on the first identification result picture and the third identification result picture, and output a determination result that the preset road scene has the moving litter if the preset road scene has the moving litter.

[0095] It can be understood that the region determination sub-unit 1320 is specifically configured to determine a set of to-be-judged regions corresponding to the minimum distance parameter from the at least one distance parameter, or determine each to-be-judged region smaller than the preset distance threshold from the at least one distance parameter.

[0096] It should be understood that more implementation details of the road litter identification device in the embodiments of the present application are described with reference to the foregoing road litter identification method, and for the sake of brevity of the description, will not be described here.

[0097] Based on the same inventive concept, the embodiments of the present application also provide a power supply circuit of a road litter identification device, as shown in Figure 7 The power supply circuit comprises a memory 704, a processor 702, and a computer program stored in the memory 704 and executable on the processor 702, and the processor 702 executes the program to implement the steps of any embodiment of the road litter identification method.

[0098] In the Figure 7 bus architecture (represented by the bus 700), the bus 700 can comprise any number of interconnected buses and bridges, which link various circuits including one or more processors represented by the processor 702 and the memory represented by the memory 704. The bus 700 can also link various other circuits such as peripheral devices, voltage stabilizers, and power management circuits, which are well known in the art, and therefore, will not be further described herein. The bus interface 705 provides an interface between the bus 700 and the receiver 701 and the transmitter 703. The receiver 701 and the transmitter 703 can be the same element, i.e., a transceiver, which provides a unit for communicating with various other devices on a transmission medium. The processor 702 is responsible for managing the bus 700 and general processing, while the memory 704 can be used to store data used by the processor 702 in performing operations.

[0099] Through the embodiment of the present application, in the case that the litter identification model can distinguish each change area of the front and rear two scene pictures, the step of post-processing is optimized, and the first group of scene pictures and the second group of scene pictures are combined to determine whether there is litter, so that the problem of litter identification error caused by too many change areas is overcome, thereby solving the technical problem of litter false detection, and improving the accuracy of litter identification.

[0100] The functions described herein can be implemented in hardware, software executed by a processor, firmware, or any combination thereof. If implemented in software executed by a processor, the functions can be stored on or transferred over as one or more instructions or code on a computer-readable medium. Other examples and implementations are within the scope and spirit of the disclosure and appended claims. For example, due to the nature of software, functions described above can be implemented using software executed by a processor, hardware, firmware, hardwiring, or combinations of any of these. Features implementing functions can also be physically located at various positions, including being distributed such that portions of functions are implemented at different physical locations. Also, as technology evolves, the underlying device implementations could change while still fulfilling the functions described herein.

[0101] In several embodiments provided in the present application, it should be understood that the disclosed technical contents can be implemented by other ways. Among them, the above-described device embodiments are only schematic, for example, the division of the units can be a logical function division, and actual implementation can have another division way, for example, a plurality of units or components can be combined or integrated into another system, or some features can be ignored or not executed. In addition, the coupling or direct coupling or communication connection between the displayed or discussed each other can be through some interface, indirect coupling or communication connection between units or modules, which can be electrical or other forms.

[0102] The units described as separate components can or can not be physically separated, and the components of the control device can or can not be physical units, that is, they can be located in one place, or they can be distributed on multiple units. According to actual needs, part or all of the units can be selected to achieve the purpose of the present embodiment.

[0103] The integrated unit, if implemented in the form of a software function unit and sold or used as an independent product, can be stored in a computer readable storage medium. Based on such understanding, the technical solutions of the present application, essentially or in other words, the part that contributes to the prior art or the whole or part of the technical solutions can be embodied in the form of a software product. The computer software product is stored in a storage medium, and includes several instructions for causing a computer device (which can be a personal computer, a server, or a network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of the present application. The aforementioned storage medium includes: a U disk, a read-only memory (ROM, Read-Only Memory), a random access memory (RAM, Random Access Memory), a mobile hard disk, a magnetic disk or an optical disk, and various media that can store program codes.

[0104] The above only describes the embodiments of the present application and is not intended to limit the present application. For those skilled in the art, the present application can have various modifications and changes. Any modification, equivalent replacement, improvement, etc. made within the spirit and principles of the present application shall be included in the scope of the claims of the present application.

Claims

1. A method of identifying road debris, characterized by, The method comprises the following steps: acquiring a first group of scene pictures and a second group of scene pictures of a preset road scene, wherein the first group of scene pictures and the second group of scene pictures each comprise a first scene picture, and the first scene picture is taken earlier than other scene pictures in the first group of scene pictures and the second group of scene pictures; processing the first group of scene pictures and the second group of scene pictures by a pre-trained litter identification model to obtain a first identification result picture and a second identification result picture representing picture changes; determining whether there is litter in the preset road scene based on the position relationship between the changed area in the first identification result picture and the changed area in the second identification result picture, comprising: if there is an overlapping area between the changed area in the first identification result picture and the changed area in the second identification result picture, it is determined that there is static litter in the preset road scene; if there is a non-overlapping area between the changed area in the first identification result picture and the changed area in the second identification result picture, it is determined whether there is moving litter in the preset road scene based on the non-overlapping area; wherein, if there are M target changed areas without overlapping area between the changed area in the first identification result picture and the changed area in the second identification result picture, M is a positive integer; taking each target changed area belonging to the first identification result picture in the M target changed areas as a current target changed area, determining the distance between the current target changed area and each target changed area belonging to the second identification result picture in the M target changed areas to obtain at least one distance parameter; determining at least one group of to-be-judged areas according to the at least one distance parameter; for each group of to-be-judged areas, if the graphic difference between the two target changed areas corresponding to the group of to-be-judged areas is less than a preset difference threshold, it is determined that the preset road scene corresponding to the to-be-judged area of the second identification result picture has moving litter; wherein, after determining that the preset road scene corresponding to the to-be-judged area of the second identification result picture has moving litter, the method further comprises the following steps: acquiring a third group of scene pictures of the preset road scene, wherein the third group of scene pictures comprises a first scene picture and a second scene picture; processing the third group of scene pictures by the litter identification model to obtain a third identification result picture representing picture changes; re-determining whether there is moving litter in the preset road scene based on the first identification result picture and the third identification result picture, and outputting a determination result that the preset road scene has moving litter if there is.

2. The road debris identification method according to claim 1, characterized by, The method further comprises a step of pre-training an original difference discrimination model, which comprises the following steps: acquiring a sample picture set, wherein the sample picture set comprises a plurality of groups of sample pictures, and each group of sample pictures comprises a first sample picture of a same road scene in a litter-free state, a second sample picture in a litter state, and a mask picture of the second sample picture; training the original difference discrimination model based on the sample picture set to obtain the litter identification model.

3. The road debris identification method according to claim 1, characterized by, Before the determining that the preset road scene has the static litter, comprising: Comparing pixel values of the first identification result picture and the second identification result picture; Determining whether there is an overlapping area between the changed area in the first identification result picture and the changed area in the second identification result picture based on the pixel value comparison result.

4. The road debris identification method according to claim 1, characterized by, After the determining that the preset road scene has the static litter, comprising: Adding an external rectangular frame to the overlapping area; Obtaining a center point coordinate of the external rectangular frame; Determining the position information of the litter according to the center point coordinate.

5. The road debris identification method according to claim 1, characterized by, The determining at least one group of to-be-judged areas according to the at least one distance parameter, comprising: Determining a group of to-be-judged areas corresponding to a minimum distance parameter from the at least one distance parameter, or Determining each to-be-judged area smaller than a preset distance threshold from the at least one distance parameter.

6. A road debris recognition device characterized by comprising: Comprising: A scene picture acquisition unit, configured to acquire a first group of scene pictures and a second group of scene pictures of a preset road scene, wherein the first group of scene pictures and the second group of scene pictures each include a first scene picture, and the first scene picture is taken earlier than other scene pictures in the first group of scene pictures and the second group of scene pictures; A result picture acquisition unit, configured to process the first group of scene pictures and the second group of scene pictures by a pre-trained litter recognition model to obtain a first identification result picture and a second identification result picture representing changes in pictures; The determination unit is configured to determine whether there is litter in the preset road scene based on the positional relationship between the changed area in the first recognition result picture and the changed area in the second recognition result picture, including: if there is an overlapping area between the changed area in the first recognition result picture and the changed area in the second recognition result picture, it is determined that there is static litter in the preset road scene; if there is a non-overlapping area between the changed area in the first recognition result picture and the changed area in the second recognition result picture, it is determined whether there is moving litter in the preset road scene based on the non-overlapping area; wherein, if there are M target changed areas without overlapping area between the changed area in the first recognition result picture and the changed area in the second recognition result picture, M is a positive integer; each target changed area belonging to the first recognition result picture in the M target changed areas is taken as a current target changed area respectively, the distance between the current target changed area and each target changed area belonging to the second recognition result picture in the M target changed areas is determined to obtain at least one distance parameter; at least one group of to-be-judged areas is determined according to the at least one distance parameter; for each group of to-be-judged areas, if the graphic difference between the two target changed areas corresponding to the to-be-judged area is less than a preset difference threshold, it is determined that the preset road scene corresponding to the to-be-judged area of the second recognition result picture has moving litter; wherein, after determining that the preset road scene corresponding to the to-be-judged area of the second recognition result picture has moving litter, the method further includes: obtaining a third group of scene pictures of the preset road scene, the third group of scene pictures including a first scene picture and a second scene picture; processing the third group of scene pictures by using the litter recognition model to obtain a third recognition result picture representing picture changes; re-determining whether there is moving litter in the preset road scene based on the first recognition result picture and the third recognition result picture, and if there is, outputting a determination result that the preset road scene has moving litter.

7. An electronic device, comprising: The memory, the processor and the computer program stored in the memory and executable on the processor are included, and the processor executes the computer program to implement the method in any one of claims 1-5. The memory, the processor and the computer program stored in the memory and executable on the processor are included, and the processor executes the computer program to implement the method in any one of claims 1-5.

Images