The training method of the garbage can detection model and the garbage can detection method

Abstract

The embodiment of the application provides a garbage can detection model training method and a garbage can detection method, and belongs to the field of image recognition. The training method comprises the following steps: acquiring an image set of a garbage can during a garbage can overturning process on a garbage truck; determining, by using an identification model, multiple sets of overturning starting frames and overturning ending frames in the image set, determining multiple frames of images between each set of overturning starting frame and overturning ending frame as a to-be-labeled training set; identifying the color of the garbage can in the to-be-labeled training set by using the identification model; identifying the fullness degree of the garbage can in the to-be-labeled training set by using the identification model, so as to obtain the fullness degree of the garbage can; determining the density grade of the garbage can according to weighing data; labeling the to-be-labeled training set by using the color, the fullness degree and the density grade, so as to obtain a training set; and inputting the training set into a to-be-trained model for iterative training, so as to obtain a detection model.

Description

Technical Field

[0001] The present invention relates to the field of image processing, and in particular to a training method for a trash can detection model and a trash can detection method. Background Art

[0002] Existing technologies typically detect the overflow level and weight of trash cans using infrared or ultrasonic ranging to measure the height inside the trash can. This requires equipping each trash can with sensors, communication modules, and IoT cards, which is costly and requires regular battery replacement, resulting in excessive labor costs. Another approach is to analyze images of the trash cans to determine the overflow level, but in reality, it is often impossible to obtain a comprehensive image of the trash can's interior to accurately determine the overflow level. If overflow is determined solely by weight, each trash can would require an electronic floor scale, which is prohibitively expensive and requires lengthy construction time. Therefore, existing technologies have yet to provide a solution that can quickly and accurately detect the overflow level and weight of trash cans. Summary of the Invention

[0003] The purpose of the embodiments of the present invention is to overcome the problem that the existing technology cannot quickly, accurately and cost-effectively detect the overflow degree and weight of a trash can, and to provide a training method for a trash can detection model and a trash can detection method.

[0004] In a first aspect, the present application provides a method for training a trash can detection model, comprising:

[0005] Get a set of images of the trash can being turned over on the garbage truck;

[0006] The recognition model is used to determine multiple groups of barrel turning start frames and barrel turning end frames in the image set, and multiple frames of images between each group of barrel turning start frames and barrel turning end frames are determined as a training set to be labeled;

[0007] Use the recognition model to identify the color of the trash cans in the training set to be labeled;

[0008] Use the recognition model to identify the unlabeled training set to obtain the overflow level of the trash can;

[0009] Determine the density level of the trash can based on the weighing data;

[0010] The training set is labeled using the trash can color, overflow degree, and density level to obtain the training set;

[0011] The training set is input into the model to be trained for iterative training to obtain the detection model.

[0012] In one embodiment of the present application, before using the recognition model to determine multiple groups of barrel flip start frames and barrel flip end frames in an image set, and determining multiple frames of images between each group of barrel flip start frames and barrel flip end frames as a training set to be labeled, the method further includes:

[0013] Determine the maximum and minimum pixel values ​​of the image set based on the model of the garbage truck; use the recognition model to determine the pixel value of the garbage can occupied area in each frame of the image set as the garbage can area size of each frame; determine whether the garbage can area size in each frame of the image set is between the maximum and minimum pixel values;

[0014] If the size of the trash can area in a frame image in the image set is not between the maximum pixel value and the minimum pixel value, the frame image is deleted.

[0015] In one embodiment of the present application, a recognition model is used to determine multiple groups of barrel flip start frames and barrel flip end frames in an image set, and multiple frames of images between each group of barrel flip start frames and barrel flip end frames are determined as a training set to be labeled, including:

[0016] Determine the pixel value weight of the garbage truck according to the model of the garbage truck;

[0017] If the maximum pixel value does not exceed the product of the size of the trash can area in the image and the pixel value weight, if the trash can in the frame image is in a flipping state and the time difference between the frame image and the previous end frame exceeds a first time threshold, then the frame image is determined to be the starting frame;

[0018] When the maximum pixel value exceeds the product of the trash can area size of the image and the pixel value weight, if the trash can in the frame image is not in a flipping state, and the time difference between the frame image and the previous image whose maximum pixel value does not exceed the product of the trash can area size of the image and the pixel value weight is greater than the second time threshold, then the frame image is determined to be the end frame.

[0019] In one embodiment of the present application, the recognition model is used to identify the overflow degree of the trash can on the unlabeled training set to obtain the overflow degree of the trash can, including:

[0020] Divide the inner area of ​​the trash can in the training set into multiple inner parts;

[0021] Use the recognition model to identify the colors of multiple trash cans;

[0022] If the color inside the bucket is different from the color of the corresponding trash can, the part of the bucket corresponding to the color inside the bucket is regarded as the overflow part;

[0023] The overflow area is obtained by summing the areas of all overflowing parts of the trash can in the image;

[0024] The overflow degree of the trash can is determined according to the ratio of the value of the overflow area to the value of the volume of the corresponding trash can.

[0025] In one embodiment of the present application, the garbage includes kitchen waste, the garbage bin includes a kitchen waste bin filled with the kitchen waste, and determining the density level of the garbage bin according to the weighing data includes:

[0026] Determine the density of each kitchen waste bin and the corresponding density level based on the weighing data of each kitchen waste bin and the corresponding volume of the bin;

[0027] Use the recognition model to identify whether the trash can in the training set is a kitchen waste bin;

[0028] In the case where the trash can is a kitchen waste can, the identification model is used to identify the amount of solid matter in the can;

[0029] Establish a mapping between the density level of food waste bins and the corresponding amount of solids;

[0030] Use the amount of solids as the density grade for your food waste bin.

[0031] In one embodiment of the present application, the garbage includes other garbage, and the garbage bin includes other garbage bins containing other garbage. Determining the density level of the garbage bin according to the weighing data includes:

[0032] The density of each other trash can and the corresponding density level are determined according to the weighing data of each other trash can and the corresponding trash can volume.

[0033] A second aspect of the present application provides a trash can detection method, comprising:

[0034] Obtain the video to be tested that includes the process of turning over the trash can;

[0035] Inputting the video to be detected into the detection model, and determining the overflow degree and density level of each trash can based on the array output by the detection model, wherein the detection model is obtained by the training method of the trash can detection model provided in the first aspect of the present application;

[0036] The weight of each trash can is determined based on its overflow level and density level.

[0037] A third aspect of the present application provides a training device for a trash can detection model, comprising:

[0038] A video source acquisition unit, used to acquire an image set of the garbage bin during the process of being turned over on the garbage truck;

[0039] A video source processing unit is used to determine multiple groups of barrel flip start frames and barrel flip end frames in the image set using a recognition model, and determine multiple frames of images between each group of barrel flip start frames and barrel flip end frames as a training set to be labeled;

[0040] A color recognition unit is used to use a recognition model to identify the color of the trash cans in the training set to be labeled;

[0041] An overflow degree recognition unit is used to use the recognition model to recognize the unlabeled training set to obtain the overflow degree of the trash can;

[0042] a density determination unit, for determining the density level of the garbage in the garbage bin based on the weighing data;

[0043] A training set acquisition unit is used to label the unlabeled training set using color, overflow degree, and density level to obtain a training set;

[0044] The model training unit is used to input the training set into the model to be trained for iterative training to obtain the detection model.

[0045] A fourth aspect of the present application provides a trash can detection device, comprising:

[0046] A video acquisition unit to be detected is used to acquire a video to be detected containing the process of turning over the trash can;

[0047] a detection unit, configured to input the video to be detected into a detection model, and determine the overflow degree and density level of each trash can based on an array output by the detection model, wherein the detection model is obtained by the training method of the trash can detection model provided in the first aspect of the present application;

[0048] The weight determination unit is used to determine the weight of each trash can according to the overflow degree and density level of each trash can.

[0049] The fifth aspect of the present application provides an electronic device, including a processor and a memory, the memory storing machine-executable instructions that can be executed by the processor, and the processor executing the machine-executable instructions to implement the training method of the trash can detection model provided in the first aspect of the present application, or the trash can detection method provided in the second aspect of the present application.

[0050] The sixth aspect of the present application provides a machine-readable storage medium, which stores instructions. When the instructions are executed by a processor, the processor implements the training method of the trash can detection model provided in the first aspect of the present application, or the trash can detection method provided in the second aspect of the present application.

[0051] Through the above technical solution, the detection model can accurately identify the overflow degree and weight of the garbage bin based on the garbage truck's bin-turning video. There is no need to install infrared or ultrasonic ranging devices or electronic floor scales on the garbage truck. It can efficiently obtain the city's garbage distribution rate, realize the dynamic deployment of garbage bins in rapid garbage collection and transportation, and better improve the city's garbage utilization rate. At the same time, it can also give relevant prompts to the garbage truck driver based on the weight obtained by detection, thereby preventing unnecessary safety accidents caused by overloading.

[0052] Other features and advantages of the embodiments of the present invention will be described in detail in the subsequent detailed description. BRIEF DESCRIPTION OF THE DRAWINGS

[0053] The accompanying drawings are used to provide a further understanding of the embodiments of the present invention and constitute a part of the specification. Together with the following detailed description, they are used to explain the embodiments of the present invention, but do not constitute a limitation of the embodiments of the present invention. In the accompanying drawings:

[0054] Figure 1 The following schematically illustrates a flow chart of a method for training a trash can detection model according to an embodiment of the present application;

[0055] Figure 2 The figure schematically shows a flow chart of a trash can detection method according to an embodiment of the present application. DETAILED DESCRIPTION

[0056] The following describes the specific embodiments of the present application in detail with reference to the accompanying drawings. It should be understood that the specific embodiments described herein are only used to illustrate and explain the present application and are not intended to limit the present application.

[0057] It should be noted that if the embodiments of the present application involve directional indications (such as up, down, left, right, front, back, etc.), the directional indications are only used to explain the relative position relationship, reversal, etc. between the components under a certain specific posture (as shown in the accompanying drawings). If the specific posture changes, the directional indications will also change accordingly.

[0058] In addition, if there are descriptions involving "first", "second", etc. in the embodiments of the present application, the descriptions of "first", "second", etc. are only for descriptive purposes and cannot be understood as indicating or implying their relative importance or implicitly indicating the number of the indicated technical features. Therefore, the features defined as "first" and "second" may explicitly or implicitly include at least one of such features. In addition, the technical solutions between the various embodiments can be combined with each other, but they must be based on the fact that they can be implemented by ordinary technicians in this field. When the combination of technical solutions is contradictory or cannot be implemented, it should be deemed that such a combination of technical solutions does not exist and is not within the scope of protection required by this application.

[0059] Figure 1 The flowchart of a method for training a pallet state recognition model according to an embodiment of the present application is schematically shown. Figure 1 As shown, in one embodiment of the present application, a method for training a trash can detection model is provided, which may include steps S100 to S700.

[0060] Step S100: Acquire an image set of the garbage bin during the process of being turned over on the garbage truck.

[0061] In the training method provided in the embodiments of the present application, input image data is identified and labeled using a trained recognition model to establish a training set. The recognition model can identify trash cans, their color, their state, and the color and amount of solid matter inside them. The trash in the trash cans is dumped into a garbage truck via a tumbler. A processor captures a set of images of the tumbler process, captured by a fixed image acquisition device, as the data source for the training data. The image set can be a complete video of the tumbler process or multiple frames of images captured at intervals by the image acquisition device.

[0062] Step S200: using the recognition model to determine multiple groups of barrel turning start frames and barrel turning end frames in the image set, and determining multiple frames of images between each group of barrel turning start frames and barrel turning end frames as a training set to be labeled.

[0063] The bin flipping process consists of multiple bin flipping start points and corresponding bin flipping end points. The process between each bin flipping start point and end point is considered the bin flipping process for a single trash can. To filter out valid images in the image set that can be used by the recognition model to identify the color and overflow level of the trash can, it is necessary to determine the bin flipping start and end points in the image set, that is, the bin flipping start and end frames in the image set. The multiple frames between the bin flipping start and end frames are used as the dataset to be labeled.

[0064] In one embodiment of the present application, before step S200, the training method further includes:

[0065] Determine the maximum and minimum pixel values ​​of the image set based on the type of garbage truck;

[0066] The recognition model is used to determine the pixel value of the trash can occupied area in each frame of the image set as the size of the trash can area in each frame of the image;

[0067] Determine whether the size of the trash can area in each frame of the image set is between the maximum pixel value and the minimum pixel value;

[0068] If the size of the trash can area in a frame image in the image set is not between the maximum pixel value and the minimum pixel value, the frame image is deleted.

[0069] It's understandable that the maximum and minimum pixel values ​​are used to determine whether an image contains a trash can, and are therefore also used to filter the image set. Garbage trucks are classified into different models based on their load capacity. The image acquisition device is positioned differently depending on the model, so the maximum and minimum pixel values ​​need to be set based on the model.

[0070] After the recognition model identifies the trash can in the image, it determines the size of the pixel value of the area occupied by the trash can as the size of the trash can area in the frame image. Because the process of turning the trash can over is a process from far to near for the image acquisition device, if the trash can is too far away from the image acquisition device, the size of the trash can area in the captured image will be too small, and it can be regarded as the absence of the trash can in the frame image. Similarly, if the trash can is too close to the image acquisition device and is about to leave the shooting range of the image acquisition device, the size of the trash can area in the captured image will be too large, and it can also be regarded as the absence of the trash can in the frame image. The processor determines whether the size of the trash can area in each frame image is between the set maximum pixel value and the minimum pixel value. If the size of the trash can area in the image is not between the maximum pixel value and the minimum pixel value, that is, the frame image can be regarded as the absence of the trash can, and is determined to be an invalid image and deleted.

[0071] In one embodiment of the present application, step S200 includes:

[0072] Determine the pixel value weight of the garbage truck according to the model of the garbage truck;

[0073] If the maximum pixel value does not exceed the product of the size of the trash can area in the image and the pixel value weight, if the trash can in the frame image is in a flipping state and the time difference between the frame image and the previous end frame exceeds a first time threshold, then the frame image is determined to be the starting frame;

[0074] When the maximum pixel value exceeds the product of the trash can area size of the image and the pixel value weight, if the trash can in the frame image is not in a flipping state, and the time difference between the frame image and the previous image whose maximum pixel value does not exceed the product of the trash can area size of the image and the pixel value weight is greater than the second time threshold, then the frame image is determined to be the end frame.

[0075] As mentioned above, garbage trucks are classified into different models based on their load capacity. Depending on the model, the image acquisition device is positioned differently. Consequently, the same trash can will occupy different proportions in the images captured by the image acquisition devices at different locations. Therefore, the pixel weights used to represent the size of the trash can area need to be set based on the model of the garbage truck. Similarly, because the image acquisition device sees a trash can moving from far away to near, the size of the trash can area can be used to represent the progress of the turning process for a single trash can.

[0076] For example, the pixel value weight is recorded as T S , the maximum pixel value is recorded as MaxAreaSize, the minimum pixel value is recorded as MiniAreaSize, and the size of the trash can area is recorded as ImageShe. When MaxAreaSize≤ImageSh×T S When the recognition model recognizes that the trash can in the frame image is in the overturning state, and the time difference Timegap1 between the frame image and the previous end frame exceeds the first time threshold T1, the frame image can be marked as the starting frame, that is, if the time difference between the current frame and the previous end frame exceeds the first time threshold T1, it can be confirmed that another trash can overturning has started. The specific size of the first time threshold T1 can be debugged and set according to the design requirements of the model; when MaxAreaSize>ImageShe×T S When the recognition model recognizes that the trash can in the frame image is not in the overturned state, and from the previous MaxAreaSize≤ImaheSh×T S If the time Timegap2 consumed from the image to the frame image is greater than the second time threshold T2, the frame image can be marked as the end frame.

[0077] Step S300: using the recognition model to identify the color of the trash cans in the training set to be labeled.

[0078] The recognition model identifies the color of the trash can in each frame of the training set. For example, the colors of the trash can include red, green, blue, and black. In one embodiment of the present application, the color of the trash can is represented by a 4-element array, such as (1, 0, 0, 0) indicates that the trash can in the frame is red, (0, 1, 0, 0) indicates that the trash can in the frame is green, (0, 0, 1, 0) indicates that the trash can in the frame is blue, and (0, 0, 0, 1) indicates that the trash can in the frame is black.

[0079] Step S400: using the recognition model to identify the unlabeled training set to obtain the overflow degree of the trash can.

[0080] The overflow degree of the trash can is also determined based on the recognition results of the recognition model.

[0081] In an embodiment of the present application, step S400 comprises:

[0082] dividing the in-bar region of the garbage can in the training set to be labeled into a plurality of in-bar parts;

[0083] recognizing the in-bar color of the plurality of garbage cans by using the recognition model;

[0084] if the in-bar color is different from the corresponding garbage can color, then taking the in-bar part corresponding to the in-bar color as an overflow part;

[0085] summing up the areas of all the overflow parts in the garbage can in the image to obtain an overflow area;

[0086] determining the overflow degree of the garbage can according to the ratio of the numerical value of the overflow area to the numerical value of the volume of the corresponding garbage can.

[0087] The processor divides the in-bar region of the garbage can in the image into a plurality of in-bar parts, and recognizes the color of each in-bar part by using the recognition model, which is recorded as the in-bar color. If the in-bar color is different from the color of the recognized garbage can, it can be considered that there is garbage in the in-bar part. The in-bar part is recorded as an overflow part. The areas of all the overflow parts in the image are summed up to obtain the overflow area of the frame image.

[0088] After obtaining the overflow area of the frame image, the ratio of the overflow area to the volume of the garbage can is calculated. The volume of the garbage can can be a fixed value set in advance. It is worth noting that the ratio calculated here is the ratio of the numerical value of the overflow area to the volume of the garbage can. The processor can determine the overflow degree of the garbage can according to the ratio. In an embodiment of the present application, the overflow degree of the garbage can is divided into 10 levels from large to small, recorded as numbers 1-10. The larger the ratio of the numerical value of the overflow area to the volume of the garbage can, the higher the overflow degree.

[0089] Step S500: determining the density level of the garbage can according to the weighing data.

[0090] Before the garbage can is turned over, the garbage can can be weighed first. According to the weighing data and the volume of the garbage can, the density of the garbage can can be determined, that is, the average density of the whole garbage can including garbage. Then the density level is determined according to the density. In an embodiment of the present application, the density level of the garbage can is divided into 10 levels from large to small, recorded as numbers 1-10.

[0091] In an embodiment of the present application, the garbage includes kitchen waste, and the garbage can includes a kitchen waste can containing kitchen waste. Step S500 comprises:

[0092] Determine the density of each kitchen waste bin and the corresponding density level based on the weighing data of each kitchen waste bin and the corresponding volume of the bin;

[0093] Use the recognition model to identify whether the trash can in the training set is a kitchen waste bin;

[0094] In the case where the trash can is a kitchen waste can, the identification model is used to identify the amount of solid matter in the can;

[0095] Establish a mapping between the density level of food waste bins and the corresponding amount of solids;

[0096] Use the amount of solids as the density grade for your food waste bin.

[0097] Because kitchen waste is a mixture of fluid and solid, and the more solids there are, the lower the density, the density grade of the kitchen waste bin is determined based on the weighing data and the amount of solids in the kitchen waste.

[0098] First, the density level of the food waste bin is determined based on the bin's weight data, which serves as the data foundation for subsequent mapping. The recognition model then identifies whether the bin in each image in the training set is a food waste bin, using its color to determine if it is a food waste bin. If the bin in the image is a food waste bin, the processor uses the recognition model to identify the amount of solid matter in the bin in that image.

[0099] The number of solids in the recognized image is matched one-to-one with the density level of the garbage bin corresponding to the image, a mapping between the number of solids and the density level of the kitchen waste bin is established, and the number of solids is used as the density level of the kitchen waste bin, that is, the density level of the kitchen waste is marked with the number of solids.

[0100] In one embodiment of the present application, the garbage includes other garbage, and the garbage bin includes other garbage bins containing other garbage. Step S500 includes:

[0101] The density of each other trash can and the corresponding density level are determined according to the weighing data of each other trash can and the corresponding trash can volume.

[0102] It is understandable that garbage other than kitchen waste can be directly labeled using the density level obtained based on the weighing data and the volume of the garbage bin.

[0103] Step S600: labeling the unlabeled training set using color, overflow degree, and density level to obtain a training set.

[0104] The color, overflow degree, and density level of the trash cans obtained in steps S100-S500 are used to label the training set to obtain a training set. In one embodiment of the present application, the trash cans are labeled in the form of a 6-element array. For example, the first 4 bits of the 6-element array represent the color of the trash can, the 5th bit represents the overflow degree of the trash can, and the 6th bit represents the density level of the trash can. For example, (1, 0, 0, 0, 8, 8) represents a red trash can with an overflow degree of 8 and a density level of 8.

[0105] Step S700: input the training set into the model to be trained for iterative training to obtain a detection model.

[0106] The labeled training set is the labeled training set, which is input into the model to be trained, and the relevant iterative training parameters are set to perform iterative training. For example, YOLOv5 is used as the model to be trained.

[0107] Through the above technical solution, the detection model can accurately identify the overflow degree and weight of the garbage bin based on the garbage truck's bin-turning video. There is no need to install infrared or ultrasonic ranging devices or electronic floor scales on the garbage truck. It can efficiently obtain the city's garbage distribution rate, realize the dynamic deployment of garbage bins in rapid garbage collection and transportation, and better improve the city's garbage utilization rate. At the same time, it can also give relevant prompts to the garbage truck driver based on the weight obtained by detection, thereby preventing unnecessary safety accidents caused by overloading.

[0108] Figure 2 The flowchart of a method for training a pallet state recognition model according to an embodiment of the present application is schematically shown. Figure 2 As shown, in one embodiment of the present application, a trash can detection method is provided, comprising:

[0109] Step S800: Obtaining a video to be detected containing the process of turning over the trash can;

[0110] Step S900: Input the video to be detected into the detection model, and determine the overflow degree and density level of each trash can based on the array output by the detection model, wherein the detection model is obtained by the training method of the trash can detection model in steps S100-S700;

[0111] Step S1000: Determine the weight of each trash can according to the overflow level and density level of each trash can.

[0112] The garbage can detection model obtained through steps S100-S700 can be directly used for identification of the garbage can overflow degree and density level. After inputting the to-be-detected video of the turnover process of the garbage can into the detection model, the detection model outputs an array corresponding to each frame of image. In an embodiment of the present application, the detection model outputs a 6-element array. The first 4 elements represent the color of the garbage can, the 5th element is the overflow degree of the garbage can, and the 6th element is the density level of the garbage can. The processor first filters out the arrays with large errors in the overflow degree and the density level by filtering, and then calculates the average value of the overflow degree and the average value of the density level of the remaining arrays as the overflow degree and the density level in the output array for the to-be-detected video. The first 4 elements of the array are added together as the color of the garbage can in the output array for the to-be-detected video.

[0113] The processor can determine the color of the garbage can in the to-be-detected video according to the position of the maximum value of the first 4 elements in the output array. For example, the first 4 elements in the output array are (2, 4, 0, 0), and the maximum value 4 is located at the second position, which indicates green. Therefore, it can be determined that the color of the garbage can is green. At the same time, the processor can also determine the weight of the garbage can according to the overflow degree and the density level in the array.

[0114] In an embodiment of the present application, the garbage includes kitchen waste, and the garbage can includes a kitchen waste can containing kitchen waste. After the garbage can detection model identifies that the color of the garbage can in the image is the color of the kitchen waste can, it starts to identify the solid quantity in the can, and determines the density level of the kitchen waste can according to the mapping of the solid quantity of the kitchen waste and the density level.

[0115] In an embodiment of the present application, a training device of a garbage can detection model is provided, which includes:

[0116] A video source acquisition unit is configured to acquire an image set of a garbage can during a turnover process on a garbage truck.

[0117] A video source processing unit is configured to determine, by using the identification model, a plurality of groups of turnover start frames and turnover end frames in the image set, and determine a plurality of frames of images between each group of turnover start frame and turnover end frame as a to-be-labeled training set.

[0118] A color identification unit is configured to identify the color of the garbage can in the to-be-labeled training set by using the identification model.

[0119] An overflow degree identification unit is configured to identify the to-be-labeled training set by using the identification model to obtain the overflow degree of the garbage can.

[0120] A density determination unit is configured to determine the density level of the garbage in the garbage can according to the weighing data.

[0121] A training set acquisition unit, configured to label the unlabeled training set using color, overflow degree, and density level to obtain a training set;

[0122] The model training unit is used to input the training set into the model to be trained for iterative training to obtain the detection model.

[0123] The training device for the trash can detection model provided in the embodiment of the present application can implement each process of steps S100-S700 in the method embodiment and can achieve the same technical effect. To avoid repetition, it will not be described here.

[0124] In one embodiment of the present application, a trash can detection device is provided, characterized by comprising:

[0125] A video acquisition unit to be detected is used to acquire a video to be detected containing the process of turning over the trash can;

[0126] a detection unit, configured to input the video to be detected into a detection model, and determine the overflow degree and density level of each trash can according to an array output by the detection model, wherein the detection model is obtained by the trash can detection model training method as described in steps S100-S700;

[0127] The weight determination unit is used to determine the weight of each trash can according to the overflow degree and density level of each trash can.

[0128] The trash can detection device provided in the embodiment of the present application can implement each process of step S800 to step S1000 in the method embodiment and can achieve the same technical effect. To avoid repetition, it will not be described here.

[0129] In one embodiment of the present application, an electronic device is provided, including a processor and a memory, wherein the memory stores machine-executable instructions that can be executed by the processor, and the processor can execute the machine-executable instructions to implement the training method of the trash can detection model in the above embodiment, or the trash can detection method in the above embodiment.

[0130] In one embodiment of the present application, a machine-readable storage medium is provided, on which instructions are stored. When the instructions are executed by a processor, the processor implements the training method of the trash can detection model in the above embodiment, or the supported trash can detection method in the above embodiment.

[0131] Those skilled in the art will appreciate that the embodiments of the present application can be provided as methods, systems, or computer program products. Therefore, the application can adopt the form of a complete hardware embodiment, a complete software embodiment, or an embodiment in combination with software and hardware. Moreover, the application can adopt the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to magnetic disk storage, CD ROM, optical storage, etc.) that contain computer-usable program code.

[0132] The present application is described with reference to the flowcharts and / or block diagrams of the methods, devices (systems), and computer program products according to the embodiments of the present application. It should be understood that each process and / or box in the flowchart and / or block diagram, as well as the combination of the processes and / or boxes in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to produce a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the steps in the process. Figure 1 a process or multiple processes and / or boxes Figure 1 These computer program instructions can also be stored in a computer-readable memory that can guide a computer or other programmable data processing device to work in a specific way, so that the instructions stored in the computer-readable memory produce a product including the instruction device, which implements the function specified in the process. Figure 1 a process or multiple processes and / or boxes Figure 1 These computer program instructions can also be loaded onto a computer or other programmable data processing device, so that a series of operation steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing instructions for implementing the process in the process. Figure 1 a process or multiple processes and / or boxes Figure 1 A step that specifies a function in one or more boxes.

[0133] In a typical configuration, a computing device includes one or more processors (CPUs), input / output interfaces, network interfaces, and memory.

[0134] The memory may include non-permanent memory in a computer-readable medium, random access memory (RAM) and / or non-volatile memory in the form of read-only memory (ROM) or flash RAM. The memory is an example of a computer-readable medium.

[0135] Computer-readable media include permanent and non-permanent, removable and non-removable media that can be implemented by any method or technology to store information. The information can be computer-readable instructions, data structures, program modules or other data. Examples of computer storage media include, but are not limited to, phase change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technology, compact disc read-only memory (CD ROM), digital versatile disc (DVD) or other optical storage, magnetic cassettes, magnetic tape, magnetic disk storage or other magnetic storage devices or any other non-transmission media that can be used to store information that can be accessed by a computing device. As defined herein, computer-readable media does not include transitory computer-readable media (transitory media), such as modulated data signals and carrier waves.

[0136] It should also be noted that the terms "comprises," "includes," or any other variations thereof are intended to encompass non-exclusive inclusion, such that a process, method, commodity, or apparatus that includes a series of elements includes not only those elements but also other elements not explicitly listed, or includes elements inherent to such process, method, commodity, or apparatus. In the absence of further limitations, an element defined by the phrase "comprises a ..." does not exclude the presence of other identical elements in the process, method, commodity, or apparatus that includes the element.

[0137] The above are merely embodiments of the present application and are not intended to limit the present application. For those skilled in the art, the present application may have various modifications and variations. Any modifications, equivalent replacements, improvements, etc. made within the spirit and principles of the present application should all be included within the scope of the claims of the present application.

Claims

1. A training method for a trash can detection model, characterized in that: include: Get a set of images of the trash can being turned over on the garbage truck; Determining the maximum pixel value and the minimum pixel value of the image set according to the model of the garbage truck; Determine the size of the pixel value of the trash can occupied area in each frame of the image set using the recognition model as the size of the trash can area in each frame of the image; Using the recognition model to determine multiple groups of barrel turning start frames and barrel turning end frames in the image set, and determining multiple frames of images between each group of barrel turning start frames and barrel turning end frames as a training set to be labeled; Using the recognition model to identify the color of the trash cans in the training set to be labeled; Using the recognition model to identify the training set to be labeled to obtain the overflow degree of the trash can; determining the density level of the trash can according to the weighing data; Labeling the to-be-labeled training set using the trash can color, the overflow degree, and the density level to obtain a training set; Inputting the training set into the model to be trained for iterative training to obtain a detection model; The method of using the recognition model to determine multiple groups of barrel turning start frames and barrel turning end frames in the image set, and determining multiple frames of images between each group of barrel turning start frames and barrel turning end frames as a training set to be labeled, includes: Determining the pixel value weight of the garbage truck according to the model of the garbage truck; If the maximum pixel value does not exceed the product of the size of the trash can area in the image and the pixel value weight, if the trash can in the frame image is in a flipping state and the time difference between the frame image and the previous end frame exceeds a first time threshold, then determine the frame image as the starting frame; In the case that the maximum pixel value exceeds the product of the trash can area size of the image and the pixel value weight, if the trash can in the frame image is not in a flipping state, and the time difference between the frame image and the previous image in which the maximum pixel value does not exceed the product of the trash can area size and the pixel value weight is greater than a second time threshold, then the frame image is determined to be the end frame.

2. The method according to claim 1, characterized in that Before determining multiple groups of barrel-turning start frames and barrel-turning end frames in the image set using the recognition model, and determining multiple frames of images between each group of barrel-turning start frames and barrel-turning end frames as a training set to be labeled, the method further includes: Determine whether the size of the trash can area in each frame of the image in the image set is between the maximum pixel value and the minimum pixel value; If the size of the trash can area in a frame image in the image set is not between the maximum pixel value and the minimum pixel value, the frame image is deleted.

3. The method according to claim 1, characterized in that The step of using the recognition model to identify the training set to be labeled to obtain the overflow degree of the trash can includes: Dividing the inner area of ​​the trash can in the training set to be labeled into multiple inner parts; Using the recognition model to identify the colors inside the plurality of trash cans; If the color of the bucket is different from the color of the corresponding trash can, the part of the bucket corresponding to the color of the bucket is regarded as the overflow part; The overflow area is obtained by summing the areas of all overflowing parts of the trash can in the image; The overflow degree of the trash can is determined according to the ratio of the value of the overflow area to the value of the volume of the corresponding trash can.

4. The method according to claim 1, wherein The garbage includes kitchen waste, and the garbage bin includes a kitchen waste bin filled with the kitchen waste. The method of determining the density level of the garbage bin according to the weighing data includes: Determining the density and corresponding density level of each of the kitchen waste bins according to the weighing data of each of the kitchen waste bins and the corresponding volume of the bin; Using the recognition model to identify whether the trash can in the training set to be labeled is the kitchen waste can; In the case where the trash can is the kitchen waste can, using the recognition model to identify the amount of solids in the can; Establishing a mapping between the density level of the food waste bin and the corresponding solid quantity; The solid quantity is used as the density grade of the food waste bin.

5. The method according to claim 1, wherein The garbage includes other garbage, and the garbage bin includes other garbage bins containing other garbage. The method of determining the density level of the garbage bin based on the weighing data includes: The density and corresponding density level of each of the other trash cans are determined according to the weighing data of each of the other trash cans and the corresponding trash can volume.

6. A method for detecting a trash can, characterized in that: include: Obtain the video to be tested that includes the process of turning over the trash can; Inputting the video to be detected into a detection model, and determining the overflow degree and density level of each trash can according to an array output by the detection model, wherein the detection model is obtained by the training method of the trash can detection model according to any one of claims 1 to 5; The weight of each trash can is determined based on its overflow level and density level.

7. A training device for a trash can detection model, characterized in that: include: A video source acquisition unit, used to acquire an image set of the garbage bin during the process of being turned over on the garbage truck; A video source processing unit is used to determine multiple groups of barrel flip start frames and barrel flip end frames in the image set using a recognition model, and determine multiple frames of images between each group of barrel flip start frames and barrel flip end frames as a training set to be labeled; A color recognition unit, configured to use the recognition model to recognize the color of the trash cans in the training set to be labeled; an overflow degree recognition unit, configured to recognize the training set to be labeled using the recognition model to obtain the overflow degree of the trash can; a density determination unit, for determining the density level of the garbage in the garbage bin based on the weighing data; a training set acquisition unit, configured to label the training set to be labeled using the trash can color, the overflow degree, and the density level to obtain a training set; A model training unit, configured to input the training set into a model to be trained for iterative training to obtain a detection model; The video source processing unit is further configured to: Determining the maximum pixel value and the minimum pixel value of the image set according to the model of the garbage truck; Determine the size of the pixel value of the trash can occupied area in each frame of the image set using the recognition model as the size of the trash can area in each frame of the image; Determining the pixel value weight of the garbage truck according to the model of the garbage truck; If the maximum pixel value does not exceed the product of the size of the trash can area in the image and the pixel value weight, if the trash can in the frame image is in a flipping state and the time difference between the frame image and the previous end frame exceeds a first time threshold, then determine the frame image as the starting frame; In the case that the maximum pixel value exceeds the product of the trash can area size of the image and the pixel value weight, if the trash can in the frame image is not in a flipping state, and the time difference between the frame image and the previous image in which the maximum pixel value does not exceed the product of the trash can area size and the pixel value weight is greater than a second time threshold, then the frame image is determined to be the end frame.

8. A garbage can detection device, characterized in that: include: A video acquisition unit to be detected is used to acquire a video to be detected containing the process of turning over the trash can; a detection unit, configured to input the video to be detected into a detection model, and determine the overflow degree and density level of each trash can according to an array output by the detection model, wherein the detection model is obtained by the training method of the trash can detection model according to any one of claims 1 to 5; The weight determination unit is used to determine the weight of each trash can according to the overflow degree and density level of each trash can.

9. An electronic device, characterized in that: It includes a processor and a memory, the memory storing machine-executable instructions that can be executed by the processor, and the processor executing the machine-executable instructions to implement the training method of the trash can detection model described in any one of claims 1 to 5, or the trash can detection method described in claim 6.

10. A machine-readable storage medium, characterized in that The machine-readable storage medium stores instructions, which, when executed by a processor, enable the processor to implement the training method for a trash can detection model according to any one of claims 1 to 5, or the trash can detection method according to claim 6.

Images