Garbage can recognition system and method based on vehicle-mounted video

Abstract

The invention relates to the technical field of garbage recycling treatment, in particular to a garbage can recognition system and method based on a vehicle-mounted video, and the system comprises a target detection module which is used for detecting a garbage can at the side edge of a vehicle body according to the preset direction of the garbage can, and setting a component detection model and a category detection model of the garbage can; the image judgment module is used for collecting a sample image corresponding to the garbage can and analyzing the feature positioning points and feature description information of the garbage can in the sample image by using a component detection model; the garbage state judgment module is used for processing feature positioning points and feature description information of the garbage cans in the sample images and determining state indexes of the garbage cans in corresponding garbage states by using a category detection model; the path planning module is used for determining the garbage distribution proportion in the garbage can according to the state index of the garbage can in the corresponding garbage state, and setting the accommodating path of the garbage can according to the garbage distribution proportion; precise recognition and efficient management of the garbage can are achieved.

Description

Technical Field

[0001] The present invention relates to the technical field of garbage collection and treatment, and specifically to a garbage can recognition system and method based on vehicle-mounted video. Background Art

[0002] Traditional garbage management systems usually rely on manual inspections or fixed sensor monitoring, which have problems such as limited coverage and lagged response. With the acceleration of urbanization, how to efficiently manage and optimize garbage collection has become an important issue in urban management.

[0003] For example, Chinese Patent Publication No. CN113989646A discloses a garbage classification detection method, which constructs a garbage can category recognition model and a garbage discrimination model corresponding to different garbage categories, obtains an image to be detected and recognizes the target garbage can in the image to be detected, uses the garbage can category recognition model to recognize the category of the target garbage can, calls the garbage discrimination model corresponding to the category, discriminates the category of the garbage in the image to be detected, determines whether the category of the garbage in the image to be detected is the category corresponding to the garbage discrimination model, and outputs the detection result.

[0004] The prior art illustrates that the garbage can realizes the detection of garbage judgment through image comparison, but in the process of identifying the garbage inside the garbage can, it is also necessary to identify the type of garbage and the location where the corresponding components are located. After completing the corresponding garbage classification, multiple points are set for the garbage to be identified, and the garbage is accommodated according to the different information identified at multiple points, making the garbage management work more orderly. At the same time, the garbage collection path also needs to be dynamically adjusted so that the corresponding types of garbage can be processed in the shortest time, improving the implementation management effect of the garbage can state. Summary of the Invention

[0005] To solve the above technical problems, the technical solution adopted by the present invention is: a garbage can recognition system based on vehicle-mounted video, including: a target detection module, which is used to detect the garbage cans on the side of the vehicle body in accordance with the preset direction of the garbage can, and set a component detection model and a category detection model for the garbage can.

[0006] An image discrimination module, which is used to collect a sample image corresponding to the garbage can and analyze the feature positioning points and feature description information of the garbage can in the sample image using the component detection model.

[0007] A garbage state judgment module, which is used to process the feature positioning points and feature description information of the garbage can in the sample image and determine the state index of the garbage can in the corresponding garbage state using the category detection model.

[0008] A path planning module, which is used to determine the proportion of garbage distribution in the garbage can according to the state index of the garbage can in the corresponding garbage state, and set the collection path of the garbage can according to the proportion of garbage distribution.

[0009] A trash can recognition method based on in-vehicle video, comprising: S1, detecting trash cans in a preset direction on the side of the vehicle body, obtaining a trash can contour image and a trash can internal image, extracting the color and shape of the trash can, and setting an RFID tag for identification.

[0010] S2, analyzing elevation points and shadow points in the trash can internal image, generating a segmented image and extracting feature information, creating simulation model information about clustering points; based on the simulation model information, setting a component detection model and a category detection model.

[0011] S3, collecting sample images of the trash can, dividing the sample images into multiple sub-images, calculating the garbage distribution probability, identifying the garbage at each position and adding feature positioning points, and setting feature description information.

[0012] S4, using the component detection model, tracking the change trend of the garbage distribution probability at the feature positioning points, determining the priority of the garbage distribution, calculating the image similarity between the top position and the bottom position of the trash can, and determining the garbage component and the volume.

[0013] S5, starting from each feature positioning point, inputting the feature description information into the category detection model to output the garbage state; evaluating the state of each feature positioning point to obtain a type score, a distribution score, a probability score, and a capacity score, and comprehensively calculating to obtain a state index.

[0014] S6, dividing the trash can into multiple sub-regions according to the state index, calculating the proportion of the garbage distribution in the trash can in each sub-region to form a distribution result vector; generating multiple moving paths according to the distribution result vector, calculating the path probability value, and selecting the path corresponding to the maximum probability value as the collection path.

[0015] The beneficial effects of the present invention are as follows: First, through the collaborative work of the target detection module and the image discrimination module, the present invention can accurately identify the types and distribution of trash cans and the garbage inside them. By detecting in the preset direction of the trash can and setting a component detection model and a category detection model, a model for the garbage inside the current trash can can be obtained, further improving the efficiency and accuracy of garbage recognition. At the same time, setting feature positioning points and feature description information for the information recognized by the model will make the garbage inside the trash can contain more feature information when being recognized, facilitating setting various classification requirements according to different needs for trash can collection, and ultimately improving the accuracy of garbage classification.

[0016] Second, the detailed status indicators provided by the garbage status judgment module of the present invention help to more reasonably arrange the garbage collection plan, analyze the garbage and set status indicators, which can specifically describe the situation of the corresponding garbage, and combine the corresponding category detection models to further describe the status indicators. At this time, the obtained status indicators are not only the definition of the garbage in the trash can, but also include the detection results of how the current garbage truck processes the garbage. This processing method can improve the speed of garbage containment, facilitate the management of complex and diverse garbage, and improve the resource utilization rate and service efficiency of the overall garbage truck processing.

[0017] Third, the present invention dynamically adjusts the containment path based on real-time data through the path planning module, reducing the driving distance and time cost. For different garbage composition ratios, the path planning module will adjust the processing method to dynamically adjust the path of the current garbage truck, reducing the situation where the garbage containment rate is reduced due to mechanical garbage containment, and improving the flexibility of the garbage truck for different garbage containment tasks, enabling the garbage truck to complete the task in the shortest time. BRIEF DESCRIPTION OF THE DRAWINGS

[0018] The present invention will be further described below in conjunction with the drawings and embodiments.

[0019] Figure 1 is a system schematic diagram of a trash can recognition system based on on-vehicle video.

[0020] Figure 2 is a flowchart of the target detection module of a trash can recognition system based on on-vehicle video.

[0021] Figure 3 is a flowchart of the image discrimination module of a trash can recognition system based on on-vehicle video.

[0022] Figure 4 is a flowchart of the garbage status judgment module of a trash can recognition system based on on-vehicle video.

[0023] Figure 5 is a flowchart of the path planning module of a trash can recognition system based on on-vehicle video.

[0024] Figure 6 is a flowchart of a trash can recognition method based on on-vehicle video. DETAILED DESCRIPTION OF THE EMBODIMENTS

[0025] The embodiments of the present invention will be described in detail below. The following described embodiments are exemplary and are only used to explain the present invention, and should not be construed as a limitation to the present invention. For those not specified in the embodiments, the technologies or conditions described in the literature in the art or according to the product instructions are followed.

[0026] Refer to Figure 1 , a trash can recognition system based on in-vehicle video, including: an object detection module, an image discrimination module, a garbage state judgment module, and a path planning module; wherein, the output end of the object detection module is connected to the image discrimination module, the output end of the image discrimination module is connected to the garbage state judgment module, and the output end of the garbage state judgment module is connected to the path planning module.

[0027] The object detection module is used to detect the trash can on the side of the vehicle body in accordance with the preset direction of the trash can, and set the composition detection model and the category detection model of the trash can.

[0028] The composition detection model is used to identify the main material of the garbage in the trash can and whether the garbage contains corresponding chemical substances.

[0029] The category detection model is used to classify the garbage and determine the classification to which the garbage belongs, such as recyclable garbage, non-recyclable garbage, hazardous garbage and other types.

[0030] The image discrimination module is used to collect the sample image corresponding to the trash can, and use the composition detection model to analyze the feature positioning points and feature description information of the trash can in the sample image.

[0031] The garbage state judgment module is used to process the feature positioning points and feature description information of the trash can in the sample image, and use the category detection model to determine the state index of the trash can under the corresponding garbage state.

[0032] The path planning module is used to determine the proportion of the garbage distribution in the trash can according to the state index of the trash can under the corresponding garbage state, and set the accommodation path of the trash can according to the garbage distribution proportion.

[0033] In an embodiment of the present invention, the object detection module mainly identifies the trash can from the preset direction of the trash can, and sets the composition detection model and the category detection model according to the identified situation.

[0034] The preset direction of the trash can represents the opening direction and angle of the trash can placed in a normal environment. According to this direction, the garbage truck can capture an image of the garbage in the trash can and identify both the trash can and the garbage at the same time.

[0035] Generally, the garbage truck will be equipped with cameras on both sides of the vehicle body. The cameras can first identify the trash can, then judge what kind of garbage is placed in the current trash can according to the image of the trash can, then view the image inside the trash can to identify the height of the garbage in the trash can, and finally collect the garbage to complete the treatment of the garbage. In this treatment process, the trash can mainly serves as the carrier to be identified. After completing the steps of discovering and inspecting the garbage, the garbage is poured into the garbage truck to complete the treatment.

[0036] However, when dealing with garbage, it is necessary to first identify the corresponding categories of garbage, control the garbage disposal according to the identified corresponding situations, and set a series of feature points to describe the garbage disposal process, so that the garbage disposal process can better meet the expected effects.

[0037] When detecting a trash can, collect the image information corresponding to the trash can, and set the composition detection model and category detection model corresponding to the current trash can according to the image information; these two models are used to assist in judging the garbage in the trash can and the trash can itself.

[0038] The target detection module is implemented by using a target detection framework, such as using algorithms like YOLO, Faster R-CNN, etc., to identify the scene where the trash can is located in the current image, and to identify the materials of the trash can and the garbage according to the colors, shapes and textures in the image; set the composition detection model related to the trash can.

[0039] The category detection model is implemented by classifying the features recognized within the target detection framework to obtain the corresponding classifications of the current trash can and the garbage, such as household garbage, office garbage, medical garbage, etc. This helps to understand the generation situation of garbage and formulate targeted treatment strategies.

[0040] In the process of the target detection module setting two detection models, the target detection module essentially needs to identify the appearance of the trash can and multiple points existing inside the trash can. At this time, the points with existence flags, existence shadows and corresponding heights will be used as the main points for analysis, and the response content corresponding to these points will be selected from the preset information, and two detection models will be set according to the response between the data, thus completing the process of preliminary target detection.

[0041] As Figure 2 shown, the implementation method of the target detection module includes: obtaining the trash can contour image and the trash can internal image of the trash can in the preset direction; the trash can contour image represents the content including the appearance contour of the trash can, mainly distinguishing the style of the trash can appearance contour, and the trash can internal image can preliminarily estimate the type and capacity of the garbage in the trash can.

[0042] According to the trash can contour image, extract the color and shape of the trash can, and set the RFID tag corresponding to the trash can. The RFID tag will contain the appearance of the trash can, record the location and area of the trash can with this appearance, and at the same time, when cleaning the garbage, relevant information will be recorded on the RFID tag corresponding to the current trash can to facilitate subsequent recording and storage of the emptying time, collection times, transportation routes, etc. of the trash can.

[0043] Based on the internal image of the trash can, identify the image elevation points and shadow points in the current trash can. The image elevation points represent the points indicating the height of the trash in the trash can in the image, and the shadow points represent the points with an obvious decrease in pixel values and points close to black in the image when taking pictures of the trash. These shadow points are likely to cover hidden features. At this time, the shadow points and the image elevation points are co - marked to identify the corresponding situation of the trash in the image.

[0044] In response to the image elevation points in the trash can, obtain the segmented images adjacent to each image elevation point, and obtain at least the feature information in one segmented image; at this time, the obtained image elevation points are segmented according to their positions to obtain multiple segmented images representing the height of the trash in the trash can, and the feature information is extracted from the segmented images, and these feature information will represent the relevant situation of the trash in the trash can.

[0045] Based on the feature information of the segmented images, obtain the clustering points corresponding to the feature information from the internal image of the trash can, and generate the simulation model information corresponding to the clustering points; the simulation model information clusters the trash in the internal image of the trash can to describe the distribution and form of various types of trash or substances in the trash can. The implementation method of generating the simulation model information corresponding to the clustering points is as follows: obtain the mapping points of each clustering point in the forward and reverse directions of the trash can, correspond the mapping points to the shadow points in the internal image of the trash can, calculate the distance between the mapping points and the shadow points, and cluster the mapping points according to the shadow points to obtain multiple clustering centers, stack the multiple clustering centers, and set the simulation model information according to the stacked clustering centers.

[0046] According to the simulation model information, set up the component detection model and the category detection model. When setting up the component detection model and the category detection model, by using the main content in the simulation model information as the verification method, select the models mainly corresponding to the currently identified trash, and use these models to further judge the trash components to increase the feature description of the trash in the trash can.

[0047] The implementation method of setting up the component detection model and the category detection model according to the simulation model information also includes: recording the simulation model information, determining the time threshold for obtaining the simulation model information, collecting the trash can contour image and the internal image of the trash can according to the time threshold, updating the trash can contour image and the internal image of the trash can, and updating the background of the trash can contour image and the internal image of the trash can, recording the background difference value corresponding to the trash can contour image and the internal image of the trash can, and selecting the component detection model and the category detection model according to the background difference value of the trash can contour image and the internal image of the trash can.

[0048] At this time, the time threshold represents the time for collecting simulation model information, which will be expressed in the form of a time threshold. Then, the data within this time threshold is calculated to update the trash can contour image and the trash can interior image. After the update, there will be certain changes in the trash can contour image and the trash can interior image. The background information existing in these changes is extracted, and the difference between this background information is calculated, that is, the pixel point difference of the background information is calculated to obtain the background difference value. Then, the component detection model and the category detection model are selected according to the size of the background difference value.

[0049] This method for background processing is essentially to establish a background model for the position of each trash can, that is, what the trash can and its surrounding environment look like when there is no garbage. When there are new contour images and interior images, they are compared with the background model to calculate the difference value between the two. Multiple methods such as pixel-level difference, edge detection difference, and color histogram difference can be used. The above difference value is quantified into one or more metrics to measure the degree of change between the current image and the background model.

[0050] Then, according to the pre-set rules, the most suitable component detection model and category detection model are selected based on the background difference value. For example, if the difference is large, it may mean an increase in the amount of garbage or a change in the type, and at this time, a more sensitive or more accurate model should be selected. As time and the environment change, continuously monitor the background difference value and dynamically adjust the models used according to the actual situation to ensure the best performance. Finally, the most suitable model for judging the internal components of the current trash can and the corresponding situation can be selected, so as to quickly complete the collection of trash can information and the collection of garbage by the garbage truck according to the different internal states of the trash can; such processing can ultimately improve the effect of garbage classification and treatment to adapt to more scenarios for garbage treatment.

[0051] In an embodiment of the present invention, the image discrimination module is mainly used to discriminate the images collected from the trash can, clarify the image situation of the existence of garbage in the trash can, and describe the information in the image using corresponding features.

[0052] In the image discrimination module, sample images of the trash can will be further collected, and the trash can in the sample images will be described by features according to the obtained sample images. Feature positioning points are set in the trash can. The feature positioning points are used to discriminate the positioning of various garbage in the trash can and group and describe the components corresponding to the garbage to obtain different garbage categories. Finally, the garbage categories are output as feature description information.

[0053] Such as Figure 3As shown in the figure, the implementation process of the image discrimination module includes: dividing the sample image into multiple sub-images of equal size according to the scene of the trash can, and calculating the garbage distribution probability of the garbage in the sub-image under the corresponding scene; the garbage distribution probability is used to represent the probability of the garbage appearing in different positions; for example, when the garbage truck collects the garbage in the trash can, the garbage is placed in multiple trash cans in a certain order, and the garbage distribution probability represents the probability of a certain type of garbage gathering at the corresponding position of the trash can.

[0054] Identify the position of the garbage in each sub-image under the garbage distribution probability, and add feature positioning points to the garbage at each position; according to the information corresponding to the feature positioning points, set the feature description information of the trash can.

[0055] At this time, the feature description information will include local features around the feature positioning points, such as color, texture, shape, etc.; in order to improve the coverage of the feature description information for the corresponding data in the trash can, the processing method of setting the feature description information of the trash can also includes: tracking the garbage distribution probability on the feature positioning points to generate the change trend of the feature positioning points; the change trend of the feature positioning points will be set according to the garbage distribution probability on the feature positioning points, and generate the change trend related to the increase value and decrease value of the garbage distribution probability of the corresponding feature points at different times.

[0056] Determine the priority of the garbage distribution under the change trend of the feature positioning points, and calculate the image similarity of the top position and the bottom position of the trash can according to the priority of the garbage distribution. Here, the priority of the garbage distribution is represented by the trend amount identified by the change trend of the feature positioning points, that is, the slope value of the change trend at the corresponding time is used as the priority of the garbage distribution at this time, and the garbage distribution probability at this time is sorted according to the priority, and the image similarities of the corresponding feature positioning points at the top position and the bottom position of the trash can are calculated in turn according to the priority size.

[0057] The image similarity of the top position is used to calculate the pixel point values with similarity at the corresponding garbage distribution probability of the feature positioning points. The image similarity of the bottom position represents the similarity value after the trash can is collected, and this value will represent the situation of the garbage in the trash can itself during the dumping process.

[0058] The image similarity at the top position is represented by the similarity between the sub-image where the feature positioning points are located and the preset garbage image at the corresponding position. This similarity is represented by the mean square error of the pixel point values of the sub-image and the preset garbage image; the image similarity of the bottom image is also compared with the preset garbage image and represented by the mean square error of the pixel point values. At the same time, the bottom image is preferably calculated after the garbage is completely contained, because only after the garbage is completely contained can the corresponding image be obtained. The bottom image is used to identify whether the garbage is completely contained. The preset garbage image is set according to the images of the garbage normally placed in the trash can. These images will be pre-annotated, annotating the garbage components and appearances in the preset garbage image. At the same time, an estimated volume value of the garbage in the trash can will be set according to the position where the garbage exists in the preset garbage image, which is convenient for assisting in judging the amount of garbage that can be contained after the garbage recognition is completed later.

[0059] Compare the sub-image corresponding to the trash can with the preset garbage image according to the image similarity at the top position and the image similarity at the bottom position of the trash can to determine the garbage components and the garbage volume in the trash can; and set the feature description information of the trash can according to the garbage components and the garbage volume in the trash can.

[0060] At this time, after obtaining the image similarity at the top position and the image similarity at the bottom position of the trash can, the form of comparing the sub-image of the trash can with the preset garbage image is to find a set of preset garbage images that are most similar to the current trash can according to the value of the image similarity, and find the garbage components and the volume of the garbage corresponding to the current trash can from the preset garbage images, and use these data as feature description information for subsequent path planning and corresponding classification of the trash can. When the preset garbage image finds the garbage components and the volume of the garbage corresponding to the current trash can, the corresponding image can be input into the component detection model, and then the output result of the component detection model is used as the identified garbage components, and the estimated volume of the garbage is found according to this component, so as to complete the construction of the feature description information.

[0061] The finally output feature description information not only includes local features around the feature positioning points, such as color, texture, shape, etc., but also includes the image similarity at the top position of the feature positioning point in the trash can, the image similarity at the bottom position of the trash can, the garbage components and the garbage volume in the trash can, and these data are comprehensively regarded as feature description information to improve the processing effect of subsequent situations related to the trash can.

[0062] In an embodiment of the present invention, the garbage state judgment module mainly uses a category detection model to identify the garbage state in the current trash can, and uses the garbage state to describe the current trash can and set state indicators.

[0063] In the garbage status judgment module, the garbage status of the trash can will be in the form of multiple scores, quantifying the situation of the trash can in different states, and identifying the actual capacity and corresponding classification in the trash can according to these contents, so as to reduce the problem of incorrect judgment of garbage capacity and improve the comprehensive recognition effect of garbage.

[0064] As Figure 4 shown, the implementation method of the garbage status judgment module includes: starting from each feature positioning point, inputting the feature description information into the category detection model, and sequentially determining the garbage status of each feature positioning point according to the output result of the category detection model; the garbage status includes garbage type, garbage distribution situation, garbage existence probability, and garbage capacity status.

[0065] Evaluate the garbage status of each feature positioning point, and sequentially obtain the type score of the feature positioning point regarding the garbage type, the distribution score regarding the garbage distribution situation, the probability score regarding the garbage existence probability, and the capacity status score regarding the garbage capacity.

[0066] Integrate the type score, distribution score, probability score, and capacity status score, and calculate the status index of the trash can in the corresponding garbage status.

[0067] The garbage type indicates the types of garbage existing near the feature positioning point, such as recyclable garbage, non-recyclable garbage, hazardous garbage, etc. The category detection model will predict the most likely garbage type according to the feature description information such as color, texture, shape, etc., and classify these garbage.

[0068] The garbage distribution situation describes the spatial distribution of garbage in the trash can, including the degree of aggregation, density, etc. By analyzing the data of multiple feature positioning points, a spatial layout map of the garbage in the entire trash can can be constructed.

[0069] The garbage existence probability represents the probability value that a specific type of garbage appears near the feature positioning point, usually a value between 0 and 1. This probability can help evaluate the possibility of the existence of garbage and reduce the risk of misjudgment.

[0070] The garbage capacity status refers to the current filling degree or remaining space of the trash can, such as whether it is close to overflowing. By comprehensively considering the garbage distribution situation and existence probability of multiple feature positioning points, the overall capacity status of the trash can can be inferred.

[0071] The type score regarding the garbage type will set a basic score according to the different garbage types at the feature positioning point. This basic score will represent the type scores of different garbage, and the basic score will be set in the range of 0 to 1.

[0072] The distribution score regarding the garbage distribution will set a distribution-related score according to the actual garbage distribution. For example, it is set according to the standard deviation of the garbage distribution at each feature positioning point, or by multiplying the image similarity of the top position of the garbage bin recognized during garbage distribution by the reciprocal of the standard deviation of the garbage distribution density at the current feature positioning point to obtain the distribution score regarding the garbage distribution. The garbage distribution density is a density value calculated based on the number of garbage existing in the area corresponding to the feature positioning point. This distribution score will represent the similarity between the actual distribution and the preset garbage model, and finally obtain a relatively clear score value. The final distribution score will be set as a score value with a value range of 0 to 1.

[0073] The probability score regarding the garbage presence probability will set a score value according to the probability of a specific type of garbage appearing at this feature positioning point. This score value will represent the normalized value of the probability of a specific type of garbage appearing at this feature positioning point, making its own value within the range of 0 to 1.

[0074] The capacity status score regarding the garbage capacity will set a score according to the proportion of the garbage in the trash can to the total volume, determine whether the garbage overflows, and set a score for the capacity at this time. For example, multiply the proportion of the garbage in the trash can to the total volume by the type score as the used capacity status score.

[0075] The final status index is expressed as: SI = w TS ×TS + w DS ×DS + w PS ×PS + w CS ×CS; where SI represents the status index, TS represents the type score, DS represents the distribution score, PS represents the probability score, CS represents the capacity status score; w TS represents the weight of the type score, w DS represents the weight of the distribution score, w PS represents the weight of the probability score, w CS represents the weight of the capacity status score; among them, the weights of the type score, distribution score, probability score, and capacity status score are set to 0.3, 0.2, 0.2, and 0.3 in sequence.

[0076] In an embodiment of the present invention, the path planning module is mainly used to obtain the proportion of the garbage distribution in the trash can from the status index according to the status index of the trash can under the corresponding garbage state, and then set the path for the trash can to accommodate according to the garbage distribution proportion.

[0077] The garbage distribution ratio is used to describe the amount of garbage in the trash can at different locations and garbage states. This garbage distribution ratio will first determine the ratio type and describe the distribution of garbage in the trash can according to the ratio type. Then, according to this distribution, the current garbage truck's collection path for the trash can will be planned, so as to complete the treatment of the garbage.

[0078] As Figure 5 shown, the implementation method of the path planning module includes: dividing the obtained trash can coordinates into multiple sub-regions according to the state indicators of the trash can in the corresponding garbage state.

[0079] Calculate the garbage distribution ratio of the trash cans in the current sub-region, identify the positions of the trash cans according to the size of the garbage distribution ratio, and set the distribution result vector of the trash cans.

[0080] Generate multiple moving paths according to the distribution result vector of the trash cans, calculate the probability values of the distribution result vector on the moving paths, and set the moving path with the maximum probability value of the distribution result vector on the moving path as the collection path of the trash can.

[0081] The above-mentioned garbage distribution ratio will be the ratio of the capacity and existence probability of the corresponding garbage in the trash can to the total capacity and existence probability under the type score, distribution score, probability score, and capacity score in the state indicators; at this time, identify the scenario of the trash can in a normal community or the location of garbage storage; at this time, there will be multiple trash cans in a sub-region. At this time, it is necessary to identify the garbage that the current garbage truck can actually collect, and at the same time, according to the corresponding situation of the trash can, determine how to collect the garbage in the community; when collecting, the garbage truck will not start collecting from the beginning, but enter the innermost position of the community along the location where the trash cans exist. Then, the garbage conditions in multiple sub-regions will be obtained, and according to these data, the collection path can be planned to collect the trash cans that are about to overflow with the highest efficiency, thus completing the problem of trash can identification and processing.

[0082] The distribution result vector will be represented as the trash can ID, location, and garbage distribution ratio, and these three data will be used as the distribution result vector. And generate moving paths according to the distribution of the distribution result vector. Each moving path contains multiple distribution result vectors, and each distribution result vector represents a node on the moving path. The nodes on the moving path are all reachable, and the moving distance between the nodes on the moving path is the shortest; then calculate the probability values of the distribution result vectors on the moving path. The probability value of the distribution result vector represents the probability that the current distribution result vector appears on the corresponding moving path. Finally, set the situation where the probability value of the distribution result vector appearing on the moving path is the largest as the collection path of the trash can to complete the collection of the trash can.

[0083] As Figure 6As shown in the figure, the present invention also provides a method for identifying trash cans based on in-vehicle video, including: S1, detecting trash cans along the side of the vehicle body in a preset direction, obtaining the contour image and the internal image of the trash can, extracting the color and shape of the trash can, and setting RFID tags for identification.

[0084] S2, analyzing the elevation points and shadow points in the internal image of the trash can, generating a segmented image and extracting feature information, creating simulation model information about clustering points; based on the simulation model information, setting up a component detection model and a category detection model.

[0085] S3, collecting sample images of the trash can, dividing the sample images into multiple sub-images, calculating the garbage distribution probability, identifying the garbage at each position and adding feature positioning points, and setting feature description information.

[0086] S4, using the component detection model, tracking the change trend of the garbage distribution probability at the feature positioning points, determining the priority of the garbage distribution, calculating the image similarity between the top position and the bottom position of the trash can, and determining the garbage component and the volume.

[0087] S5, starting from each feature positioning point, inputting the feature description information into the category detection model to output the garbage state; evaluating the state of each feature positioning point to obtain a type score, a distribution score, a probability score, and a capacity score, and comprehensively calculating to obtain a state index.

[0088] S6, dividing the trash can into multiple sub-regions according to the state index, calculating the proportion of the garbage distribution in each sub-region of the trash can to form a distribution result vector; generating multiple moving paths according to the distribution result vector, calculating the path probability values, and selecting the path corresponding to the maximum probability value as the collection path.

[0089] Although the embodiments of the present invention have been shown and described above, it can be understood that the above embodiments are exemplary and should not be construed as limiting the present invention. Those of ordinary skill in the art can make changes, modifications, substitutions, and variations to the above embodiments within the scope of the present invention, and still be covered by the protection scope of the present invention.

Claims

1. A trash can recognition system based on vehicle-mounted video, characterized in that: include: The target detection module is used to detect the trash cans on the side of the vehicle body according to the preset direction of the trash cans, and set the component detection model and category detection model of the trash cans; An image recognition module is used to collect sample images corresponding to the trash cans and use a component detection model to analyze the feature positioning points and feature description information of the trash cans in the sample images; The garbage state judgment module is used to process the feature positioning points and feature description information of the garbage bin in the sample image, and use the category detection model to determine the state index of the garbage bin in the corresponding garbage state; The path planning module is used to determine the garbage distribution ratio in the garbage bin according to the status indicators of the garbage bin in the corresponding garbage state, and set the collection path of the garbage bin according to the garbage distribution ratio.

2. The trash can identification system based on vehicle-mounted video according to claim 1, characterized in that: The implementation of the target detection module includes: Obtaining a trash can outline image and an interior image of the trash can in a preset direction; According to the outline image of the trash can, the color and shape of the trash can are extracted, and the RFID tag corresponding to the trash can is set; Based on the internal image of the trash can, identify the image elevation points and shadow points in the current trash can; In response to the image elevation points in the trash can, the segmented images adjacent to each image elevation point are obtained, and feature information in at least one segmented image is obtained; Based on the feature information of the segmented image, cluster points corresponding to the feature information are obtained from the internal image of the trash can, and simulation model information corresponding to the cluster points is generated; According to the simulation model information, set the component detection model and the category detection model.

3. The trash can identification system based on vehicle-mounted video according to claim 2 is characterized in that: The implementation method of generating simulation model information corresponding to cluster points is: Get the mapping points of each cluster point in the forward and reverse directions of the trash can, correspond the mapping points to the shadow points in the image inside the trash can, calculate the distance between the mapping points and the shadow points, cluster the mapping points according to the shadow points, get multiple cluster centers, superimpose the multiple cluster centers, and set the simulation model information according to the superimposed cluster centers.

4. The trash can identification system based on vehicle-mounted video according to claim 2 is characterized in that: According to the simulation model information, the implementation method of setting the component detection model and the category detection model also includes: The simulation model information is recorded, and a time threshold for obtaining the simulation model information is determined. According to the time threshold, a trash can outline image and a trash can interior image of the trash can are collected, and the trash can outline image and the trash can interior image are updated. The background difference values ​​corresponding to the trash can outline image and the trash can interior image are recorded, and a component detection model and a category detection model are selected according to the background difference values ​​between the trash can outline image and the trash can interior image.

5. The trash can identification system based on vehicle-mounted video according to claim 1, characterized in that: The implementation process of the image discrimination module includes: The sample image is divided into multiple sub-images of equal size according to the scene of the trash can, and the garbage distribution probability of the garbage in the sub-image in the corresponding scene is calculated; Identify the location of garbage in each sub-image under the probability of garbage distribution, and add feature positioning points to the garbage at each location; set the feature description information of the garbage bin according to the information corresponding to the feature positioning points.

6. The trash can identification system based on vehicle-mounted video according to claim 5, characterized in that: The processing methods for setting the characteristic description information of the trash can also include: Track the probability of garbage distribution at the feature positioning points and generate the change trend of the feature positioning points; Determine the priority of garbage distribution under the change trend of the feature positioning point, and calculate the image similarity of the top position and the image similarity of the bottom position of the garbage bin according to the priority of the garbage distribution; According to the image similarity at the top position and the image similarity at the bottom position of the trash can, the sub-image corresponding to the trash can is compared with the preset trash image to determine the composition and volume of the trash in the trash can; and according to the composition and volume of the trash in the trash can, the feature description information of the trash can is set.

7. The trash can identification system based on vehicle-mounted video according to claim 1, characterized in that: The implementation methods of the garbage status judgment module include: Starting from each feature location point, the feature description information is input into the category detection model. According to the output result of the category detection model, the garbage status of each feature location point is determined in turn; the garbage status includes garbage type, garbage distribution, garbage existence probability and garbage capacity status; Evaluate the garbage status of each feature location point, and sequentially obtain the type score of the feature location point regarding the garbage type, the distribution score regarding the garbage distribution, the probability score regarding the probability of garbage existence, and the capacity status score regarding the garbage capacity; The type score, distribution score, probability score and capacity status score are combined to calculate the status index of the trash can in the corresponding garbage state.

8. The trash can identification system based on vehicle-mounted video according to claim 7, characterized in that: The status indicator is expressed as: SI=w TS ×TS+w DS ×DS+w PS ×PS+w CS ×CS; Among them, SI represents the status index, TS represents the type score, DS represents the distribution score, PS represents the probability score, and CS represents the capacity status score; w TS represents the weight of the type score, w DS represents the weight of the distribution score, w PS represents the weight of the probability score, w CS Indicates the weight of the capacity score.

9. The trash can identification system based on vehicle-mounted video according to claim 1, characterized in that: The implementation methods of the path planning module include: According to the state indicators of the trash cans in the corresponding trash states, the obtained trash can coordinates are divided into a plurality of sub-areas; Calculate the garbage distribution ratio of the garbage bins in the current sub-area, identify the locations of the garbage bins according to the garbage distribution ratio, and set the distribution result vector of the garbage bins; A plurality of moving paths are generated according to the distribution result vector of the trash can, the probability value of the distribution result vector on the moving path is calculated, and the moving path with the maximum probability value of the distribution result vector on the moving path is set as the receiving path of the trash can.

10. A method for identifying trash cans based on vehicle-mounted video, using the trash can identification system based on vehicle-mounted video as claimed in claim 1, characterized in that: include: S1, detect the trash can on the side of the vehicle in a preset direction, obtain the outline image and the internal image of the trash can, extract the color and shape of the trash can, and set an RFID tag for identification; S2, analyzing the elevation points and shadow points in the internal image of the trash can, generating a segmented image and extracting feature information, creating simulation model information about clustering points; based on the simulation model information, setting a component detection model and a category detection model; S3, collect sample images of the trash can, divide the sample images into multiple sub-images, calculate the probability of garbage distribution, identify the garbage at each location and add feature positioning points, and set feature description information; S4, using the component detection model, tracks the probability change trend of garbage distribution at the feature positioning point, determines the priority of garbage distribution, calculates the image similarity of the top and bottom positions of the garbage bin, and determines the garbage composition and volume; S5, starting from each feature location point, input feature description information into the category detection model and output the garbage status; evaluate the status of each feature location point to obtain the type score, distribution score, probability score and capacity score, and comprehensively calculate the status index; S6, dividing the trash can into multiple sub-areas according to the state index, calculating the garbage distribution ratio of the trash can in each sub-area, and forming a distribution result vector; generating multiple moving paths according to the distribution result vector, calculating the path probability value, and selecting the path corresponding to the maximum probability value as the receiving path.

Images