A mobile vehicle-mounted intelligent garbage information management method and system

Abstract

The present invention relates to the field of artificial intelligence technology, and provides a mobile vehicle-mounted intelligent garbage information management method and system, including: in the case where the type of garbage cannot be identified, matching the garbage picture information put in with the garbage picture information stored in advance. If no garbage picture information of the same category can be matched, store its garbage information and use it to judge whether the garbage is successfully put in later; if garbage picture information of the same category is matched, judge the correctness of the current garbage placement according to the garbage classification categories corresponding to the garbage picture information stored in advance. The present invention can adaptively update the garbage picture information in the database, and without the need to identify the specific type of garbage, it can further realize the judgment of the classification results of newly added types of garbage, with high generalization ability and strong evaluation reliability.

Description

Technical Field

[0001] The present invention relates to the field of artificial intelligence, and particularly to a mobile vehicle-mounted intelligent garbage information management method and system. Background Art

[0002] With the rapid economic development and the improvement of people's living standards, the amount of garbage has inevitably increased. The main ways of garbage disposal are centralized incineration or burial, which fails to make better use of garbage resources and results in low utilization efficiency of garbage. If garbage can be identified and classified and its recyclable properties can be fully utilized, the resource utilization rate can be improved.

[0003] The existing intelligent garbage classification methods mainly identify and classify garbage scanned images through deep learning and other means. For example, the patent document with the patent publication number CN108455127B discloses an intelligent classification trash can, etc. However, with the increasing variety of garbage today, the databases or training sets of artificial intelligence cannot fully cover all types, and due to the various forms of garbage, the neural network cannot achieve good generalization ability.

[0004] Currently, the classification awareness of garbage throwers still needs to be improved, and the classification of garbage is not yet reasonable enough. There are often mistakes in garbage classification, resulting in the failure to fully utilize garbage resources. In order to improve the classification accuracy and better utilize garbage resources, it is necessary to visualize the results of garbage classification work so as to evaluate the work of garbage throwers. Summary of the Invention

[0005] The purpose of the present invention is to provide a mobile vehicle-mounted intelligent garbage information management method and system, which are used to solve the problem that with the increase in the variety of garbage, the existing garbage classification evaluation methods cannot reliably evaluate the garbage classification effect.

[0006] To solve the above technical problems, the technical solutions adopted by the present invention are specifically as follows:

[0007] The present invention provides a mobile vehicle-mounted intelligent garbage management method, including the following steps:

[0008] Identify the identity information of the thrower, and obtain the garbage picture information and garbage classification category of the thrown garbage;

[0009] Identify the type of garbage according to the garbage picture information, and determine whether the type of garbage can be identified;

[0010] If the type of garbage cannot be identified, obtain the pre-stored garbage picture information, the garbage classification category and garbage classification confidence corresponding to each garbage picture information;

[0011] Match the information of the garbage pictures to be put in with the information of each garbage picture stored in advance, and determine whether garbage picture information of the same category is matched;

[0012] If garbage picture information of the same category cannot be matched, set an initial classification confidence setting threshold for the garbage picture information to be put in, store the classification category of the garbage to be put in, the garbage picture information, and the corresponding initial classification confidence setting threshold, and determine that the current garbage disposal is correct;

[0013] If garbage picture information of the same category is matched, determine whether the classification confidence corresponding to the matched garbage picture information of the same category is greater than the classification confidence setting threshold;

[0014] If it is greater than the classification confidence setting threshold, determine whether the classification category of the garbage to be put in is the same as the classification category corresponding to the matched garbage picture information of the same category;

[0015] If the classification category of the garbage to be put in is the same as the classification category corresponding to the matched garbage picture information of the same category, determine that the current garbage disposal is correct; otherwise, determine that the current garbage disposal is incorrect.

[0016] Further, if the classification category of the garbage to be put in is the same as the classification category corresponding to the matched garbage picture information of the same category, increase the classification confidence corresponding to the matched garbage picture information of the same category; if the classification category of the garbage to be put in is different from the classification category corresponding to the matched garbage picture information of the same category, decrease the classification confidence corresponding to the matched garbage picture information of the same category.

[0017] Further, the formula for increasing the classification confidence corresponding to the matched garbage picture information of the same category is:

[0018]

[0019] The formula for decreasing the classification confidence corresponding to the matched garbage picture information of the same category is:

[0020]

[0021] Among them, is the classification confidence before update, is the classification confidence after update, is the memory coefficient, set to 0.95.

[0022] Further, the step of matching the garbage picture information to be put with the pre-stored garbage picture information to determine whether the same type of garbage picture information is matched includes:

[0023] Process the garbage picture information to be put to obtain the corresponding shape features, structural texture features, and gray texture features;

[0024] Perform similarity matching one by one on the obtained corresponding shape features, structural texture features, and gray texture features with the shape features, structural texture features, and gray texture features corresponding to the pre-stored garbage picture information respectively to obtain the shape feature similarity, structural texture feature similarity, and gray texture feature similarity;

[0025] According to the shape feature similarity, structural texture feature similarity, and gray texture feature similarity, obtain the final garbage type similarity;

[0026] If the final garbage type similarity is greater than the set threshold of the garbage type similarity, it is determined that the garbage picture information to be put and the pre-stored garbage picture information are of the same type of garbage picture information, otherwise it is determined that the garbage picture information to be put and the pre-stored garbage picture information are not of the same type of garbage picture information.

[0027] Further, the step of processing the garbage picture information to be put to obtain the corresponding shape features, structural texture features, and gray texture features includes:

[0028] The garbage picture information includes an RGB garbage image and a depth garbage image. According to the RGB garbage image, a gray image is obtained;

[0029] Take the depth garbage image as the guiding image and the RGB garbage image as the original image, perform guided filtering to obtain an output image, and the output image is the shape feature;

[0030] Subtract the output image from the guiding image to obtain the structural texture feature;

[0031] Subtract the output image from the original image to obtain the gray texture feature.

[0032] Further, the step of obtaining the final garbage type similarity according to the shape feature similarity, structural texture feature similarity, and gray texture feature similarity includes:

[0033] Perform weighted summation on the shape feature similarity, structural texture feature similarity, and gray texture feature similarity to obtain the final garbage type similarity, where the weight values corresponding to the shape feature similarity, structural texture feature similarity, and gray texture feature similarity are the first weight value, the second weight value, and the third weight value respectively.

[0034] Further, if the similarity of the final garbage types is greater than the set threshold of the garbage type similarity, determine the minimum similarity among the shape feature similarity, the structural texture feature similarity, and the gray texture feature similarity, reduce the weight value corresponding to the minimum similarity, and correspondingly increase the weight values corresponding to the other two similarities.

[0035] Further, the formula for reducing the weight value corresponding to the minimum similarity is:

[0036]

[0037] The formula for correspondingly increasing the weight values corresponding to the other two similarities is:

[0038]

[0039] Among them, is the weight value corresponding to the minimum similarity, is the weight value obtained after reducing the weight value corresponding to the minimum similarity, is the weighted sum value of the other two similarities, 、 are the weight values corresponding to the other two similarities, 、 are the weight values obtained after increasing the weight values corresponding to the other two similarities, 、 are the other two similarities.

[0040] Further, identify the identity information of the delivery personnel, and give corresponding evaluation indicators for the classification work of the delivery personnel within a certain period of time. The calculation formula of the evaluation indicator is:

[0041]

[0042] gives corresponding evaluation indicators for the classification work of the garbage delivery personnel within a certain period of time, is the total number of correct garbage classifications by the same delivery personnel within a certain period of time, is the total number of garbage classifications by the same delivery personnel within a certain period of time.

[0043] The present invention also provides a mobile vehicle-mounted intelligent garbage management system, including a processor and a memory. The processor is used to process the instructions stored in the memory to implement the mobile vehicle-mounted intelligent garbage information management method.

[0044] The present invention has the following beneficial effects:

[0045] The present invention matches the information of the garbage pictures put in with the information of each garbage picture stored in advance. If no garbage picture information of the same category can be matched, the information of the garbage picture put in will be stored and used for the matching of the garbage picture information put in next time, so as to update the database automatically. If garbage picture information of the same category is matched, when the confidence level of the garbage classification corresponding to the pre-stored garbage picture information is relatively high, the garbage classification category corresponding to the pre-stored garbage picture information will be directly used as the standard for correct classification, and then the correctness of the garbage classification category put in this time will be judged, without the need to identify the specific type of garbage. Since the present invention can adaptively update the garbage picture information in the database as the types of garbage increase, and can judge the classification result of the newly added types of garbage without identifying the specific types of garbage, it has a relatively high generalization ability and strong evaluation reliability. Brief Description of the Drawings

[0046] In order to more clearly illustrate the technical solutions and advantages in the embodiments of the present invention or the prior art, the following will briefly introduce the drawings required to be used in the description of the embodiments or the prior art. Obviously, the drawings in the following description are only some embodiments of the present invention. For those of ordinary skill in the art, other drawings can be obtained based on these drawings without creative efforts.

[0047] Figure 1 is the flowchart of the mobile vehicle-mounted intelligent garbage information management method of the present invention;

[0048] Figure 2 is the top view of the mobile vehicle-mounted intelligent garbage information management device of the present invention;

[0049] Figure 3 is the bottom view of the mobile vehicle-mounted intelligent garbage information management device of the present invention. Detailed Embodiment

[0050] In order to further elaborate on the technical means and effects adopted by the present invention to achieve the predetermined invention purpose, the following will, in combination with the accompanying drawings and preferred embodiments, elaborate in detail on a mobile vehicle-mounted intelligent garbage information management method and device proposed according to the present invention, its specific implementation manner, structure, features and effects.

[0051] Method Embodiment:

[0052] This embodiment provides a mobile vehicle-mounted intelligent garbage information management method, and the flowchart corresponding to this method is as Figure 1As shown, this method identifies the identity information of waste disposal personnel and the waste image information of the disposed waste, and then evaluates the waste classification work based on the disposed waste image information in combination with the pre-stored waste image information and their corresponding waste classification categories. As the types of waste increase, this method can adaptively update the waste image information in the database, and without the need to identify the specific type of waste, it can further achieve the judgment of the classification results of newly added types of waste, with high generalization ability and strong evaluation reliability.

[0053] To implement this mobile vehicle-mounted intelligent waste information management method, this embodiment also provides a device, which is provided with: an RFID identity recognition module, a waste image capture module, a waste classification work evaluation module, and a waste weighing module.

[0054] Among them, the RFID identity recognition module: collects the personnel image information through the front-facing camera and outputs it to the RFID recognition sub-module. At the same time, it generates RFID trigger information to activate the RFID recognition sub-module, and the RFID recognition sub-module identifies the collected personnel image to obtain the personnel identity information.

[0055] The waste image capture module: collects the surface image of the trash can through the overhead camera and extracts its waste picture and waste category information.

[0056] The waste classification work evaluation module: obtains the identity information of the waste disposal personnel through the RFID identity recognition module, and obtains the waste picture information through the waste image capture module, and then evaluates the waste classification work done by the waste disposal personnel.

[0057] The waste weighing module: combines the waste image capture module and the electronic weighing device deployed on the device to simply obtain rough data of the waste weight through image perception.

[0058] The top view and bottom view of the device are respectively as Figure 2 and Figure 3As shown in the figure, the device includes a bottom tabletop, a vertical support surface, and a top tabletop. The bottom tabletop is set on the ground, and the upper part is used to place trash cans. The lower part of the vertical support surface is fixed to one side edge of the bottom tabletop, and its upper part is fixed to one side edge of the connecting top tabletop. The bottom tabletop and the top tabletop are arranged in parallel and are both located on the same side of the vertical support surface. Among them, in order to obtain the identity information of the waste thrower, the front-facing camera is set at a position slightly above the middle of the vertical support surface of the device facing the trash can. By setting the front-facing camera at this position, during the process of the waste thrower walking towards the device, the front-facing camera can just face the waste thrower directly, so as to collect the face image of the waste thrower. In order to obtain the waste picture, the overhead camera is set at the center of the bottom of the top tabletop. During the process of the waste thrower throwing waste, the overhead camera can just collect the waste image information in the trash can. Each module in the device is interconnected and is used to implement the above-mentioned mobile vehicle-mounted intelligent waste information management method, including the following steps:

[0059] (1)Identify the identity information of the thrower, and obtain the waste picture information and waste classification category thrown.

[0060] The face image collected by the front-facing camera is processed by the human key point detection network, and the network outputs a heat map. The heat map contains the identified human key points. With two key points of the left and right shoulders and the human center key point, the three points form a triangle. Using the heat map as the content of the triangle to represent the basic human outline of the person. This way of using the triangle content to represent the basic human outline of the person, that is, using an image (RGB image) to achieve distance judgment, is beneficial to reducing the equipment cost and does not require a depth camera. Specifically, when a person moves towards this device, the area of the basic human outline image of the person will continue to increase. In order to determine whether this person is a waste thrower, the front-facing camera collection rate is set to 5 frames per second, and 20 frames are used as the time threshold When the continuous increase duration of the basic human outline of the person is greater than or equal to the time threshold it is determined that a person is moving towards this device. At this time, this device will generate RFID trigger information and semantic segmentation network call information.

[0061] The semantic segmentation network call information is used to call the semantic segmentation network, and this part of the content will be described in detail in step (2).

[0062] The RFID trigger information is transmitted to the RFID identification sub-module to activate the sub-module. Meanwhile, the frontal camera transmits the captured image information of the person to the RFID identification sub-module. The RFID identification sub-module matches the received person image information with the identity image information of each waste disposal person stored in the chip, so as to determine whether the person is a waste disposal person and the identity information of the waste disposal person.

[0063] The above method of obtaining the identity information of waste disposal personnel by triggering the RFID device can reduce the system power consumption. In addition, it should be emphasized that the waste disposal personnel here refer to the staff specifically responsible for waste classification. To obtain the waste classification category and waste picture information of the disposal, a top-down camera is set at the position of the top surface of the device facing the trash can. This top-down camera is an RGB-D camera, which collects pictures of the surface of the waste in the trash can from a top-down perspective. The top-down camera extracts the waste information of the waste disposal person's hand. Specifically, a circle is generated with the key points of the human hand as the center and a radius of 1 / 2 of the diameter of the trash can's disposal opening. The union of the circles is set as the first region of interest. This first region of interest is the picture information of the waste disposed by the waste disposal person. To make the obtained waste pictures clearer, frame-by-frame comparison is performed on the pictures inside the trash can through the depth channel. When there are differences between the pictures inside the trash can in two adjacent frames, it means that this is the time period when the waste disposal person is disposing of waste. The stage before this time period is called the pre-disposal stable stage, and the stage after this time period is called the post-disposal stable stage. In the pre-disposal stable stage and the post-disposal stable stage, it represents the time period when there is no change in the waste inside the trash can. By comparing any set of images in the two stable stages, the picture information of the newly placed waste in the trash can can be obtained. This waste picture information includes RGB waste images and depth waste images.

[0064] In addition, to obtain the waste classification category of this disposal, all the waste in the images obtained in the pre-disposal stable stage is identified, and the waste classification category corresponding to the vast majority of the waste is used as the waste classification category of this disposal. That is to say, under normal circumstances, almost most of the waste can be classified correctly. Therefore, we can judge the waste classification category corresponding to the trash can based on the types of waste in the trash can before this waste disposal. The identification of all the waste in the images obtained in the pre-disposal stable stage can be achieved through a semantic segmentation network. The specific implementation process belongs to the prior art and will not be elaborated here. Of course, as other implementation methods, the trash cans can also be numbered, that is, each trash can corresponds to a set waste classification category. In this way, when it is known which trash can the waste is disposed into, the waste classification category corresponding to this disposal can be directly known.

[0065] (2)Identify the type of garbage based on the garbage image information and determine whether the type of garbage can be identified.

[0066] Among them, according to the semantic segmentation network call information generated by this device in step (1), the pre-trained semantic segmentation network is called, and the garbage image information obtained is identified through the semantic segmentation network. This semantic segmentation network is essentially composed of a neural network, and this neural network has a large number of open-source data sets. Pixel-level classification is performed through the semantic segmentation network. The labels during the training of the semantic segmentation network include various common garbage categories for preliminary identification. Here, various common garbage categories include plastic bottles, waste paper, aluminum cans, etc. The purpose of only including common garbage categories is to avoid excessive annotation costs for semantic segmentation. Since the composition of the semantic segmentation network by the neural network and the training process of the semantic segmentation network are both well-known technologies, they will not be elaborated here.

[0067] If the type information of the garbage can be identified through the semantic segmentation network, then it is possible to directly judge whether the current garbage disposal is correct based on the identified type information of the garbage and the classification category in which the garbage disposer disposes. For example, if the garbage disposed this time is a mineral water bottle and the semantic segmentation network identifies that this garbage is a mineral water bottle, if the garbage bin into which the garbage disposer disposes is a recyclable garbage bin, it means that the current garbage disposal is correct. If the garbage bin into which the garbage disposer disposes is a harmful garbage bin, then the current garbage disposal is incorrect. If the type information of the garbage cannot be identified, the type of the garbage is unknown and further judgment is required.

[0068] (3)If the type of the garbage cannot be identified, then obtain the pre-stored garbage image information, the corresponding garbage classification category and garbage classification confidence of each garbage image information.

[0069] Among them, in the case where the semantic segmentation network cannot identify the type of the garbage, that is, when the training set of the semantic segmentation network is not complete enough, in order to judge the classification situation of the garbage disposer, a database is pre-constructed in this embodiment. This database pre-stores some garbage image information, the corresponding garbage classification category and garbage classification confidence of each garbage image information. When the type of the garbage cannot be identified through step (2), this database will process and store the garbage image information disposed. The pre-stored garbage image information obtained includes RGB garbage images and depth garbage images.

[0070] (4)Match the disposed garbage image information with the pre-stored garbage image information of each item to judge whether the garbage image information of the same category is matched.

[0071] Among them, before matching, it is necessary to obtain the shape features, structural texture features, and grayscale texture features corresponding to the garbage to be put in according to the information of the put garbage pictures. The obtaining steps are as follows:

[0072] Obtain a grayscale image from the RGB garbage image in the information of the put garbage pictures, use the depth garbage image in the information of the put garbage pictures as the guiding image, use the grayscale image as the initial image, and filter the initial image by means of guided filtering to obtain an output image, and the output image is the shape feature; Guided filtering is a prior art, but the method of using a depth image to guide grayscale image filtering can achieve better beneficial effects in the present invention. The reason is that for a depth image, the shape edge gradient of the depth image is relatively obvious, but it contains a lot of texture noise, such as mineral water bottles; for a grayscale image, its shape edge gradient is smaller than the surface color gradient of the obtained image and is easy to be removed during threshold segmentation. The method of using guided filtering can make the shape surface as smooth as possible while retaining the shape edge, which is more convenient for the accuracy of shape feature comparison; Then subtract the output image from the guiding image to obtain the surface structural texture feature information as the structural texture feature, subtract the output image from the initial image, and obtain the surface grayscale texture information through the local binary pattern (LBP algorithm) as the grayscale texture feature.

[0073] Similarly, according to the information of each pre-stored garbage picture, obtain the corresponding shape features, structural texture features, and grayscale texture features. Since the steps of obtaining the three features according to the information of each pre-stored garbage picture are the same as the steps of obtaining the three features according to the put garbage pictures, they will not be elaborated here.

[0074] Before matching, set weights for the shape features, structural texture features, and grayscale texture features , and set their initial weight values to 1 / 3. Rotate the ROI (region of interest) based on the shape feature so that the ROI in one frame of the image rotates to have the maximum similarity with the shape feature map of another frame of the image. The measurement method uses cosine similarity, and then perform weighted summation on the similarity of these three types of features to obtain the final similarity. The formula for this final similarity is:

[0075]

[0076] Since the proportions of the similarities among the three types of features of shape features, structural texture features, and grayscale texture features are different, the final similarity is used to represent the similarity of the three types of features of an object. Among them, is the final similarity, is the shape feature similarity, is the structural feature similarity, is the grayscale feature similarity.

[0077] If the final similarity is greater than or equal to the preset threshold of the garbage type similarity, it is determined that the garbage picture information put in is the same type of garbage picture information as the pre-stored garbage picture information; otherwise, it is determined that the garbage picture information put in is not the same type of garbage picture information as the pre-stored garbage picture information.

[0078] (5) If no same type of garbage picture information can be matched, set an initial classification confidence threshold for the garbage picture information put in, store the classification category of the garbage put in, the garbage picture information, and the corresponding initial classification confidence threshold, and determine that the current garbage disposal is correct.

[0079] That is to say, if no same type of garbage picture information can be matched for the garbage picture information, it means that the garbage put into the trash can is a garbage outside the database, and there is no such garbage picture information in the database. Then, it is necessary to set the initial classification confidence threshold to 0.2, store the classification category of the garbage, the garbage picture information, and the corresponding initial classification confidence threshold of 0.2, and determine that the current garbage disposal is correct.

[0080] (6) If the same type of garbage picture information is matched, determine whether the classification confidence corresponding to the matched same type of garbage picture information is greater than the classification confidence threshold.

[0081] (7) If it is greater than the classification confidence threshold, determine whether the classification category of the garbage put in is the same as the classification category corresponding to the matched same type of garbage picture information.

[0082] The specific situation is as follows:

[0083] If the classification category of the garbage put in is the same as the classification category corresponding to the matched same type of garbage picture information, determine that the current garbage disposal is correct.

[0084] In addition, to improve the accuracy of judging the result of the next garbage disposal, on the basis that the classification category of the garbage put in is the same as the classification category corresponding to the matched same type of garbage picture information, increase the classification confidence corresponding to the matched same type of garbage picture information. The formula is:

[0085]

[0086] Wherein, is the classification pre-confidence before update, is the classification confidence after update, is the memory coefficient, set to 0.95.

[0087] If the classified garbage category put in is different from the classified garbage category corresponding to the garbage picture information of the same category matched, it is determined that the garbage put in this time is incorrect.

[0088] In addition, in order to improve the accuracy of judging the result of the next garbage put in, on the basis that the classified garbage category put in is different from the classified garbage category corresponding to the garbage picture information of the same category matched, the classification confidence of the garbage picture information of the same category matched will be reduced. The formula is:

[0089]

[0090] The classification confidence before update and the classification confidence after update are in a proportional relationship. Among them, is the classification confidence before update, is the classification confidence after update, is the memory coefficient, set to 0.95.

[0091] It should also be noted that if the garbage picture information of the same category is matched, the minimum similarity among the shape feature similarity, the structural texture feature similarity and the gray texture feature similarity is determined, and the weight value corresponding to the minimum similarity is updated to make the updated weight value smaller than the one before update. After the weight value of the feature map with the smallest garbage similarity is updated, since the weight values of the shape feature, the structural texture feature and the gray texture feature are classified as 1 / 3, 1 / 3, 1 / 3 and their sum is 1, when the weight value of the minimum similarity decreases, the other two weight values need to be increased accordingly.

[0092] For example, in the case where the garbage picture information of the same category is matched, if the value of the shape feature weight is the smallest at this time, then the shape feature weight needs to be updated at this time. The corresponding update method is:

[0093]

[0094] Among them, is the value of the shape feature weight before update that needs to be updated, is the value of the shape feature weight after update that needs to be updated, is the value of the weighted sum of the other two similarities.

[0095] After the shape feature weight is updated, the weights corresponding to the structural texture feature and the gray texture feature are increased accordingly. The corresponding formula is:

[0096] ,

[0097] Among them, is the similarity of the structural feature map, is the similarity of the gray feature map, is the value before the update of the weight of the structural feature to be updated, is the value after the update of the weight of the structural feature to be updated, is the value before the update of the weight of the grayscale feature to be updated, is the value after the update of the weight of the grayscale feature to be updated.

[0098] Through the above method of weight update, the weights of each feature map in the similarity calculation process are optimized, and the situation where the feature data of the same type of garbage is inconsistent is adaptively solved. For example, the structural textures of mineral water bottles may not be consistent originally, so the weight of the structural texture similarity will decrease adaptively, improving the accuracy of its judgment.

[0099] (8) In order to better know the accuracy rate of the garbage put by the garbage putter within a certain period of time, feedback is given on the result of the garbage put by the garbage putter. Then, according to the identity information extraction of the garbage putter in step (1), corresponding evaluation indicators are given for the result of the classification work of the garbage putter within a certain period of time. The calculation formula of the evaluation indicator is:

[0100]

[0101] If the total number of correct classifications by the putter is certain, the total number of correct classifications by the putter is proportional to the evaluation indicator of the putter's classification work; if the total number of correct classifications of the garbage by the putter is certain, the total number of correct classifications by the putter is inversely proportional to the evaluation indicator of the putter's classification work; therefore, the ratio function of the total number of correct classifications of the garbage by the putter to the total number of classifications of the garbage by the putter is the evaluation indicator of the putter's classification work. Among them, gives the corresponding evaluation indicator for the classification work of the garbage putter within a certain period of time, is the total number of correct classifications of the garbage by the same putter within a certain period of time, is the total number of classifications of the garbage by the same putter within a certain period of time.

[0102] In addition, the garbage weighing module of this device can also directly obtain the rough data of the garbage weight according to the image. This image is obtained by the front camera of the device in step (1). When the garbage putter obtains multiple consecutive frames of images before putting the garbage and multiple consecutive frames of images after putting the garbage, select the images with the same position of the garbage putter before and after for comparison, locate through the center key point of the foot, and use the key point of the neck as the perpendicular point to make a perpendicular line perpendicular to the line connecting the key point of the neck and the center point of the human body, and the angle between the line connecting the key point of the neck and the key point of the shoulder and the perpendicular line Characterize the change in the load-bearing of a person's arm when picking up garbage. Since the arm strength of different people is inconsistent, this difference is fuzzily classified into four levels, that is, Divide it into four equal percentile intervals, so there are a total of six weight fuzzy levels, which are used for subsequent rough weight data.

[0103] According to the weight fuzzy level of the garbage, it can be combined with the electronic weighing device installed on the bottom table of this device. By obtaining the weight of the garbage in real time through the electronic weighing device, a reference value can be provided for the rough weight information of the garbage, which is convenient for directly obtaining the rough data of the garbage weight through image perception, and is used for roughly estimating the garbage weight of other garbage stations that only have cameras but no this device installed.

[0104] The above-mentioned mobile vehicle-mounted intelligent garbage information management method, through the way of confidence iteration update, determines the classification category corresponding to the failed classification garbage based on data statistics, without the need for manual database or training set update; through the weight update method, optimizes the weights of each feature map in the similarity calculation process, and adaptively solves the situation where the feature data of the same type of garbage is inconsistent. For example, the structural texture of mineral water bottles may not be the same originally, then the weight of the structural texture similarity will adaptively decrease, improving the judgment accuracy.

[0105] System embodiment:

[0106] This embodiment provides a mobile vehicle-mounted intelligent garbage information management system, which includes a processor and a memory. The processor is used to process the instructions stored in the memory to implement the mobile vehicle-mounted intelligent garbage information management method in the above method embodiment. Since the mobile vehicle-mounted intelligent garbage information management method has been introduced in detail in the method embodiment, it will not be elaborated here.

[0107] It should be noted that: the above sequence of the embodiments of the present invention is only for description and does not represent the superiority or inferiority of the embodiments. And the above specific embodiments of this specification have been described. Other embodiments are within the scope of the appended claims. In some cases, the actions or steps recorded in the claims can be executed in a different order from that in the embodiments and still achieve the desired results. Additionally, the processes depicted in the drawings do not necessarily require the specific order or continuous order shown to achieve the desired results. In certain embodiments, multi-tasking and parallel processing are also possible or may be advantageous.

[0108] Each embodiment in this specification is described in a progressive manner. The same or similar parts between each embodiment can be referred to each other, and the key point of each embodiment is to illustrate the differences from other embodiments.

[0109] The above are only the preferred embodiments of the present invention and are not intended to limit the present invention. Any modifications, equivalent substitutions, improvements, etc. made within the spirit and principle of the present invention shall be included within the protection scope of the present invention.

Claims

1. A mobile vehicle-mounted intelligent garbage information management method, characterized in that, It includes the following steps: Identify the identity information of the garbage thrower, and obtain the garbage throwing picture information and the garbage classification category; Identify the garbage type according to the garbage picture information, and determine whether the garbage type can be recognized; If the garbage type cannot be recognized, obtain the pre-stored garbage picture information, the corresponding garbage classification category and the garbage classification confidence level of each garbage picture information; Match the garbage picture information of the throw with the pre-stored garbage picture information of each to determine whether the garbage picture information of the same category is matched; If the garbage picture information of the same category cannot be matched, set an initial garbage classification confidence level setting threshold for the garbage picture information of the throw, store the garbage classification category, the garbage picture information and the corresponding initial garbage classification confidence level setting threshold of the throw, and determine that the current garbage throw is correct; If the garbage picture information of the same category is matched, determine whether the garbage classification confidence level corresponding to the matched garbage picture information of the same category is greater than the garbage classification confidence level setting threshold; If it is greater than the garbage classification confidence level setting threshold, determine whether the garbage classification category of the throw and the garbage classification category corresponding to the matched garbage picture information of the same category are the same; If the garbage classification category of the throw and the garbage classification category corresponding to the matched garbage picture information of the same category are the same, determine that the current garbage throw is correct; Otherwise, determine that the current garbage throw is incorrect; If the garbage classification category of the throw and the garbage classification category corresponding to the matched garbage picture information of the same category are the same, increase the garbage classification confidence level corresponding to the matched garbage picture information of the same category; If the garbage classification category of the throw and the garbage classification category corresponding to the matched garbage picture information of the same category are different, decrease the garbage classification confidence level corresponding to the matched garbage picture information of the same category.

2. The mobile vehicle-mounted intelligent garbage information management method according to claim 1, wherein The formula for increasing the garbage classification confidence level corresponding to the matched garbage picture information of the same category is: The formula for decreasing the garbage classification confidence level corresponding to the matched garbage picture information of the same category is: Among them, is the pre-classification confidence of garbage before update, is the confidence of garbage classification after update, is the memory coefficient, set to 0.

95.

3. The mobile vehicle-mounted intelligent garbage information management method according to claim 1, wherein, The step of matching the garbage picture information of the throw with the pre-stored garbage picture information of each to determine whether the garbage picture information of the same category is matched includes: Process the garbage picture information of the throw to obtain the corresponding shape feature, structure texture feature and gray texture feature; Perform similarity matching of the obtained corresponding shape feature, structure texture feature and gray texture feature with the corresponding shape feature, structure texture feature and gray texture feature of the pre-stored garbage picture information of each respectively to obtain the shape feature similarity, structure texture feature similarity and gray texture feature similarity; Obtain the final garbage type similarity according to the shape feature similarity, structure texture feature similarity and gray texture feature similarity; If the final garbage type similarity is greater than the set threshold of the garbage type similarity, it is determined that the garbage picture information put in is the same type of garbage picture information as the pre-stored garbage picture information; otherwise, it is determined that the garbage picture information put in is not the same type of garbage picture information as the pre-stored garbage picture information.

4. The mobile vehicle-mounted intelligent garbage information management method according to claim 3, wherein, The steps of processing the input garbage picture information to obtain corresponding shape features, structural texture features, and gray texture features include: The garbage picture information includes an RGB garbage image and a depth garbage image. A grayscale image is obtained according to the RGB garbage image. Taking the depth garbage image as the guidance image and the RGB garbage image as the original image, performing guided filtering to obtain an output image, and the output image is the shape feature. Subtracting the output image from the guidance image to obtain the structural texture feature. Subtracting the output image from the original image to obtain the gray texture feature.

5. The mobile vehicle-mounted intelligent garbage information management method according to claim 3 or 4, characterized in that The steps of obtaining the final garbage type similarity according to the shape feature similarity, structural texture feature similarity, and gray texture feature similarity include: Performing weighted summation on the shape feature similarity, structural texture feature similarity, and gray texture feature similarity to obtain the final garbage type similarity, where the weights corresponding to the shape feature similarity, structural texture feature similarity, and gray texture feature similarity are the first weight, the second weight, and the third weight respectively.

6. The mobile vehicle-mounted intelligent garbage information management method according to claim 5, wherein, It also includes: If the final garbage type similarity is greater than the set threshold of the garbage type similarity, determine the minimum similarity among the shape feature similarity, structural texture feature similarity, and gray texture feature similarity, reduce the weight corresponding to the minimum similarity, and correspondingly increase the weights corresponding to the other two similarities.

7. The mobile vehicle-mounted intelligent garbage information management method according to claim 6, characterized in that The formula for reducing the weight corresponding to the minimum similarity is: The formula for correspondingly increasing the weights corresponding to the other two similarities is: Among them, is the weight value corresponding to the minimum similarity, is the weight value obtained after reducing the weight value corresponding to the minimum similarity, is the value obtained by weighted summation of the other two similarities, and are the weight values corresponding to the other two similarities, and are the weight values obtained after increasing the weight values corresponding to the other two similarities, and are the other two similarities.

8. The mobile vehicle-mounted intelligent garbage information management method according to claim 1, wherein It also includes: Identifying the identity information of the person putting in the garbage, and giving corresponding evaluation indicators for the classification work of the person putting in the garbage within a certain period of time. The calculation formula of the evaluation indicator is: Give corresponding evaluation indicators for the classification work of waste disposal personnel within a certain period of time, is the total number of correct garbage classifications by the same disposal personnel within a certain period of time, is the total number of garbage classifications by the same disposal personnel within a certain period of time.

9. A mobile vehicle-mounted intelligent garbage information management system, characterized in that, It includes a processor and a memory. The processor is used to process the instructions stored in the memory to implement the mobile vehicle-mounted intelligent garbage information management method according to any one of claims 1-8.

Images