Method and device for sorting household solid waste based on machine vision, and storage medium

Abstract

The application discloses a kind of based on machine vision's life solid waste classification processing method, device and storage medium, it is related to machine vision technical field.Based on machine vision's life solid waste classification processing method includes: distinguishing the life solid waste type corresponding to life solid waste in image, and through extracting the HOG feature of life solid waste, the target life solid waste corresponding to the same life solid waste in continuous image is identified, the tracking to life solid is realized, finally, life solid waste is classified and sorted by sorting robot.The based on machine vision's life solid waste classification processing method, device and storage medium disclosed in the application can realize the automatic classification and sorting of life solid waste, improve the efficiency of life solid waste classification processing, reduce labor cost.

Description

TECHNICAL FIELD

[0001] The application belongs to the technical field of machine vision, and particularly relates to a life solid waste classification processing method and device based on machine vision and a storage medium. BACKGROUND

[0002] With the development of economy, the promotion of urbanization and the improvement of living standards, life solid waste garbage has also increased rapidly, thereby increasing the pressure on the urban garbage collection, transportation and processing system, seriously hindering the development of cities and affecting the healthy life of people, and the classification and recycling of life solid waste is of great significance to solve the above problems.

[0003] Life solid waste classification is a necessary step for life solid waste recycling, and at present, life solid waste classification is mostly in the form of manual sorting, which has low sorting efficiency and needs to consume a large amount of manpower, and because of the poor working environment, long-term work in such an environment will affect the health of workers. Therefore, it is necessary to provide an automatic classification scheme for life solid waste. SUMMARY

[0004] The purpose of the application is to provide a life solid waste classification processing method and device based on machine vision and a storage medium to solve the above problems existing in the prior art.

[0005] In order to achieve the above purpose, the application adopts the following technical scheme:

[0006] In a first aspect, the application provides a life solid waste classification processing method based on machine vision, comprising:

[0007] real-time acquisition of an image of an area where a conveying belt for conveying life solid waste is located;

[0008] inputting the current acquired image into a pre-trained life solid waste detection model for life solid waste detection to identify a plurality of life solid wastes in the current acquired image and life solid waste types corresponding to the plurality of life solid wastes;

[0009] extracting HOG features of areas where the plurality of life solid wastes in the current acquired image are located to obtain HOG features corresponding to each life solid waste in the current acquired image;

[0010] based on the HOG features corresponding to each life solid waste in the latest at least two consecutive images, identifying target life solid wastes corresponding to the same life solid waste in the latest at least two consecutive images;

[0011] Based on the coordinates of the target living solid waste corresponding to the same living solid waste in the latest at least two consecutive images in the pixel coordinate system, the trajectory of the target living solid waste is tracked, so that the sorting robot classifies and sorts the target living solid waste based on the living solid waste type corresponding to the target living solid waste.

[0012] Based on the above disclosure, the present application obtains images of the area where the conveying belt for conveying living solid waste is located in real time; inputs the currently obtained image into a pre-trained living solid waste detection model for living solid waste detection, identifies a plurality of living solid wastes in the currently obtained image and the living solid waste types corresponding to the plurality of living solid wastes, so that the living solid waste types corresponding to the living solid wastes in the image can be identified. Then, the HOG features of the areas where the plurality of living solid wastes in the currently obtained image are located are extracted, and the HOG features corresponding to each living solid waste in the currently obtained image are obtained; based on the HOG features corresponding to each living solid waste in the latest at least two consecutive images, the target living solid waste corresponding to the same living solid waste in the latest at least two consecutive images is identified; based on the coordinates of the target living solid waste corresponding to the same living solid waste in the latest at least two consecutive images in the pixel coordinate system, the trajectory of the target living solid waste is tracked, so that the sorting robot classifies and sorts the target living solid waste based on the living solid waste type corresponding to the target living solid waste. In this way, by extracting the HOG features of the living solid waste, the target living solid waste corresponding to the same living solid waste in the consecutive images can be identified, so that the tracking of the living solid waste is realized. Furthermore, under the premise of identifying the living solid waste and the living solid waste type corresponding thereto and realizing the tracking of the living solid waste, the sorting robot can classify and sort the living solid waste, so that the automatic classification and sorting of the living solid waste are realized.

[0013] Through the above design, the present application can identify the living solid waste type corresponding to the living solid waste in the image, and identify the target living solid waste corresponding to the same living solid waste in the consecutive images by extracting the HOG features of the living solid waste, so that the tracking of the living solid waste is realized. Furthermore, under the premise of identifying the living solid waste and the living solid waste type corresponding thereto and realizing the tracking of the living solid waste, the sorting robot can classify and sort the living solid waste, so that the automatic classification and sorting of the living solid waste are realized, the efficiency of the classification and processing of the living solid waste is improved, the labor cost is reduced, and the impact on the health of the staff due to long-term work in a harsh working environment is avoided.

[0014] In one possible design, before inputting the currently obtained image into the pre-trained living solid waste detection model for living solid waste detection, the method further includes:

[0015] Distortion correction is performed on the currently obtained image.

[0016] In a possible design, the tracking of the trajectory of the target domestic solid waste in the world coordinate system comprises:

[0017] The coordinates of the target domestic solid waste corresponding to the same domestic solid waste in the latest at least two frames of images in the pixel coordinate system are converted into world coordinate system coordinates to obtain the coordinates of the target domestic solid waste corresponding to the same domestic solid waste in the latest at least two frames of images in the world coordinate system.

[0018] The trajectory of the target domestic solid waste in the world coordinate system is tracked based on the coordinates of the target domestic solid waste corresponding to the same domestic solid waste in the latest at least two frames of images in the world coordinate system.

[0019] In a possible design, the coordinates of the target domestic solid waste in the pixel coordinate system are converted into world coordinate system coordinates based on the following formula:

[0020]

[0021] wherein, represents an intrinsic matrix of the camera, represents an extrinsic matrix of the camera, Z represents depth information of the camera, u represents the horizontal coordinate of the target domestic solid waste in the pixel coordinate system, v represents the vertical coordinate of the target domestic solid waste in the pixel coordinate system, X W represents the projection coordinate of the target domestic solid waste on the x-axis in the world coordinate system, Y W represents the projection coordinate of the target domestic solid waste on the y-axis in the world coordinate system, Z W represents the projection coordinate of the target domestic solid waste on the z-axis in the world coordinate system.

[0022] In a possible design, the tracking of the trajectory of the target domestic solid waste in the world coordinate system comprises:

[0023] The real-time position of the target domestic solid waste in the world coordinate system is calculated.

[0024] The method further comprises:

[0025] The target domestic solid waste is sorted by a sorting robot based on the real-time position of the target domestic solid waste in the world coordinate system, and the target domestic solid waste is classified and stored based on the type of domestic solid waste corresponding to the target domestic solid waste.

[0026] In a possible design, the target life solid waste is multiple, and the target life solid waste detected first is sorted before the target life solid waste detected later when the target life solid waste is sorted by the sorting robot.

[0027] In a possible design, the life solid waste detection model is a YOLOv7 model.

[0028] In a second aspect, the present application provides a life solid waste classification processing device based on machine vision, which comprises:

[0029] An acquisition unit is configured to acquire images of an area where a conveying belt for conveying life solid waste is located in real time.

[0030] A detection unit is configured to input the currently acquired images into a pre-trained life solid waste detection model to perform life solid waste detection, and identify multiple life solid wastes in the currently acquired images and life solid waste types corresponding to the multiple life solid wastes.

[0031] An extraction unit is configured to extract HOG features of areas where the multiple life solid wastes in the currently acquired images are located, to obtain HOG features corresponding to each life solid waste in the currently acquired images.

[0032] An identification unit is configured to identify target life solid wastes corresponding to the same life solid waste in the latest at least two consecutive images based on the HOG features corresponding to each life solid waste in the latest at least two consecutive images.

[0033] A tracking unit is configured to track a trajectory of the target life solid waste based on coordinates of the target life solid waste corresponding to the same life solid waste in the latest at least two consecutive images in a pixel coordinate system, so that the target life solid waste is classified and sorted by a sorting robot based on the life solid waste type corresponding to the target life solid waste.

[0034] In a third aspect, the present application provides another life solid waste classification processing device based on machine vision, which comprises a memory, a processor and a transceiver connected in sequence and in communication, wherein the memory is configured to store a computer program, the transceiver is configured to receive and send messages, and the processor is configured to read the computer program and execute the life solid waste classification processing method based on machine vision as described in the first aspect or any possible design of the first aspect.

[0035] In a fourth aspect, the present application provides a computer readable storage medium, which stores instructions, and when the instructions are run on a computer, the life solid waste classification processing method based on machine vision as described in the first aspect or any possible design of the first aspect is executed.

[0036] In a fifth aspect, the present application provides a computer program product comprising instructions which, when executed on a computer, cause the computer to perform the machine vision-based household solid waste classification processing method according to the first aspect or any possible design of the first aspect.

[0037] Advantages:

[0038] The machine vision-based household solid waste classification processing method, device and storage medium provided by the present application can identify the household solid waste type corresponding to the household solid waste in the image, identify the target household solid waste corresponding to the same household solid waste in the continuous image by extracting the HOG feature of the household solid waste, realize tracking of the household solid waste, and then under the premise of identifying the household solid waste and the household solid waste type corresponding thereto and realizing tracking of the household solid waste, the household solid waste can be classified and sorted by the sorting robot, automatic classification and sorting of the household solid waste is realized, the efficiency of household solid waste classification processing is improved, the labor cost is reduced, the impact on the health of the staff due to long-term work in a harsh working environment is avoided, and the present application is convenient for practical application and promotion. BRIEF DESCRIPTION OF DRAWINGS

[0039] Figure 1 The flowchart of the machine vision-based household solid waste classification processing method provided by the present application is shown in the figure;

[0040] Figure 2 The block diagram of the machine vision-based household solid waste classification processing device provided by the present application is shown in the figure;

[0041] Figure 3 The block diagram of another machine vision-based household solid waste classification processing device provided by the present application is shown in the figure. DETAILED DESCRIPTION

[0042] In order to more clearly illustrate the technical solutions of the embodiments of the present application or the prior art, the present application will be briefly introduced below in combination with the drawings and the description of the embodiments or the prior art. Obviously, the following description of the drawings structure is only some embodiments of the present application, and for those skilled in the art, other drawings can be obtained without creative labor. It should be noted that the description of these embodiments is used to help understand the present application, but does not constitute a limitation on the present application.

[0043] It should be understood that although the terms first, second, etc. can be used herein to describe various elements, these elements should not be limited by these terms. These terms are only used to distinguish one element from another. For example, a first element can be called a second element, and similarly a second element can be called a first element, without departing from the scope of the example embodiments of the present application.

[0044] It should be understood that, for the term "and / or" that can appear in the present article, it is only a description of the association relationship of the associated objects, which means that there can be three relationships, for example, A and / or B, which can represent three cases of A alone, B alone, and A and B together; for the term " / and" that can appear in the present article, it is another description of the association relationship of another associated object, which means that there can be two relationships, for example, A / and B, which can represent two cases of A alone and A and B together; in addition, for the character " / " that can appear in the present article, it generally represents an "or" relationship between the associated objects before and after it.

[0045] In order to realize the automatic classification of household solid waste, the embodiment of the present application provides a household solid waste classification processing method, device and storage medium based on machine vision, which can realize the automatic classification and sorting of household solid waste and improve the efficiency of household solid waste classification processing.

[0046] The household solid waste classification processing method based on machine vision provided by the embodiment of the present application can be applied to a user terminal device in communication connection with a camera. The user terminal device can be, but is not limited to, an industrial computer, a personal computer, a smart phone, a tablet computer, a laptop computer, a personal digital assistant, etc. It can be understood that the execution subject does not constitute a limitation on the embodiment of the present application.

[0047] The household solid waste classification processing method based on machine vision provided by the embodiment of the present application will be described in detail below.

[0048] As shown in Figure 1 The flow chart of the household solid waste classification processing method based on machine vision provided by the first aspect of the present application can include, but is not limited to, the following steps S101-S105.

[0049] Step S101. Real-time acquisition of images of the area where the conveying belt for conveying household solid waste is located.

[0050] In the embodiment of the present application, a conveying belt for conveying household solid waste can be provided, and the household solid waste is conveyed through the conveying belt. The camera can be a depth camera, and the camera can be arranged above the conveying belt with the field of view area facing the area where the conveying belt is located.

[0051] When it is necessary to classify and process the household solid waste, the household solid waste can be continuously conveyed through the conveying belt, the camera can acquire images of the area where the conveying belt for conveying household solid waste is located in real time, and the acquired images can be uploaded to the user terminal device for analysis by the user terminal device.

[0052] Step S102. The current acquired image is input into the pre-trained household solid waste detection model for household solid waste detection, and a plurality of household solid wastes in the current acquired image and household solid waste types corresponding to the plurality of household solid wastes are identified.

[0053] In the embodiments of the present application, the household solid waste detection model for household solid waste detection is pre-trained, which can be, but is not limited to, a Faster R-CNN model, a YOLO (You Only Look Once) model, etc. In the embodiments of the present application, the household solid waste detection model adopts a YOLOv7 model.

[0054] When training the household solid waste detection model, sample pictures containing various types of household solid wastes can be collected, and the household solid wastes in the sample pictures are labeled, including the types and position information of the household solid wastes. The household solid wastes in the sample pictures can cover various types of household solid wastes in daily life, such as various colors of PVC, metal recyclables, glass, paper, etc.

[0055] In one or more embodiments, in order to improve the accuracy of the model, the sample pictures can also be expanded to increase the training samples during the training of the household solid waste detection model. When expanding the sample pictures, operations such as rotating, adjusting brightness, adding noise, etc. can be performed on the sample pictures.

[0056] After acquiring the image of the area where the conveying belt is located each time, the current acquired image can be input into the pre-trained household solid waste detection model for household solid waste detection, and a plurality of household solid wastes (including position information of the plurality of household solid wastes) in the current acquired image and household solid waste types corresponding to the plurality of household solid wastes are identified.

[0057] In one or more embodiments, considering that the image captured by the camera can be distorted, the current acquired image can also be corrected for distortion before being input into the pre-trained household solid waste detection model for household solid waste detection. The distortion parameters in the distortion correction process can be determined through multiple experiments, and will not be described in detail.

[0058] Step S103. HOG features of the areas where the plurality of household solid wastes in the current acquired image are located are extracted, and HOG features corresponding to each household solid waste in the current acquired image are obtained.

[0059] HOG (Histogram of Oriented Gradient, Histogram of Oriented Gradient) feature is a feature descriptor used for object detection in computer vision and image processing, which will not be described in detail.

[0060] Step S104. Based on the HOG features corresponding to each household solid waste in the latest at least two consecutive images, the target household solid waste corresponding to the same household solid waste in the latest at least two consecutive images is identified.

[0061] Specifically, the HOG features corresponding to each household solid waste in adjacent two images in the at least two consecutive images can be calculated for similarity, and two household solid wastes respectively located in adjacent two images and having a similarity of the corresponding HOG features exceeding a preset threshold are judged as the target household solid waste corresponding to the same household solid waste. Based on this, the target household solid waste corresponding to the same household solid waste in the latest at least two consecutive images can be identified.

[0062] It can be understood that if the HOG feature corresponding to a certain household solid waste in an image is similar to the HOG features of at least two household solid wastes in an adjacent image, at this time, the similarity of other parameters such as the size and the orientation of the edge of the household solid waste can also be considered to judge whether the two household solid wastes respectively located in adjacent two images are the target household solid waste corresponding to the same household solid waste.

[0063] Step S105. Based on the coordinates of the target household solid waste corresponding to the same household solid waste in the latest at least two consecutive images in the pixel coordinate system, the trajectory of the target household solid waste is tracked, so that the target household solid waste is classified and sorted by the sorting robot based on the type of the household solid waste corresponding to the target household solid waste.

[0064] Specifically, when tracking the trajectory of the target household solid waste, the coordinates of the target household solid waste corresponding to the same household solid waste in the latest at least two consecutive images in the pixel coordinate system can be first converted into world coordinate system coordinates to obtain the coordinates of the target household solid waste corresponding to the same household solid waste in the latest at least two consecutive images in the world coordinate system. Then, based on the coordinates of the target household solid waste corresponding to the same household solid waste in the latest at least two consecutive images in the world coordinate system, the trajectory of the target household solid waste in the world coordinate system is tracked. Tracking the trajectory of the target household solid waste in the world coordinate system can mean calculating the real-time position of the target household solid waste in the world coordinate system.

[0065] In the embodiments of the present application, the coordinates of the target household solid waste in the pixel coordinate system can be converted into world coordinate system coordinates based on the following formula:

[0066]

[0067] wherein, represents the intrinsic matrix of the camera, represents the extrinsic matrix of the camera, Z represents the depth information of the camera, urepresents the horizontal coordinate of the target domestic solid waste in the pixel coordinate system, v represents the vertical coordinate of the target domestic solid waste in the pixel coordinate system, X W represents the projection coordinate of the target domestic solid waste on the x-axis in the world coordinate system, Y W represents the projection coordinate of the target domestic solid waste on the y-axis in the world coordinate system, Z W represents the projection coordinate of the target domestic solid waste on the z-axis in the world coordinate system.

[0068] In one or more embodiments, after tracking the trajectory of the target domestic solid waste, the target domestic solid waste can be sorted by the sorting robot based on the real-time position of the target domestic solid waste in the world coordinate system, and classified and stored based on the type of domestic solid waste corresponding to the target domestic solid waste.

[0069] When sorting the target domestic solid waste, the target domestic solid waste detected first is sorted before the target domestic solid waste detected later. For example, during the classification process of the domestic solid waste, if the target domestic solid waste A is detected based on the first and second images, and the target domestic solid waste B is detected based on the second and third images, it can be determined that the target domestic solid waste A is detected before the target domestic solid waste B, and when sorting the target domestic solid waste, the target domestic solid waste A can be sorted by the sorting robot first, and then the target domestic solid waste B can be sorted by the sorting robot. In this way, the target domestic solid waste entering the conveyor belt first can be sorted first, avoiding the situation of missing sorting due to failure to sort in time.

[0070] In addition, considering that a single sorting robot may not be able to sort the domestic solid waste on the conveyor belt in time, in one or more embodiments, multiple sorting robots can also be provided to sort the domestic solid waste on the conveyor belt.

[0071] In summary, the method for classifying and processing domestic solid waste based on machine vision provided by the present application can identify the type of domestic solid waste corresponding to the domestic solid waste in the image, and identify the target domestic solid waste corresponding to the same domestic solid waste in the continuous image by extracting the HOG feature of the domestic solid waste, thereby realizing tracking of the domestic solid waste. Further, under the premise of identifying the domestic solid waste and the type of domestic solid waste corresponding thereto and realizing tracking of the domestic solid waste, the domestic solid waste can be classified and sorted by the sorting robot, realizing automatic classification and sorting of the domestic solid waste, improving the efficiency of classification and processing of domestic solid waste, reducing labor costs, avoiding the impact on the health of workers due to long-term work in harsh working environment, facilitating practical application and promotion.

[0072] Please refer to Figure 2The second aspect of the embodiment of the present application provides a household solid waste classification processing device based on machine vision, which comprises

[0073] An acquisition unit is configured to acquire images of an area where a conveying belt for conveying household solid waste is located in real time.

[0074] A detection unit is configured to input the currently acquired images into a pre-trained household solid waste detection model to perform household solid waste detection, and identify a plurality of household solid wastes in the currently acquired images and household solid waste types corresponding to the plurality of household solid wastes.

[0075] An extraction unit is configured to extract HOG features of areas where the plurality of household solid wastes in the currently acquired images are located, to obtain HOG features corresponding to each household solid waste in the currently acquired images.

[0076] An identification unit is configured to identify target household solid wastes corresponding to the same household solid waste in the latest at least two consecutive images based on the HOG features corresponding to each household solid waste in the latest at least two consecutive images.

[0077] A tracking unit is configured to track a trajectory of the target household solid waste based on coordinates of the target household solid waste corresponding to the same household solid waste in the latest at least two consecutive images in a pixel coordinate system, so that the target household solid waste is classified and sorted by a sorting robot based on the household solid waste type corresponding to the target household solid waste.

[0078] The working process, working details and technical effects of the household solid waste classification processing device based on machine vision provided by the second aspect of the embodiment can be referred to the first aspect of the embodiment, and will not be described here.

[0079] As shown in Figure 3 The third aspect of the embodiment of the present application provides another household solid waste classification processing device based on machine vision, which comprises a memory, a processor and a transceiver connected in sequence and in communication, wherein the memory is configured to store a computer program, the transceiver is configured to receive and send messages, and the processor is configured to read the computer program and execute the household solid waste classification processing method based on machine vision as described in the first aspect of the embodiment.

[0080] For example, the memory can include, but is not limited to, random access memory (RAM), read-only memory (ROM), flash memory, first-in-first-out memory (FIFO), first-in-last-out memory (FILO), and the like; the processor can be, but is not limited to, a microprocessor of STM32F105 series, an ARM (Advanced RISC Machine) processor, an X86 architecture processor, or a processor integrated with a neural-network processing unit (NPU); and the transceiver can be, but is not limited to, a WiFi (Wireless Fidelity) transceiver, a Bluetooth transceiver, a General Packet Radio Service (GPRS) transceiver, a ZigBee transceiver, a 3G transceiver, a 4G transceiver, a 5G transceiver, and the like.

[0081] The fourth aspect of the embodiment provides a computer-readable storage medium storing instructions of the machine vision-based household solid waste classification and processing method according to the first aspect of the embodiment, i.e., the computer-readable storage medium stores instructions, and when the instructions are executed on a computer, the machine vision-based household solid waste classification and processing method according to the first aspect is executed. The computer-readable storage medium is a carrier storing data, and can include, but is not limited to, a floppy disk, an optical disk, a hard disk, a flash memory, a USB flash disk, a Memory Stick, and the like. The computer can be a general-purpose computer, a special-purpose computer, a computer network, or another programmable device.

[0082] The fifth aspect of the embodiment provides a computer program product containing instructions, which, when executed on a computer, cause the computer to execute the machine vision-based household solid waste classification and processing method according to the first aspect of the embodiment. The computer can be a general-purpose computer, a special-purpose computer, a computer network, or another programmable device.

[0083] It should be understood that, in the following description, specific details are provided to facilitate a full understanding of the example embodiments. However, a person of ordinary skill in the art should understand that the example embodiments can be implemented without these specific details. For example, systems can be shown in block diagrams to avoid obscuring the examples with unnecessary details. In other examples, well-known processes, structures, and techniques can not be shown in unnecessary detail to avoid obscuring the example embodiments.

[0084] Finally, it should be noted that the above description is only the preferred embodiment of the present application, and is not intended to limit the protection scope of the present application. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present application shall be included in the protection scope of the present application.

Claims

1. A method for sorting and processing solid waste based on machine vision, characterized in that, The method comprises: real-time acquisition of an image of an area where a conveying belt for conveying household solid waste is located; inputting the currently acquired image into a pre-trained household solid waste detection model for household solid waste detection, to identify a plurality of household solid wastes in the currently acquired image and household solid waste types corresponding to the plurality of household solid wastes; extracting HOG features of areas where the plurality of household solid wastes in the currently acquired image are located, to obtain HOG features corresponding to each household solid waste in the currently acquired image; based on the HOG features and the orientations of the edges corresponding to each household solid waste in the latest at least two consecutive images, identifying target household solid wastes corresponding to the same household solid waste in the latest at least two consecutive images; based on the coordinates of the target household solid wastes corresponding to the same household solid waste in the latest at least two consecutive images in a pixel coordinate system, tracking the trajectories of the target household solid wastes, so that the target household solid wastes are classified and sorted by the sorting robot based on the household solid waste types corresponding to the target household solid wastes. 2.The machine vision-based solid waste classification processing method according to claim 1, wherein, Before the currently acquired image is inputted into the pre-trained household solid waste detection model for household solid waste detection, the method further comprises: performing distortion correction on the currently acquired image. 3.The machine vision-based solid waste classification processing method according to claim 1, wherein, The tracking of the trajectories of the target household solid wastes in the world coordinate system comprises: converting the coordinates of the target household solid wastes corresponding to the same household solid waste in the latest at least two consecutive images in the pixel coordinate system into world coordinate system coordinates, to obtain the coordinates of the target household solid wastes corresponding to the same household solid waste in the latest at least two consecutive images in the world coordinate system; tracking the trajectories of the target household solid wastes in the world coordinate system based on the coordinates of the target household solid wastes corresponding to the same household solid waste in the latest at least two consecutive images in the world coordinate system. 4.The machine vision-based solid waste classification processing method according to claim 3, wherein, The conversion of the coordinates of the target household solid wastes in the pixel coordinate system into the world coordinate system coordinates is based on the following formula: wherein, represents an intrinsic matrix of the camera, represents an extrinsic matrix of the camera, Z represents depth information of the camera, u represents an x-coordinate of the target municipal solid waste in the pixel coordinate system, v represents a y-coordinate of the target municipal solid waste in the pixel coordinate system, X W represents a projection coordinate of the target municipal solid waste on the x-axis in the world coordinate system, Y W represents a projection coordinate of the target municipal solid waste on the y-axis in the world coordinate system, Z W represents a projection coordinate of the target municipal solid waste on the z-axis in the world coordinate system. 5.The machine vision-based solid waste classification processing method according to claim 3, wherein, The tracking of the trajectories of the target household solid wastes in the world coordinate system comprises: calculating the real-time position of the target household solid waste in the world coordinate system; The method further comprises: based on the real-time position of the target household solid waste in the world coordinate system, sorting the target household solid waste by the sorting robot, and classifying and storing the target household solid waste based on the household solid waste type corresponding to the target household solid waste. 6.The machine vision-based solid waste classification processing method according to claim 5, wherein, The target household solid waste is a plurality of target household solid wastes, and a target household solid waste detected earlier is sorted before a target household solid waste detected later. 7.The machine vision-based household solid waste sorting method according to claim 1, wherein, The household solid waste detection model is a YOLOv7 model. 8.A machine vision-based solid waste classification processing device, characterized by The method comprises: an acquisition unit configured to acquire an image of an area where a conveying belt for conveying household solid waste is located in real time; a detection unit configured to input the currently acquired image into a pre-trained household solid waste detection model for household solid waste detection, to identify a plurality of household solid wastes in the currently acquired image and household solid waste types corresponding to the plurality of household solid wastes; The extraction unit is configured to extract HOG features of regions where the living solid wastes are located in the current acquired image, to obtain HOG features corresponding to each living solid waste in the current acquired image. The recognition unit is configured to recognize target living solid wastes corresponding to the same living solid waste in the latest at least two continuous images based on the HOG features and the edge orientations of each living solid waste in the latest at least two continuous images. The tracking unit is configured to track a trajectory of the target living solid waste based on coordinates of the target living solid waste corresponding to the same living solid waste in the latest at least two continuous images in a pixel coordinate system, so that the target living solid waste is classified and sorted by the sorting robot based on a living solid waste type corresponding to the target living solid waste. 9.A machine vision-based solid waste classification processing device, characterized by The computer readable storage medium has instructions stored thereon, and when the instructions are executed on the computer, the method for classifying and processing living solid wastes based on machine vision is executed.

10. A computer-readable storage medium, characterized in that, The computer readable storage medium has instructions stored thereon, and when the instructions are executed on the computer, the method for classifying and processing living solid wastes based on machine vision is executed.

Images