Kitchen garbage classification method based on image recognition

Abstract

The invention discloses a kitchen garbage classification method based on image recognition, and the method comprises the steps: obtaining a multi-view real-time image in a garbage can through adjusting camera parameters and a moving distance, preprocessing the image, and screening the basic shape and direction of boundary information; identifying a garbage contour based on the boundary information, extracting non-kitchen garbage information, and comparing the non-kitchen garbage information with a preset library to obtain a hue of the non-kitchen garbage information; analyzing color change in the boundary, comparing the color change with a preset hue library, and rendering boundary information to obtain a detailed rendering image; comparing the rendered image at the previous sampling moment, checking the change of the pixel points of the preset hue, adjusting the color according to the missing condition of the pixel points, and checking the identifiability of the rendered area; and checking whether the current rendered image completely covers the rendered image at the previous moment, if no new area exists, re-sampling, and if yes, further processing and marking a newly added non-kitchen garbage rendered area, and recording the characteristics of the newly added non-kitchen garbage rendered area. The garbage classification efficiency and accuracy are improved, manual intervention is reduced, and environment protection and resource recovery are facilitated.

Description

Technical Field

[0001] The present invention relates to the field of garbage classification, and in particular to a kitchen waste classification method based on image recognition. Background Art

[0002] As an effective means of environmental protection, garbage classification can effectively reduce the amount of garbage and reduce the pollution of garbage disposal to the environment. Through classified recycling, recyclable garbage such as plastics and paper can be reused, reducing the amount of landfill and incineration, thereby reducing the pollution of soil, water and atmospheric environment.

[0003] At present, garbage classification faces the challenge of insufficient public awareness, especially the classification of kitchen waste, whose accuracy directly affects resource recycling. Long-term lack of education leads to classification difficulties, and kitchen waste is easily mixed with other garbage, reducing recycling efficiency. Under the requirements of environmental protection policies and green living concepts, it is necessary to strengthen public education, popularize classification knowledge, and improve classification capabilities. At the same time, strengthen supervision to ensure accurate classification and timely correct improper behavior. Through multi-party efforts, promote garbage classification work and achieve garbage reduction, resource utilization and harmlessness. Summary of the invention

[0004] This application provides a kitchen waste classification method based on image recognition to better standardize and manage the delivery of kitchen waste and reduce the difficulty of subsequent classification and processing of kitchen waste.

[0005] This application provides a kitchen waste classification method based on image recognition, including:

[0006] S101, adjusting the parameters and moving distance of the multi-view surveillance camera according to the internal situation of the trash can, obtaining real-time image information collected by the multi-view surveillance camera in the trash can, pre-processing the collected image information, and screening the basic shape and direction of the boundary information in the real-time image information;

[0007] S102, identifying the outline of the garbage based on the basic shape and direction of the boundary information of the image information at each angle, extracting the information of the non-kitchen waste, and comparing it with a preset boundary information library to obtain a preset color tone corresponding to the boundary information;

[0008] S103, analyzing the color change in the boundary information, comparing the color change path with a preset color tone feature library, rendering the boundary information in the real-time image information at the corresponding sampling moment according to the preset color tone and obtaining a detailed shape rendering image;

[0009] S104, compare the current rendering with the previous sampling moment, determine whether the pixel of the preset color tone in the boundary information is missing, if the pixel is not missing, update the color of the rendering at the previous sampling moment as the reference color and then execute S105; if the pixel is missing, update the color of the remaining part of the pixel in the boundary information, check whether there is a rendering area that may be recognized by the machine after the updated rendering, if so, execute S106, if not, return to S101 for the next round of resampling;

[0010] S105, after the rendering image at the current sampling moment covers the rendering image at the previous sampling moment, whether there is an uncovered rendering area, if there is no uncovered area, return to S101 for the next round of resampling; if there is an uncovered area, enter S106;

[0011] S106, marking the rendering area as the non-kitchen waste newly added in this delivery, and recording detailed features such as the preset color tone, shape, color change path, etc. of this delivery.

[0012] Preferably, the extraction method in step S102 comprises:

[0013] Determine whether the boundary information is continuous and uninterrupted. If the boundary information is continuous and uninterrupted, it is determined to be non-kitchen waste.

[0014] Preferably, the S104 includes:

[0015] The reference color is a pre-set color that cannot be recognized by the machine;

[0016] Preferably, the step S104 includes:

[0017] Updates the image range including rendering bounds information as the base color.

[0018] Preferably, before performing S104 judgment, the following steps are further included:

[0019] Determine whether there is color rendered within a preset threshold range of the rendered image boundary information. If so, replace the color of that part with a transparent channel.

[0020] Preferably, the step S101 of screening the basic shape and direction of the boundary information in the real-time image information further includes:

[0021] S201, extracting the boundary information of garbage from the preprocessed image using an edge detection algorithm;

[0022] S202, identifying a basic shape area in the boundary information based on the edge detection result;

[0023] S203, recording the starting point, end point, length, direction and other parameters of the detected straight line and arc;

[0024] S204, analyzing each basic shape and its directional features, recording the extension or arrangement direction of each basic shape area, and comparing it with the data in the boundary information library.

[0025] Preferably, the S203 includes:

[0026] For the detected straight line segment, record its starting point and end point coordinates, calculate the length through the coordinates, and use the slope or direction angle to indicate the direction; for the arc segment, record the center coordinates, radius, starting angle, and ending angle to describe the shape and direction of the arc.

[0027] Preferably, the method further comprises:

[0028] S301, after identifying the basic shape area of ​​non-kitchen waste, further analyzing the continuity and bending characteristics between the shapes;

[0029] S302, evaluating the dynamic change characteristics of the shape by calculating the angle and curvature changes between adjacent shape areas of the recognition information image;

[0030] S303, combining the identified basic shape areas and their continuity and bending characteristics (straight line segments and arc segments) according to their relative positions and connection relationships in the image information to form a complete shape dynamic path;

[0031] S304, extracting path length, number of bends, and average curvature from the shape dynamic path, and using these features for classification and recognition;

[0032] S305, comparing the identified features with the shape dynamic path library to identify the type of garbage.

[0033] Preferably, the S303 further includes:

[0034] Complete shape dynamic path, including recording key points on the path, path length, and angle features of continuous bends.

[0035] Preferably, the method further comprises:

[0036] S401, extracting boundary information within the trash can and color information within the boundary;

[0037] S402, tracking the continuous change of color within the boundary, and recording the starting point, end point, and change type of the color change as a color change path;

[0038] S403, analyzing the characteristics of the change type, change frequency, and change amplitude of the color change path;

[0039] S404, combining the features of the shape dynamic path and the color change path to form a comprehensive feature of the garbage;

[0040] S405, comparing the comprehensive features with the templates in the comprehensive feature library to identify the type of garbage.

[0041] Preferably, the multi-viewing surveillance camera in S101 further includes:

[0042] S501, deploying a multi-angle camera system to capture images of garbage in the garbage bin from different perspectives;

[0043] S502, analyzing the image of each camera to extract garbage boundary information, and generating a cross line at a specific angle based on the garbage boundary information, with the endpoint of the cross line accurately positioned on the garbage boundary;

[0044] S503, fusing the image features on multiple intersection lines to construct a multi-dimensional feature representation of garbage;

[0045] S504, inputting the multi-dimensional features into a deep learning convolutional neural network, comparing them with a preset boundary information feature library, and outputting the classification and recognition results of the garbage according to the comparison results;

[0046] S505, dynamically adjust the control unit to monitor the garbage delivery speed and light changes, and adjust the camera parameters and recognition algorithm parameters in real time according to the monitoring results.

[0047] Preferably, the S502 includes:

[0048] The cross line is a virtual line segment generated based on the boundary information, and is used to represent the shape and characteristics of the garbage; the endpoint is positioned as the intersection of the cross line and the garbage boundary, and is used to accurately describe the outline of the garbage.

[0049] Preferably, the step 501, deploying a multi-angle camera system, further includes:

[0050] S601, mounts the camera on a support structure with spring arms, allowing fine adjustment of the camera in both horizontal and vertical directions;

[0051] S602, when placing garbage, the operator needs to quickly adjust the aiming angle and focal length of the camera according to the location and speed of the garbage placement;

[0052] S603, the modified camera system continues to perform the multi-angle image acquisition task.

[0053] One or more technical solutions provided in this application have at least the following technical effects or advantages:

[0054] 1. By identifying boundary information and dividing basic geometric shapes, the type of garbage can be determined more accurately. Shape recognition simplifies image information, eliminates some interference, and reduces the complexity of processing. Especially kitchen waste, because it is scattered and has no fixed shape, and its boundary information is discontinuous, it contrasts with non-kitchen waste (such as wine bottles, boxes, etc. with clear structures and clear boundaries). Combining the changes in length, angle and curvature of adjacent shape intervals, a complete shape dynamic path can be constructed, retaining the spatial structure and change trajectory, and enhancing the coherence and integrity of information. This method achieves accurate capture and efficient classification of complex garbage forms, significantly improving the accuracy of classification and processing efficiency.

[0055] 2. The combination of tracking continuous color changes and a comprehensive feature library enhances the accuracy of garbage classification. By deeply tracking the continuity of color changes and recording its characteristics in detail, the subtle differences and dynamic evolution of garbage colors are captured, making up for the limitations of relying on boundary information and shape dynamic paths in the recognition process. Furthermore, based on the constructed comprehensive feature library, the system achieves a deep fusion of shape and color change features; this comprehensive feature not only reflects the physical form of the garbage, but also incorporates rich information about its color, making the recognition process more comprehensive and accurate. At the same time, this method maintains reasonable control of the amount of calculation, ensuring efficiency and feasibility in practical applications.

[0056] 3. The deployed high-resolution, night vision, waterproof and dustproof multi-angle camera system ensures the comprehensive and clear capture of garbage image information in the trash can. The introduced cross-line recognition technology accurately depicts the shape of the garbage and locates the boundary. Combined with multi-angle feature fusion, it significantly improves the recognition accuracy of complex-shaped garbage by at least 10%, and builds a multi-dimensional and consistent feature representation. At the same time, the camera is equipped with an adjustable elastic component and spring arm design, which gives it a highly flexible alignment capability to adapt to garbage placement at different heights and angles, reduce monitoring blind spots, and ensure image information quality. BRIEF DESCRIPTION OF THE DRAWINGS

[0057] Figure 1 This is a schematic diagram of a method for classifying kitchen waste based on image recognition according to an embodiment of the present invention;

[0058] Figure 2 A schematic diagram of a process of screening basic shapes and directions of boundary information in real-time image information according to an embodiment of the present invention; DETAILED DESCRIPTION

[0059] To facilitate the understanding of the present invention, the present application will be described more comprehensively below with reference to the relevant drawings; the drawings show preferred embodiments of the present invention, but the present invention can be implemented in many different forms and is not limited to the embodiments described herein; on the contrary, the purpose of providing these embodiments is to enable a more thorough and comprehensive understanding of the disclosed content of the present invention.

[0060] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by technicians in the technical field to which the present invention belongs; the terms used in the specification of the present invention are only for the purpose of describing specific embodiments and are not intended to limit the present invention; the term "and / or" used herein includes any and all combinations of one or more related listed items.

[0061] Embodiment 1: Figure 1 As shown in FIG. 1 , the kitchen waste classification method based on image recognition specifically includes:

[0062] S101, according to the internal situation of the trash can, adjust the parameters and moving distance of the multi-view surveillance camera, obtain real-time image information collected by the multi-view surveillance camera in the trash can, pre-process the collected image information, and screen the basic shape and direction of the boundary information in the real-time image information.

[0063] Among them, the preprocessing of the collected image information is to perform operations such as denoising, contrast enhancement, and grayscale on the collected images to improve the accuracy and efficiency of subsequent image analysis. Reference can be made to relevant existing technologies, and the present invention will not elaborate on this.

[0064] The image information is collected once before and after the input, with the interval time not exceeding 2 seconds.

[0065] Specifically, the multi-view surveillance camera installed inside the trash can is used to capture real-time image information inside the trash can from different angles to ensure monitoring without blind spots. The multi-view surveillance camera collects image information wirelessly and / or wiredly.

[0066] Among them, the multi-view surveillance camera needs to have high resolution and wide-angle field of view to cover most areas inside the trash can.

[0067] In some embodiments, the basic shape and direction of the boundary information in the real-time image information is screened, such as Figure 2 As shown, specifically including:

[0068] S201, extracting garbage boundary information from the preprocessed image using an edge detection algorithm.

[0069] Among them, the edge detection algorithm selects the Canny edge detection algorithm. First, Gaussian filtering is used to smooth the image to reduce noise interference. The standard deviation σ of the Gaussian filter is set to 1.4, and then the gradient amplitude and direction are calculated to highlight the edge features. Subsequently, the edge is extracted by non-maximum suppression and dual threshold detection. The high and low thresholds in the dual threshold detection are set to 100 and 200 (grayscale value range 0-255), respectively. The specific operation can refer to the relevant existing technology, which will not be elaborated in the present invention.

[0070] S202: Based on the edge detection result, identify the basic shape area in the boundary information.

[0071] Among them, in the S202, in the identification of straight line detection, the formula of the Hough transform algorithm is: ρ=xcos(θ)+ysin(θ), the step size of ρ is set to 1 pixel, the step size of θ is set to 1 degree, and the accumulator threshold is set to 100. Only when the value in the accumulator exceeds this threshold, it is determined that a straight line is detected. This threshold is used to filter out noise and unnecessary straight lines to ensure that only obvious straight lines are detected. The points in the image space are converted to the parameter ρ-θ space, and the local maximum value in the accumulator is found to detect the straight line, which can effectively identify obvious straight line boundary information in the image information.

[0072] For example, we obtain a 640x480 pixel image of kitchen waste, which has been edge detected to highlight the straight line boundaries. In the Hough transform algorithm, let ρ range from 0 to the length of the image diagonal (about 800 pixels), and θ range from 0 to π / 2 (symmetry needs to be considered); for each point (x, y) in the edge image, convert it into multiple (ρ, θ) pairs and update the accumulator array; after traversing all points, find the (ρ, θ) pairs whose values ​​in the accumulator exceed the threshold (100); for each detected (ρ, θ) pair, use the formula ρ = xcos(θ) + ysin(θ), and draw a straight line on the result image to represent the straight line boundaries such as the edge of the trash can and the wrinkles of the trash bag, and finally successfully identify and extract the straight line boundary information in the image.

[0073] Specifically, after extracting the straight lines and arcs, they are distinguished according to their geometric features (such as shape, length, direction, etc.) and classified into different basic shape areas. In case of approximate shape contours, the contours can be simplified into simple basic geometric shapes.

[0074] S203, recording the starting point, end point, length, direction and other parameters of the detected straight line and arc.

[0075] Among them, for the detected straight line segment, its starting point and end point coordinates are recorded, the length is calculated by the coordinates, and the direction is represented by the slope or direction angle; for the arc segment, the center coordinates, radius, starting angle and ending angle are recorded to describe the shape and direction of the arc.

[0076] S204, analyzing each basic shape and its directional features, recording the extension or arrangement direction of each basic shape area, and comparing it with the data in the boundary information library.

[0077] Specifically, the directional features of each basic shape are calculated according to the recorded parameters, such as the slope of a straight line, the tangent direction of an arc, etc. The analyzed directional features are compared with the template data in the boundary information library to determine the type of garbage in the current image.

[0078] The boundary information database contains geometric feature templates (such as shape, size, direction, etc.) of various types of non-kitchen waste and corresponding classification information. The template data is obtained through machine learning training from a large amount of non-kitchen waste information collected in the early stage.

[0079] S102, based on the basic shape and direction of the boundary information of the image information at each angle, the outline of the garbage is identified, and the information of non-kitchen waste is extracted, and the information is compared with the preset boundary information library to obtain the preset color tone corresponding to the boundary information.

[0080] Among them, the boundary information extraction method of non-kitchen waste is to determine whether the boundary information is continuous and uninterrupted to reduce misjudgment and improve the accuracy of classification. If the boundary information is continuous and uninterrupted, it is judged as non-kitchen waste;

[0081] Specifically, determine whether the boundary information is continuous and uninterrupted. If the boundary information is continuous and uninterrupted, it is determined to be non-kitchen waste. The Hough transform algorithm is used to detect straight line segments in the image, and the contour tracking algorithm (contour tracking based on Freeman chain code) is combined to detect continuous curve boundaries. For the detected boundaries, check whether they form a closed continuous contour. If the boundary information constitutes a closed, unbroken continuous contour, it is preliminarily determined that the object is not kitchen waste, because kitchen waste often presents scattered and irregular shapes, and it is difficult to form a continuous boundary.

[0082] S103, analyzing the color changes within the boundary information, comparing the color change path with the preset color tone feature library, rendering the boundary information in the real-time image information at the corresponding sampling moment according to the preset color tone and obtaining a detailed shape rendering diagram; the rendering includes rendering the boundary information as the corresponding preset color tone and rendering the boundary information as a transparent channel.

[0083] Among them, the preset hues are basic RGB colors in the chromaticity such as red, blue, gray, black, etc.

[0084] Specifically, the color space conversion (from RGB to HSV) is used to enhance the distinguishability of color features. Then, the K-means clustering algorithm segmentation technique is used to identify the color areas within the boundary information and track the continuous changes of these color areas to form color change paths. These color changes are compared with the preset color tone feature library, for which a library containing a variety of typical color change features of non-kitchen waste is constructed. This feature library is constructed by collecting a large amount of sample data and extracting the color change features (starting color, ending color, color gradient mode, etc.) of each non-kitchen waste at different viewing angles. During the comparison process, the cosine similarity metric is used to measure the degree of match between the color change path in the real-time image and the preset color tone feature library, so as to achieve accurate rendering of the boundary information and generate detailed shape renderings.

[0085] For example, suppose a plastic bottle has been screened out and the color changes inside the bottle are analyzed. The light blue in the upper part gradually transitions to the dark blue in the lower part. There are also some green or transparent areas (possibly labels or bottle caps). The color change path is compared with the data in the preset color tone feature library, and the area inside the boundary information is rendered according to the preset color tone in the feature item (such as the gradient of light blue and dark blue). Finally, a rendering is obtained, in which the outline of the plastic bottle is clearly outlined, and the color changes are accurately restored and displayed. The boundary information outside the rendering becomes a transparent channel, and the image information only shows the identified non-kitchen waste.

[0086] S104, compare the current rendering with the previous sampling moment, determine whether the pixel points of the preset color tone in the boundary information are missing, if there are no missing pixels, update the color of the rendering at the previous sampling moment as the reference color and then execute S105; if there are missing pixels, update the color of the retained part of the pixels in the boundary information, check whether there is a rendering area that may be recognized by the machine after the updated rendering, if so, execute S106, if not, return to S101 for the next round of resampling.

[0087] Before executing this step, it is necessary to determine whether there is color rendered within the preset threshold value (brightness, color intensity, saturation, and the specific threshold setting should be set according to the external environment of the trash can) of the rendering boundary information. If so, replace the color of this part with a transparent channel.

[0088] The reference color is a pre-set color that cannot be recognized by the machine.

[0089] Specifically, the delivery of garbage is a dynamic behavior, and the delivery process may cause the objects to move to a certain extent, so certain noise may be generated after the image coverage is completed. When analyzing new objects in the covered synthetic image, a preset threshold range is added to reduce the detection interference caused by the movement of objects. After the error is corrected through this step, step S104 is entered.

[0090] S105, after the rendering image at the current sampling moment covers the rendering image at the previous sampling moment, whether there is an uncovered rendering area, if there is no uncovered area, return to S101 for the next round of resampling; if there is an uncovered area, enter S106.

[0091] S106, marking the rendering area as the non-kitchen waste newly added in this delivery, and recording and updating detailed features such as preset color tone, shape, color change path, etc. of this delivery.

[0092] It should be noted that when processing real-time image information, the real-time and efficiency of the algorithm need to be optimized according to the specific location and environmental conditions of the garbage to ensure that the task can be completed efficiently within the limited time. When using this method, image processing and machine learning algorithms have certain requirements for computing resources, so it is recommended to use a server or edge computing device equipped with a high-performance GPU for real-time processing. In order to record and analyze huge amounts of image data, sufficient storage space should be ensured, and the stability and security of data transmission should be maintained. In addition, the camera and computing equipment need to be connected to a stable power supply to ensure long-term stable operation. If the image data needs to be transmitted to the cloud for further analysis or storage, a stable network connection is essential.

[0093] The technical solutions in the above embodiments of the present application have at least the following technical effects or advantages:

[0094] 1. By identifying the basic shape of the boundary information and dividing it according to basic geometric shapes, the type and attributes of the garbage can be judged more accurately. Shape is one of the important characteristics of an object. Identifying the basic shape of an object can eliminate interference items that do not match the type of garbage, simplifying complex image information into basic shape information, and reducing the complexity of subsequent processing. By identifying direction information, a more complete feature description of garbage image information can be constructed, thereby improving the accuracy of recognition and the processing speed of image information, and more flexibly responding to various types of garbage. At the same time, more detailed and accurate recognition results can better help the construction of subsequent renderings.

[0095] 2. Compared with other types of garbage, kitchen waste (such as peels, leftovers, residues, etc.) is scattered and has no fixed shape, so there is no relatively fixed and continuous boundary information in the image information; while other types of garbage (such as wine bottles, boxes, bowls, plastic bags, etc.) often have continuous and / or large-scale boundary information due to their own structural design. To achieve fast and accurate judgment of kitchen waste and non-kitchen waste, there is no need to classify and judge the garbage one by one, but only to classify and judge the garbage when it is judged to be non-kitchen waste.

[0096] Embodiment 2: In some embodiments, the basic shape area in the boundary information is identified based on the edge detection result. If the boundary information of the filtered real-time image information is irregular garbage and continuous bends and straight curves, it will inevitably affect the integrity of the boundary information extraction to a certain extent, thereby affecting the identification of the garbage type.

[0097] In some embodiments, in identifying the basic shape and direction of the boundary information, a combined recognition method is introduced to combine the characteristics of the straight lines, arcs and continuous bends of the basic shape area to form a new recognition feature.

[0098] Furthermore, the recognition accuracy can be further improved by identifying and analyzing the dynamic path of the shape of irregular garbage (such as plastic bags), that is, the continuous combination of straight lines and arcs on its boundary and its bending changes.

[0099] In some embodiments, screening the basic shape and direction of the boundary information in the real-time image information further includes the following:

[0100] S301, after identifying the basic shape area of ​​non-kitchen waste, further analyze the continuity and bending characteristics between the shapes.

[0101] Among them, the continuity and bending characteristics include recording the starting point, end point, length, curvature, bending angle, direction, and curvature of each straight line and arc.

[0102] For example, first identify all basic shape areas in the image information that are not food waste (such as plastic bottles). Then, for each shape area, the system analyzes its bending characteristics, including the bending angle (such as the angle between the bottle mouth and the bottle body), direction (such as whether the bottle body is bent upward or downward), and curvature (a quantity that describes the degree of curvature of the curve, which is 0 for a straight line segment and calculated based on the radius and length of the arc for an arc segment).

[0103] S302, evaluating the dynamic change characteristics of the shape by calculating the angle and curvature changes between adjacent shape areas of the recognition image information.

[0104] Among them, the calculation formula of trigonometric function angle is:

[0105]

[0106] Given the lengths of two line segments AB and AC and the angle θ between them (θ is the center angle), the cosine theorem can be used to calculate the angle, where a and b are the lengths of line segments AB and AC respectively, and c is the length of line segment BC.

[0107] Specifically, a geometric calculation method is used to calculate the angle between adjacent shape areas using trigonometric functions (cosine theorem). The curvature change is calculated using a local curvature estimation algorithm, which evaluates the curvature change by calculating the curvature radius of each point on the shape boundary. Specific parameters include a sampling step size of 2 pixels to smooth the curvature estimate.

[0108] S303: Combining the identified basic shape areas and their continuity and bending characteristics (straight line segments and arc segments) according to their relative positions and connection relationships in the image information to form a complete shape dynamic path.

[0109] Specifically, key points on the path, path length, and angle features of continuous bends are recorded.

[0110] S304, extracting the path length, the number of bends, and the average curvature from the shape dynamic path, and using these features for classification and recognition.

[0111] S305, comparing the identified features with the shape dynamic path library to identify the type of garbage.

[0112] The shape dynamic path library contains shape dynamic path templates and classification information of various types of non-kitchen waste. The shape dynamic path library is obtained by machine learning training based on a large number of shape dynamic paths collected in the early stage.

[0113] The technical solutions in the above embodiments of the present application have at least the following technical effects or advantages:

[0114] 1. Combined with the evaluation of the length, angle and curvature changes between adjacent shape areas, a complete shape dynamic path is formed by combining the basic shape areas. This not only retains the spatial structure and change trajectory of the shape, but also enhances the coherence and integrity of the shape information, thus achieving accurate capture and decomposition of complex shape objects.

[0115] 2. The key features extracted from the dynamic path of the shape (path length, number of bends, average curvature) are effectively used in classification and identification. This path combination method can more intuitively understand the overall shape and dynamic change process of the garbage. At the same time, these features can reflect the uniqueness and difference of the garbage shape, providing a reliable basis for classification and identification, and significantly improving the accuracy and efficiency of classification.

[0116] Embodiment 3: In some embodiments, the method for classifying kitchen waste based on image recognition further includes:

[0117] S401, extracting boundary information within the trash can and simultaneously extracting color information within the boundary.

[0118] S402, tracking the continuous change of color within the boundary, and recording the starting point, end point, and change type of the color change as a color change path.

[0119] Among them, the change types include pure color, gradient color, and mutation color, which is convenient for comprehensively capturing the dynamic change characteristics of garbage color.

[0120] S403, analyzing the characteristics of the change type, change frequency, and change amplitude of the color change path.

[0121] S404, combining the features of the shape dynamic path and the color change path to form a comprehensive feature of the garbage.

[0122] S405, comparing the comprehensive features with the templates in the comprehensive feature library to identify the type of garbage.

[0123] Among them, the comprehensive feature library contains the shape dynamic paths (path length, number of bends, average curvature, etc.) and color change paths (type of change, frequency of change, amplitude of change, etc.) of various types of non-kitchen waste and their classification information. The comprehensive feature library is collected from a large number of image data of different types of non-kitchen waste in the early stage, and the shape dynamic path and color change path features of the garbage are extracted using steps S401 to S403 to organize the extracted comprehensive features into templates, each template containing all key parameters of the shape dynamic path and color change path of a specific garbage.

[0124] The technical solutions in the above embodiments of the present application have at least the following technical effects or advantages:

[0125] 1. Based on the continuous change of color, further tracking the continuity of color change and recording the change characteristics in detail can make up for the shortcomings of relying solely on boundary information and shape dynamic paths, especially in maintaining the original amount of calculation, reducing the calculation burden as much as possible to keep it within the acceptable range for practical applications.

[0126] 2. Based on the comprehensive feature library, the features of shape and color changes are combined to significantly reduce the proportion of kitchen waste misidentified as non-kitchen waste, while reducing the misidentification between non-kitchen waste. The recognition accuracy should reach more than 99.5%, especially when dealing with complex garbage, the misidentification rate should be controlled below 0.5%.

[0127] Embodiment 4: In some embodiments, the multi-view surveillance camera of the kitchen waste classification method based on image recognition further includes:

[0128] S501, deploy a multi-angle camera system to capture images of garbage in the garbage bin from different viewing angles.

[0129] Specifically, at least two cameras are deployed in the trash can to capture real-time images inside the trash can from different angles, ensuring that the coverage areas of each camera overlap, so as to facilitate subsequent cross-line generation and feature fusion.

[0130] Among them, the selection of camera models should take into account the characteristics of resolution (more than 5 million pixels), night vision capability (automatically adjusting exposure and white balance), waterproof and dustproof (reaching IP67 or higher waterproof and dustproof standards), the field of view angle should not be less than 120 degrees, and the installation position should ensure that the panoramic view of the garbage can be captured.

[0131] S502, analyzing the image of each camera to extract garbage boundary information, based on the garbage boundary information, generating a cross line of a specific angle, and the end point of the cross line is accurately positioned on the garbage boundary.

[0132] Among them, the cross line is a virtual line segment generated based on the boundary information, which is used to represent the shape and characteristics of the garbage; the endpoint is located as the intersection of the cross line and the garbage boundary, which is used to accurately describe the outline of the garbage.

[0133] S503, fusing the image features on multiple intersection lines to construct a multi-dimensional feature representation of the garbage.

[0134] Specifically, features are extracted from each intersection line, such as length, angle, and curvature, and these features are normalized to eliminate the differences caused by different cameras and viewing angles. A feature fusion algorithm (weighted average) is used to fuse features from different sources into a unified multi-dimensional feature representation. Relevant prior art may be referred to, and the present invention will not elaborate on this.

[0135] S504, input the multi-dimensional features into the deep learning convolutional neural network, compare them with the preset boundary information feature library, and output the classification and recognition results of the garbage based on the comparison results.

[0136] Among them, the deep learning convolutional neural network uses a classification model based on convolutional neural network (CNN) (VGG16, pre-trained on the ImageNet dataset) and takes the extracted multi-dimensional features as input. The specific parameters of the model include the learning rate set to 0.001, the optimizer selected as Adam, and the loss function as the cross entropy loss. Through transfer learning, the pre-trained weights are first loaded, and then fine-tuned on the garbage classification dataset.

[0137] Specifically, the boundary information feature library is pre-trained or updated to ensure that it contains the latest garbage category and feature information. The features to be identified are input into the recognition algorithm for comparison and classification, and the classification and recognition results are output, which may include garbage category, confidence level and other information.

[0138] S505, adjusting the control unit to monitor the garbage delivery speed and light changes, and adjusting the camera parameters and recognition algorithm parameters in real time according to the monitoring results.

[0139] Among them, the camera parameters include exposure time, focal length, white balance, etc.; the algorithm parameters include classification threshold and learning rate.

[0140] Specifically, when the camera parameters detect significant changes in ambient light, the camera exposure time is automatically adjusted according to the light intensity. For example, in a dark environment, the exposure time is increased to capture more light; conversely, in a strong light environment, the exposure time is reduced to prevent overexposure. If the garbage is placed at a fast speed or a long distance, the focal length is appropriately increased to bring the picture closer; conversely, the focal length is reduced to expand the field of view. The automatic white balance algorithm adjusts the image color according to the current lighting conditions to ensure accurate color restoration of the image. In terms of algorithm parameters, the threshold of the classification algorithm is dynamically adjusted according to the feedback results of real-time classification and recognition. For example, if misclassification occurs repeatedly, the classification threshold is appropriately lowered to increase sensitivity; if the classification accuracy rate is stable at a high level, the threshold can be considered to be increased to reduce false alarms. At the same time, during the online learning process of the deep learning model, the learning rate is dynamically adjusted according to the effect of model training and the needs of the current task. The adjustment of the learning rate is intended to balance the convergence speed and final performance of the model.

[0141] It should be noted that the specific value of the adjustment range will be set according to the actual needs of the camera model, recognition algorithm characteristics and application scenarios.

[0142] The technical solutions in the above embodiments of the present application have at least the following technical effects or advantages:

[0143] 1. By deploying a multi-angle camera system, we can ensure that the images of the garbage in the trash can are captured from multiple perspectives, which not only improves the comprehensiveness and accuracy of the images, but also makes the subsequent feature extraction and recognition more reliable. The camera's high resolution, night vision capability, and waterproof and dustproof characteristics further ensure that clear image data can be obtained in various environments.

[0144] 2. The cross lines not only accurately reflect the shape and characteristics of the garbage, but also accurately locate the garbage boundary through their endpoints, providing a basis for subsequent feature fusion. The introduction of multi-angle cross line recognition technology has increased the recognition accuracy of complex-shaped garbage by at least 10%; by extracting, normalizing and fusing the features on multiple cross lines, a multi-dimensional feature representation of the garbage is constructed. This method effectively integrates information from different cameras and perspectives, eliminates differences, improves feature consistency, and provides more comprehensive and accurate feature data for subsequent classification and recognition.

[0145] Embodiment 5: In some embodiments, a multi-angle camera system is deployed, further comprising:

[0146] S601, mounts the camera on a support structure with spring arms, allowing fine adjustment of the camera in both horizontal and vertical directions.

[0147] Specifically, by manually adjusting the length and angle of the spring arm, the camera can be flexibly aimed at different areas inside the trash can, ensuring that the camera can be flexibly rotated within a certain range to adapt to garbage placement needs at different heights and angles.

[0148] S602, when throwing garbage, the operator needs to quickly adjust the aiming angle and focal length of the camera according to the location and speed of the garbage throwing.

[0149] Among them, the camera's field of view must be able to cover the main disposal area in the trash can to reduce blind spots.

[0150] S603, the modified multi-angle camera system continues to perform the multi-angle image information acquisition task.

[0151] The technical solutions in the above embodiments of the present application have at least the following technical effects or advantages:

[0152] 1. By adding adjustable elastic components, the camera can flexibly adapt to garbage placement at different heights and angles, greatly improving the flexibility of camera alignment. This design enables the camera to easily adapt to the internal structure of different garbage cans and the specific needs of garbage placement, accurately adjust the position and angle of the camera, reduce blind spots in monitoring, and help capture clearer and more comprehensive garbage images. At the same time, the adjustment of the camera's support structure is simpler and faster, making it easier for operators to quickly get started and adjust.

[0153] 2. The design of the spring arm allows the camera to rotate flexibly within a certain range. It can cope with garbage placement at different heights and angles without replacing or repositioning the camera. It can adapt to different situations through fine-tuning, reducing the limited field of view or image quality problems caused by fixed installation, thereby reducing the cost and maintenance difficulty of reinstallation or adjustment.

[0154] The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. For those skilled in the art, the present invention may have various modifications and variations. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present invention shall be included in the protection scope of the present invention.

Claims

1. Kitchen waste classification method based on image recognition, It is characterized in that include: S101, adjusting the parameters and moving distance of the multi-view surveillance camera according to the internal situation of the trash can, obtaining real-time image information collected by the multi-view surveillance camera in the trash can, pre-processing the collected image information, and screening the basic shape and direction of the boundary information in the real-time image information; S102, identifying the outline of the garbage based on the basic shape and direction of the boundary information of the image information at each angle, extracting the information of the non-kitchen waste, and comparing it with the preset boundary information library to obtain the preset color tone corresponding to the boundary information; the extraction method is: judging whether the boundary information is continuous and uninterrupted, and if the boundary information is continuous and uninterrupted, judging it as non-kitchen waste; S103, analyzing the color change in the boundary information, comparing the color change path with a preset color tone feature library, rendering the boundary information in the real-time image information at the corresponding sampling moment according to the preset color tone and obtaining a detailed shape rendering image; S104, compare the current rendering with the previous sampling moment, determine whether the pixel points of the preset color tone in the boundary information are missing, if there are no missing pixels, update the color of the rendering at the previous sampling moment to the reference color, and then execute S105; if there are missing pixels, update the color of the remaining part of the pixel points in the boundary information, and check whether there is a rendering area that may be recognized by the machine after the updated rendering. If so, execute S106, if not, return to S101 for the next round of resampling; the updating includes rendering the image range in the boundary information as the reference color; the reference color is a preset color that cannot be recognized by the machine; S105, after the rendering image at the current sampling moment covers the rendering image at the previous sampling moment, whether there is any uncovered rendering area, if there is no uncovered area, return to S101 for the next round of resampling; If there is an uncovered area, proceed to S106; S106, marking the rendering area as the non-kitchen waste newly added in this delivery, and recording detailed features such as the preset color tone, shape, color change path, etc. of this delivery.

2. The kitchen waste classification method based on image recognition as claimed in claim 1, It is characterized in that The step S101 of screening the basic shape and direction of the boundary information in the real-time image information further includes: S201, extracting the boundary information of garbage from the preprocessed image using an edge detection algorithm; S202, based on the edge detection result, identify the basic shape area in the boundary information; wherein, in identifying the straight line detection, the Hough transform algorithm has the formula: ρ = xcos(θ) + ysin(θ), the step size of ρ is set to 1 pixel, the step size of θ is set to 1 degree, and the accumulator threshold is set to 100. Only when the value in the accumulator exceeds this threshold, it is determined that a straight line is detected. This threshold is used to filter out noise and unnecessary straight lines to ensure that only obvious straight lines are detected. The points in the image space are converted to the parameter ρ-θ space, and the local maximum value in the accumulator is found to detect the straight line, which can effectively identify the obvious straight line boundary information in the image information. S203, recording the starting point, end point, length, direction and other parameters of the detected straight line and arc; S204, analyzing each basic shape and its directional features, recording the extension or arrangement direction of each basic shape area, and comparing it with the data in the boundary information library.

3. The kitchen waste classification method based on image recognition as claimed in claim 2, It is characterized in that The method further comprises: S301, after identifying the basic shape area of ​​non-kitchen waste, further analyzing the continuity and bending characteristics between the shapes; S302, evaluating the dynamic change characteristics of the shape by calculating the angle and curvature changes between adjacent shape areas of the recognition information image; S303, combining the identified basic shape areas and their continuity and bending characteristics according to their relative positions and connection relationships in the image information to form a complete shape dynamic path; S304, extracting path length, number of bends, and average curvature from the shape dynamic path, and using these features for classification and recognition; S305, comparing the identified features with the shape dynamic path library to identify the type of garbage.

4. The kitchen waste classification method based on image recognition as claimed in claim 1, It is characterized in that The method further comprises: S401, extracting boundary information within the trash can and color information within the boundary; S402, tracking the continuous change of color within the boundary, and recording the starting point, end point, and change type of the color change as a color change path; S403, analyzing the characteristics of the change type, change frequency, and change amplitude of the color change path; S404, combining the features of the shape dynamic path and the color change path to form a comprehensive feature of the garbage; S405, comparing the comprehensive features with the templates in the comprehensive feature library to identify the type of garbage.

5. The kitchen waste classification method based on image recognition as claimed in claim 1, It is characterized in that The workflow of the multi-view surveillance camera in S101 further includes: S501, deploying a multi-angle camera system to capture images of garbage in the garbage bin from different perspectives; S502, analyzing the image of each camera to extract garbage boundary information, and generating a cross line at a specific angle based on the garbage boundary information, with the endpoint of the cross line accurately positioned on the garbage boundary; S503, fusing the image features on multiple intersection lines to construct a multi-dimensional feature representation of the garbage; S504, inputting the multi-dimensional features into a deep learning convolutional neural network, comparing them with a preset boundary information feature library, and outputting the classification and recognition results of the garbage according to the comparison results; S505, dynamically adjust the control unit to monitor the garbage delivery speed and light changes, and adjust the camera parameters and recognition algorithm parameters in real time according to the monitoring results.

6. The method for classifying kitchen waste based on image recognition as claimed in claim 5, It is characterized in that The S501, deploying a multi-angle camera system, further includes: S601, mounts the camera on a support structure with spring arms, allowing fine adjustment of the camera in both horizontal and vertical directions; S602, when placing garbage, the operator needs to quickly adjust the aiming angle and focal length of the camera according to the location and speed of the garbage placement; S603, the modified camera system continues to perform the multi-angle image acquisition task.

7. The kitchen waste classification method based on image recognition as claimed in claim 1, It is characterized in that The S104, before making the judgment, also includes the step of judging whether there is color rendered within a preset threshold range of the boundary information of the rendering image, and if so, replacing the color of that part with a transparent channel.

8. The kitchen waste classification method based on image recognition as claimed in claim 2, It is characterized in that In S203, for the detected straight line segment, the coordinates of the starting point and the end point are recorded, the length is calculated by the coordinates, and the direction is represented by any one of the slope and the direction angle; for the arc segment, the coordinates of the center point, radius, starting angle and ending angle are recorded to describe the shape and direction of the arc.

9. The method for classifying kitchen waste based on image recognition as claimed in claim 3, It is characterized in that The complete shape dynamic path in S303 includes recording key points on the path, path length, and angle features of continuous bends.

10. The method for classifying kitchen waste based on image recognition as claimed in claim 5, It is characterized in that In S502, the cross line is a virtual line segment generated based on the boundary information, which is used to represent the shape and characteristics of the garbage; the endpoint is located as the intersection of the cross line and the garbage boundary, which is used to accurately describe the outline of the garbage.

Images