Intelligent solid waste sorting system based on image recognition

Abstract

The invention relates to the technical field of image processing, in particular to an intelligent solid waste sorting system based on image recognition. According to the method, the edge gradient intensity and the gray-level co-occurrence contrast in the image are uniformly mapped to the same scale space, and the confidence scoring path is constructed in combination with the image frame sequence, so that the stability and the distinction degree of label scoring in solid waste image recognition can be improved; main trend labels in a scoring path are mined through a scoring direction and a fluctuation analysis mechanism, the main label judgment accuracy of image classification can be enhanced, fine-grained recognition is carried out on junk image structure abnormity through grid division and tension index extraction, abnormal structures such as deformation and fracture in the image are marked and positioned, and the accuracy of classification of the junk image is improved. On this basis, the boundary structure of the image is updated through connection reconstruction and triangulation, effective restoration of an abnormal region is realized, consistency evaluation is performed in combination with a main label path and a topological structure, and the robustness of identification output and the structural integrity of a classification result are improved.

Description

Technical Field

[0001] The present invention relates to the technical field of image processing, and in particular to an intelligent solid waste sorting system based on image recognition. Background Art

[0002] The field of image processing technology mainly involves core issues such as image acquisition, analysis, recognition and processing, covering multiple technical directions such as image acquisition, preprocessing, feature extraction, image recognition and understanding, image enhancement, image segmentation, image compression and reconstruction. Its goal is to achieve intelligent analysis and effective utilization of image content.

[0003] Among them, the traditional solid waste intelligent sorting system based on image recognition refers to a system that determines the type of garbage by collecting and identifying solid waste images, and guides the sorting equipment to perform corresponding separation operations accordingly. It mainly focuses on how to accurately and automatically classify solid waste under conditions of complex composition and changeable appearance.

[0004] When judging solid waste images in complex environments, existing technologies rely on image acquisition and preliminary recognition to directly classify types. They lack the ability to detect and process image structural anomalies, resulting in label misjudgment in situations such as blurred image boundaries, distorted configurations, and incomplete targets. When the scores of different labels in consecutive image frames are unstable, it is difficult to establish a label evolution trend path, resulting in the recognition logic being unable to continuously track effective label changes. For example, when there are obvious cracks or occlusions in the image, the system cannot identify and repair the missing structure, often resulting in classification interruptions or misleading sorting equipment to perform erroneous actions. In addition, under the interference of label signal noise, existing systems find it difficult to confirm a stable label path, thereby affecting the final classification reliability and system response efficiency. Summary of the Invention

[0005] The purpose of the present invention is to solve the shortcomings of the prior art and propose an intelligent solid waste sorting system based on image recognition.

[0006] In order to achieve the above objectives, the present invention adopts the following technical solutions: The solid waste intelligent sorting system based on image recognition includes: The confidence score generation module obtains the edge intensity gradient features and grayscale co-occurrence contrast features in the solid garbage image frame sequence and performs a uniform amplitude adjustment. It then compares them with similar features counted in the image samples. Based on the comparison results, it generates a confidence value score for the corresponding candidate label for the current image frame and constructs a scoring path sequence. The main label identification module analyzes the scoring direction of each candidate label in the scoring path sequence and identifies the main label path, and outputs a main label mapping table; The structural anomaly detection module divides the solid waste image into multiple grid structures, extracts the tension index of each grid structure, and marks the areas with abnormal tension index fluctuations as potential structural anomaly areas; The topology repair module performs local repair and reconstruction on the intersection of the grid boundaries in the potential structural abnormality area to obtain a repaired topology image; The sorting and classification module combines the repaired topological image with the main label mapping table to classify and identify the solid waste image to obtain a solid waste sorting result.

[0007] As a further solution of the present invention, the scoring path sequence includes a label scoring value sequence, a scoring direction change sequence, and a label scoring stability index; the main label mapping table specifically includes the main label identification result, activation threshold parameter, and label priority order; the potential structural abnormality area includes the boundary continuity mutation area, the tension abnormality concentration area, and the grid deformation expansion area; the repaired topological image specifically refers to the closed boundary path, the connection completion area, and the structural reconstruction fragment; the solid waste sorting result includes the main label classification identification, the image structure status, and the sorting control instruction.

[0008] As a further solution of the present invention, the confidence score generation module includes: The image feature normalization submodule obtains solid waste images and performs frame operations, collecting edge intensity gradient features and grayscale co-occurrence contrast features of each frame in the image frame sequence, and maps the two types of features to the unit interval numerical space through the normalization method to generate unified dimension features; Feature label mapping submodule: Project the unified dimension feature as a two-dimensional coordinate point into the feature similarity space, calculate the Euclidean distance with the candidate label feature coordinates counted in the image sample, and determine the corresponding candidate label confidence score according to the inverse of the distance value; The scoring path construction submodule calls the candidate tag confidence score, arranges the scoring results of the candidate tags in each frame in sequence according to the image frame order, and connects the scoring points of the same candidate tag in each frame to obtain a scoring path sequence.

[0009] As a further solution of the present invention, the primary tag identification module includes: The score change extraction submodule calls the score path sequence, calculates the score direction difference of each candidate tag in consecutive image frames, extracts the score jump frequency, score amplitude and direction continuity parameters corresponding to the tag path, and generates the candidate tag score change feature; The candidate tag screening submodule determines the consistency and fluctuation range of each candidate tag path in the scoring direction based on the candidate tag score change characteristics, screens the path with the most stable scoring direction and the smallest amplitude fluctuation range, and generates a main tag candidate path identifier; The main label output submodule sets the confidence activation threshold according to the variance of the main label candidate path identification score path, determines the main label path under the current image frame sequence, and establishes a mapping relationship between the label path and the confidence activation threshold to generate a main label mapping table.

[0010] As a further solution of the present invention, the structural anomaly detection module includes: The grid division submodule obtains the solid waste image area, divides each area into an equilateral triangle grid structure, sequentially numbers the grids to locate the intersection position relationship, and generates an image grid structure coordinate set; The tension extraction submodule calculates three types of indicators, namely, the side length change rate, the angle offset, and the diagonal length difference corresponding to the intersection position of each grid in the image grid structure coordinate set, summarizes them into unit tension description parameters, and generates grid tension index data; The abnormal area identification submodule judges the fluctuation direction and amplitude change of each tension index in the continuous grid structure based on the grid tension index data, selects areas with continuous index deviation and boundary mutation rate greater than the judgment standard, and marks them as potential structural abnormal areas.

[0011] As a further solution of the present invention, the topology repair module includes: The boundary extraction submodule extracts the coordinate sequence of the grid boundary intersection position in the potential structural anomaly area, identifies the connection loss and angle misalignment information between each intersection position, and generates a fracture boundary intersection coordinate set; The connection generation submodule sets interpolation rules based on the fracture boundary intersection coordinate set according to the connection angle deviation and the side length similarity, adds auxiliary intersection points at the missing connection positions, establishes auxiliary connection chains according to the shortest connection path, and generates a boundary reconstruction connection path; The structure output submodule combines and updates the boundary reconstruction connection path with the original grid structure of the image through a triangulation algorithm, performs closed repair processing on the boundary of the abnormal area, and obtains a repaired topological image.

[0012] As a further solution of the present invention, the sorting and classification module includes: The path matching submodule calls the repaired topology image and the main label mapping table, extracts the confidence scores corresponding to the structural connection paths and the main label paths, performs path structure consistency and score threshold judgment, and generates a main label matching judgment result; The label confirmation submodule selects the main labels with scores higher than the activation threshold based on the main label matching judgment result, confirms the garbage category corresponding to the target main label as the recognition output of the current image, and generates solid garbage identification label data; The result generation submodule calls the solid waste identification label data, uses the identification label as the classification basis to generate matching sorting instructions, performs image-level classification judgment and action binding through conditional random fields, and obtains the solid waste sorting results.

[0013] Compared with the prior art, the advantages and positive effects of the present invention are: In the present invention, by uniformly mapping the edge gradient intensity and grayscale co-occurrence contrast in the image to the same scale space and combining the image frame sequence to construct a confidence scoring path, the stability and discrimination of label scoring in solid waste image recognition can be improved. The main trend labels in the scoring path are mined through the scoring direction and fluctuation analysis mechanism, which can enhance the accuracy of main label judgment of image classification. The structural anomalies of garbage images are fine-grainedly identified through grid division and tension index extraction, so that abnormal structures such as deformation and fracture in the image can be marked and located. On this basis, the boundary structure of the image is updated through connection reconstruction and triangulation, and the effective repair of the abnormal area is achieved. The main label path and the topological structure are combined for consistency evaluation, which helps to improve the robustness of the recognition output and the structural integrity of the classification results. Finally, high-matching sorting instructions are generated through image-level recognition results to enhance the accuracy and response efficiency of the sorting action. BRIEF DESCRIPTION OF THE DRAWINGS

[0014] Figure 1 is a system flow chart of the present invention; Figure 2 A flowchart of a confidence score generation module of the present invention; Figure 3 This is a flow chart of the main tag identification module of the present invention; Figure 4 This is a flow chart of the structural anomaly detection module of the present invention; Figure 5 This is a flowchart of the topology repair module of the present invention; Figure 6 This is a flow chart of the sorting and classification module of the present invention. DETAILED DESCRIPTION

[0015] In order to make the purpose, technical solutions and advantages of the present invention more clearly understood, the present invention will be further described in detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present invention and are not intended to limit the present invention.

[0016] In the description of the present invention, it should be understood that the terms "length," "width," "up," "down," "front," "back," "left," "right," "vertical," "horizontal," "top," "bottom," "inside," "outside," and the like, indicating positions or relationships, are based on the positions or relationships shown in the accompanying drawings and are intended only to facilitate the description of the present invention and simplify the description. They do not indicate or imply that the devices or elements referred to must have a specific orientation, be constructed, or operate in a specific orientation. Therefore, they should not be construed as limiting the present invention. Furthermore, in the description of the present invention, "plurality" means two or more, unless otherwise expressly and specifically defined.

[0017] See also Figure 1 , the solid waste intelligent sorting system based on image recognition includes: The confidence score generation module obtains the edge intensity gradient features and grayscale co-occurrence contrast features in the solid garbage image frame sequence and performs a uniform amplitude adjustment. It then compares them with similar features counted in the image samples. Based on the comparison results, it generates a confidence value score for the corresponding candidate label for the current image frame and constructs a scoring path sequence. The main label identification module analyzes the scoring direction of each candidate label in the scoring path sequence and identifies the main label path, and outputs the main label mapping table; The structural anomaly detection module divides the solid waste image into multiple grid structures, extracts the tension index of each grid structure, and marks the areas with abnormal tension index fluctuations as potential structural anomaly areas; The topology repair module performs local repair and reconstruction on the intersection of grid boundaries in the potential structural anomaly area to obtain the repaired topology image; The sorting and classification module combines the repaired topological image and the main label mapping table to classify and identify the solid waste image and obtain the solid waste sorting results; The scoring path sequence includes the label scoring value sequence, the scoring direction change sequence, and the label scoring stability index. The main label mapping table specifically includes the main label recognition results, activation threshold parameters, and label priority order. The potential structural abnormality area includes the boundary continuity mutation area, the tension abnormality concentration area, and the grid deformation expansion area. The repaired topological image specifically refers to the closed boundary path, the connection completion area, and the structural reconstruction fragment. The solid waste sorting result includes the main label classification identification, image structure status, and sorting control instructions.

[0018] See also Figure 2 , the confidence score generation module includes: The image feature normalization submodule obtains solid waste images and performs frame operations, collecting edge intensity gradient features and grayscale co-occurrence contrast features of each frame in the image frame sequence, and maps the two types of features to the unit interval numerical space through the normalization method to generate unified dimension features; The solid garbage image is acquired and framed. The acquired image frame sequence comes from a fixed camera installed above the garbage sorting conveyor belt. The camera continuously acquires images at set time intervals to construct a sequence. After acquiring each frame, the edge intensity gradient feature of the image is first extracted. The Sobel operator is used to calculate the gradient of the pixels in the horizontal and vertical directions. Suppose the horizontal gradient of a pixel in the image is for , vertical gradient for , then its edge intensity gradient Expressed as: ; This operation traverses each pixel in the image and generates a gradient map as the feature representation of the image edge intensity. Then, the grayscale co-occurrence contrast feature is extracted and the image is grayscale quantized to set the grayscale level to 3 to convert the image into grayscale value. 、 、 The two-dimensional matrix is ​​constructed, and then the co-occurrence pixel pairs are counted in the horizontal direction to construct the co-occurrence matrix. Let the obtained Gray-level co-occurrence matrix for ,in and Represents the grayscale value index, Indicates grayscale The pixel and grayscale are The probability that the pixels appear horizontally adjacent in the image, followed by the contrast Calculated using the following formula: ; Item by item calculation is as follows: , , , , , , , , The grayscale contrast feature of the image obtained by summing up all non-zero items is Finally, the image edge intensity gradient eigenvalue and grayscale co-occurrence contrast eigenvalue are normalized and mapped to the unit interval respectively, and combined into the two-dimensional feature vector of the current image frame, forming the unified dimensional feature of the solid garbage image in the current frame.

[0019] Feature label mapping submodule: Project the unified dimension features as two-dimensional coordinate points into the feature similarity space, calculate the Euclidean distance with the candidate label feature coordinates counted in the image sample, and determine the corresponding candidate label confidence score based on the inverse of the distance value; The obtained unified dimension features are used as two-dimensional coordinate points Projected into the feature similarity space, the space stores the statistical feature center coordinates of each candidate label in the image sample library, where 、 Represent the normalized edge intensity gradient and grayscale co-occurrence contrast eigenvalue of the image frame respectively. The two-dimensional feature points of the current frame are , suppose there are three candidate labels A, B, and C preset in the system, and their feature centers are: Label A is , label B is , label C is , calculate the Euclidean distance between the current frame and these three center points respectively, the formula is ; in Indicates the current image frame and label The characteristic distance, Representation Label The center coordinates of , substituting the data into the data are: ; ; .

[0020] Then, a confidence score is constructed based on the inverse value of each distance using the formula ;in, Representation Label The confidence score of To avoid division by zero error for the constant, substitute the following into the calculation: , , The above three categories of candidate labels and their corresponding scores are recorded as the label score data under the current frame. The system repeats the above process on all image frames, builds a confidence score list for each candidate label in turn that expands over time, and finally outputs the candidate label confidence score.

[0021] The scoring path construction submodule calls the candidate label confidence score, arranges the scoring results of the candidate label in each frame in the order of the image frames, and connects the scoring points of the same candidate label in each frame to obtain the scoring path sequence; The confidence score of the candidate label is called, and the corresponding candidate label scores in each frame image are arranged in chronological order to construct the scoring timeline corresponding to the label. During the scoring timeline construction process, the existing candidate label score data in the current frame is extracted one by one for all frame sequences in the order of frame numbers, and the score points are classified and sorted according to the label dimension. Under the condition that the label dimension remains unchanged, the score points of the same label in consecutive frames are connected in chronological order to form a scoring trajectory. The trajectory uses the frame sequence index as the horizontal axis and the score value as the vertical axis to continuously form a scoring trend curve. At the same time, the same operation is performed on all labels to generate multiple independent scoring paths. The paths have the same length and the same time base within the image frame sequence. The scoring path construction process keeps the label identifier and the score value bound unchanged. Ultimately, each path uniquely corresponds to a candidate label. After the system completes all scoring paths, it merges them into a scoring path sequence set. The set is indexed by label order to form a two-dimensional structure for subsequent main label recognition module call processing, and finally a scoring path sequence is obtained.

[0022] See also Figure 3 , the main tag recognition module includes: The score change extraction submodule calls the score path sequence, calculates the score direction difference of each candidate label in consecutive image frames, extracts the score jump frequency, score amplitude and direction continuity parameters corresponding to the label path, and generates the candidate label score change feature; To call the scoring path sequence, you must first use the image processing system to extract the label scoring path from the target video or real-time monitoring image sequence. The scoring path reflects the change in the confidence of a candidate label between image frames. For example, in a vehicle detection scenario, a vehicle is continuously tracked and identified as the same ID label, then its score for each frame is [0.85, 0.87, 0.91, 0.88, 0.82] to form a path. The difference between the score values ​​of each two adjacent frames in the path is calculated to obtain the score direction difference. For example, if the score difference between the first and second frames is 0.02, the calculated direction is positive. If the difference is negative, the direction is downward. The direction change trend of the entire sequence can be statistically calculated. Identify the score direction continuity parameter, which can be expressed as the ratio of the number of positive and negative direction conversions of the score. For example, if the direction changes twice in 5 frames, the direction continuity is 60%. The score jump frequency can be calculated by setting a jump threshold (for example, 0.05) and counting the number of times the score difference exceeds the threshold. For example, in the above score, if the score of the third frame increases by 0.04, it will not be counted as a jump, and if the score of the fourth frame decreases by 0.03, it will still not be counted as a jump. However, if the score of the fifth frame decreases by 0.06, it will be counted as one jump. The jump frequency is 1 / 4=25%. The score amplitude can be the absolute value of the maximum score difference, such as |0.91-0.82|=0.09. All these parameters constitute the score change characteristics of the candidate label.

[0023] The candidate tag screening submodule determines the consistency and fluctuation range of each candidate tag path in the scoring direction based on the candidate tag score change characteristics, screens the path with the most stable scoring direction and the smallest amplitude fluctuation range, and generates the main tag candidate path identifier; Based on the change characteristics of candidate label scores, the score direction consistency of each candidate label path is first calculated. Specifically, the score direction consistency ratio is counted, that is, the ratio of directions that have not changed in a continuous score sequence. For example, if the score direction sequence is [+, +, +, -, -], the first three frames are consistent directions, accounting for 3 / 4=75%. Then, the score amplitude fluctuation range is calculated. The difference between the maximum score value and the minimum score value can be taken to represent the fluctuation range. For example, if the score is [0.91, 0.87, 0.83, 0.85, 0.89], the maximum and minimum difference is 0.91-0.83=0.08. Then set the threshold for high directional consistency to 0.8 and the threshold for small score fluctuation range to 0.05. If the score directional consistency ratio of the candidate label is greater than 0.8 and the fluctuation range is less than 0.05, it is determined to be the path with the most stable score direction and the smallest amplitude fluctuation. For example, label A with a score consistency of 0.9 and a fluctuation of 0.03 is selected as the main label candidate path. Conversely, if label B has a score consistency of 0.7 and a fluctuation of 0.06, it is excluded.

[0024] The main label output submodule sets the confidence activation threshold according to the variance of the main label candidate path identification score path, determines the main label path under the current image frame sequence, and establishes a mapping relationship between the label path and the confidence activation threshold to generate a main label mapping table; The confidence activation threshold is set according to the variance of the score path identified by the main label candidate path. First, the variance of the score value sequence in the selected main label path is calculated. For example, the score sequence is [0.88, 0.87, 0.89, 0.91, 0.88], and its mean is 0.886. The variance is the average of the squares of the differences between each score and the mean, that is, [(0.88-0.886)²+(0.87-0.886)²+…] / 5≈0.00019. The confidence activation threshold can be set based on experience. The value and variance mapping setting, for example, variance < 0.0002 is set to a high confidence threshold of 0.85, variance 0.0002-0.0005 is set to a medium confidence threshold of 0.80, and variance > 0.0005 is set to a low confidence threshold of 0.75. In the above example, the variance is 0.00019, corresponding to a threshold of 0.85, that is, the label path is identified as the main label path in this frame sequence. Finally, the path and the corresponding confidence threshold are combined to form a mapping relationship, such as label ID_15 → confidence 0.85.

[0025] See also Figure 4 , the structural anomaly detection module includes: The grid division submodule obtains the solid waste image area, divides each area into an equilateral triangle grid structure, sequentially numbers the grids to locate the intersection position relationship, and generates an image grid structure coordinate set; To obtain the solid garbage image area, it is necessary to extract the boundary outline of the garbage target through the image segmentation algorithm, then perform regional cropping on the target area, limit the calculation area range, and divide the interior of the area into a grid structure composed of equilateral triangles with a fixed side length (such as 20 pixels). In actual operation, OpenCV is used to draw each edge, and each triangular mesh is numbered from left to right and from top to bottom. The coordinates of each mesh node are calculated by pixel position. For example, the image origin is the upper left corner (0, 0). If the side length of the first mesh is 20 pixels, the first node is (0, 0), the second node is (20, 0), and the third node is (10, 17). In this way, a triangular mesh is constructed and the coordinate relationship between the intersections is recorded. The coordinate values ​​of all nodes are combined into an image grid structure coordinate set, where each coordinate item is represented in the form of a triple, that is, (number, x coordinate, y coordinate). For example, the third point is (3, 40, 34). This structure supports subsequent tension extraction based on geometric relationships.

[0026] The tension extraction submodule calculates three types of indicators: the side length change rate, angle offset, and diagonal length difference corresponding to the intersection position of each grid in the image grid structure coordinate set, summarizes them into unit tension description parameters, and generates grid tension index data; Three indicators are calculated: the rate of change of side length, the angle offset, and the difference in diagonal length corresponding to the intersection position of each grid in the image grid structure coordinate set. Each triangular grid is compared frame by frame in the adjacent frames of the image sequence. The lengths of the three sides of the triangle are extracted and their change ratio relative to the initial frame is calculated. For example, if the initial side length is 20 pixels and the current frame side length is 22 pixels, the change rate is (22-20) / 20=0.1, or 10%. The angle offset is calculated from the three-point coordinates. For example, if the initial angle is 60 degrees and the current frame angle is 66 degrees, the offset is 6 degrees. The diagonal length difference can be calculated by connecting the lengths of non-adjacent triangle vertices. If the initial diagonal is 28 pixels and the current frame is 30 pixels, the difference is 2 pixels. These three indicators are recorded by grid number and summarized into a tension description parameter set. The three parameters of each grid form a tension vector. For example, the tension description of grid A is (10%, 6 degrees, 2 pixels). All grid tension parameters are summarized to generate a grid tension indicator dataset for subsequent structural deformation analysis.

[0027] The abnormal area identification submodule determines the fluctuation direction and amplitude change of each tension index in the continuous grid structure based on the grid tension index data, and selects areas with continuous index deviation and boundary mutation rate greater than the judgment standard, marking them as potential structural abnormal areas; Based on the grid tension index data, the fluctuation direction and amplitude change of each tension index in the continuous grid structure are judged respectively. The tension value of the same numbered grid on the time axis needs to be analyzed by first-order difference. The fluctuation direction is judged by calculating the difference sign of the tension value of consecutive frames. For example, the tension of frame 1 is 0.1, frame 2 is 0.12, the direction is positive, and frame 3 is 0.11, then the direction becomes negative. If the same direction is greater than 3 frames in a row, it is determined to be a trend fluctuation area. Then the fluctuation amplitude is divided into intervals. The high fluctuation interval of the side length change rate is set to >0.2, the medium is 0.1~0.2, The low value is ≤0.1, the medium value of the angle offset is 5-10 degrees, and the high value is >10 degrees. The medium value of the diagonal difference is 1-3 pixels, and the high value is >3 pixels. If all parameters meet the medium or high range at the same time, it is determined to be a potential anomaly. The boundary mutation rate threshold is further set. If the tension difference between a grid and the adjacent grid is greater than the set value (for example, the difference in the tension side length change rate is >0.15), it is considered a boundary mutation. If this mutation rate exceeds 40% of the total number of consecutive grids, it is used as a judgment standard, and the area is marked as a potential structural anomaly area for further tracking or warning prompts.

[0028] See also Figure 5 , the topology repair module includes: The boundary extraction submodule extracts the coordinate sequence of the grid boundary intersection positions within the potential structural anomaly area, identifies the connection loss and angle misalignment information between each intersection position, and generates a fracture boundary intersection coordinate set; To extract the coordinate sequence of the grid boundary intersection positions within the potential structural anomaly region, we first need to obtain the grid numbers marked as anomalies in the previous step. Then, based on the grid structure coordinate set, we extract the boundary intersection coordinates of these grids. For example, if the abnormal grids are numbered 10, 11, and 12 and are located at adjacent positions in the image area, we read the coordinates of the triangle vertices corresponding to the numbers and list the boundary vertex positions in order to form a coordinate sequence. Next, we traverse the adjacent intersection point pairs and check one by one whether there is a connection loss in the image, that is, whether the corresponding edge is blocked, broken, or missing. This can be determined by calculating the grayscale continuity between adjacent points. If the grayscale difference of an edge connection exceeds a set threshold (such as greater than 40), the connection is considered broken. At the same time, angular misalignment is detected in the geometric structure by comparing the difference between the angle of adjacent connection vectors and the standard angle in the ideal grid (such as 60 degrees). If the difference is greater than 10 degrees, it is considered an angular misalignment. This information is combined to form a broken boundary intersection coordinate set. Each record contains the coordinates of two intersection points, the connection status, and the angular deviation value.

[0029] The connection generation submodule sets interpolation rules based on the fracture boundary intersection coordinate set and the connection angle deviation and edge length similarity. It then adds auxiliary intersection points at the missing connection locations, establishes auxiliary connection chains according to the shortest connection path, and generates boundary reconstruction connection paths. Based on the fracture boundary intersection coordinate set, the connection angle deviation and side length similarity are first judged for each set of fracture coordinate pairs, and the angle difference between the actual connection vector and the ideal direction is calculated. For example, if the deviation between the actual connection direction and the adjacent triangle side direction is 12 degrees, it is within the acceptable range (if the upper limit of the deviation is set to 15 degrees). The side length similarity is compared by comparing the distance between the two intersection points with the average side length of the region. For example, if the average side length is 20 pixels and the current distance is 22 pixels, the relative deviation is (22-20) / 20=0.1, that is, 10%. If the side length similarity threshold is set If the value is within 20%, it is considered a match. After the interpolation connection conditions are met, an auxiliary intersection point is inserted between the two intersection points. The coordinates of the interpolation point are obtained according to the midpoint of the line segment or piecewise linear interpolation. For example, interpolation between coordinates (100, 120) and (140, 130) gives the midpoint (120, 125). All interpolation points are connected to form an auxiliary connection chain. To reduce the path length, the group with the shortest total path length is selected from multiple connectable paths as the final connection chain. This connection chain combines the coordinates of the original point and the auxiliary point to generate a boundary reconstruction connection path.

[0030] The structure output submodule combines and updates the boundary reconstruction connection path with the original grid structure of the image through the triangulation algorithm, performs closed repair processing on the abnormal area boundary, and obtains the repaired topological image; The boundary reconstruction connection path is combined with the original grid structure of the image through the triangulation algorithm. First, the original grid structure node coordinates in the image (for example, 120 points) are merged with the auxiliary intersection points (for example, 12) added by the connection generation submodule to generate a grid containing The coordinates of each point are expressed in pixels, for example 、 All points are organized in a numbered index format to prepare for subsequent triangulation.

[0031] In the specific triangulation process, the Delaunay algorithm is used, and its core judgment condition is: if any three points 、 、 To construct a triangle, determine whether its circumscribed circle contains other points and calculate the center of the circumscribed circle and the denominator used for determination The formula used is as follows: ; The meaning of each parameter is: :point The horizontal and vertical coordinates of :point The horizontal and vertical coordinates of :point The horizontal and vertical coordinates of : The common denominator term in the formula for the circumcircle of a triangle.

[0032] when Calculate the coordinates of the center of the circumscribed circle as follows: ; ; Among these expressions, Indicates a point The squared distance to the origin can be used to describe the positional relationship between a point and the center of a circumscribed circle.

[0033] Next, to verify whether the triangle meets the Delaunay condition, calculate any other point Distance to the center of the circumcircle ,like If the distance is greater than the distance from any vertex of the triangle to the center of the circle (that is, the radius of the circumscribed circle), then the triangle is legal.

[0034] Let three points be: , , .

[0035] calculate: ; ; ; Therefore, the center of the circle is , the radius is any point (such as ) to the center of the circle .

[0036] After completing the generation of all legal triangular units, the system will enter the image grid structure update and closure repair process. This process requires first identifying the boundary grid units adjacent to the abnormal area in the original image grid, extracting their numbers and coordinates, and performing boundary docking operations with the newly added grid units through triangulation. The specific operation process is: first, all vertices in the newly added triangular units are arranged in order, and the end-to-end connection is matched with the vertex sequence of the original grid units. If any two vertices are not connected by an existing edge in the image, a connecting edge is automatically added and its attribute is set to "auxiliary edge". The auxiliary edge will be marked as a fictitious connection by the system to avoid it being mistakenly identified as a real structural edge in subsequent analysis. Then, according to the angle and overlap ratio between all new edges and the old grid edges, it is judged whether the boundary has formed a closed state. If there is an opening in the loop (that is, the edge is not completely closed), the connecting segments are inserted at both ends of the opening to form a closed loop with the minimum area to complete the closed repair operation. In the process of constructing the closed structure, the continuity and direction consistency of the edges must be maintained to ensure that each triangle can form a face mesh data structure with its three vertices and adjacent units. Finally, the structural information of these newly added triangles - including three-point coordinates, edge connection sequence, triangle number, and the index of the repair area to which they belong - is written into the topological structure diagram, and the unit structure of the abnormal area in the original image grid is marked as "replaced", indicating that the area has been repaired. The system will then output the completely updated image grid structure, in which the original damaged part of the abnormal area has been replaced by the newly constructed triangular unit to form a coherent, complete, and closed topological image.

[0037] See also Figure 6 , the sorting and classification modules include: The path matching submodule calls the repaired topology image and the main label mapping table, extracts the confidence score corresponding to the structural connection path and the main label path, performs path structure consistency and score threshold judgment, and generates the main label matching judgment result; The repaired topology image and the main label mapping table are called. First, all connection paths in the repaired image are extracted. This path consists of the connected structure generated by the aforementioned triangulation and closed repair. Each path contains a set of continuous triangle units and a sequence of their constituent vertex coordinates. Then, the main label path information is retrieved from the main label mapping table. The corresponding main label path contains the topological number sequence of its start and end nodes and the associated confidence score. The system searches the topological graph structure one by one to see whether there is a path that matches the node order, edge connection direction, and topological structure type (such as linear, tree, or grid) of the main label path structure. If the structural path is a complete match or can be considered an approximate match within the tolerance range, a scoring threshold judgment is performed. The scoring threshold required for main label matching is set to 0.80. If the confidence score of the main label path is higher than this value, it is considered a match; if it is lower, it is considered an unmatched match. Finally, the main label matching judgment result is output, and the matching label number and judgment flag are recorded for subsequent label confirmation processing.

[0038] The label confirmation submodule selects the main labels with scores higher than the activation threshold based on the main label matching results, confirms the garbage category corresponding to the target main label as the recognition output of the current image, and generates solid garbage identification label data; Based on the main label matching judgment result, all label paths that are determined to be matched successfully are traversed and their scores are screened to see if they are higher than the activation threshold. The threshold comes from the scoring standard set by the main label output submodule mentioned above. For example, if the activation threshold is set to 0.85, only labels with scores at or above this value are retained. For example, labels ID_05 with a score of 0.89 and ID_07 with a score of 0.91 meet the conditions, while ID_12 with a score of 0.82 is eliminated. The corresponding label content of the filtered label path is indexed and matched with the classification binding relationship in the garbage identification database to confirm the garbage type represented by the main label. For example, ID_07 corresponds to the "plastic bottle" category and ID_05 corresponds to the "glass bottle" category. The final output target main label is the path label with the highest score and passes the activation threshold filtering. The system identifies it as the main garbage type contained in the current image and generates solid garbage identification label data. This data structure includes information such as label number, category name, score value and path node index sequence for subsequent sorting.

[0039] The result generation submodule calls the solid waste identification label data, uses the identification label as the classification basis to generate matching sorting instructions, and uses the conditional random field to perform image-level classification judgment and action binding to obtain the solid waste sorting results; Call the solid waste identification label data, take the main label of the repaired grid area in each image as the initial classification candidate, and build a graph structure model ,in An abstract graph model representing the entire image structure, Represents the set of all image grid region nodes (such as ), Represents the set of spatial adjacent edges between these nodes, each node Represents a spatially continuous structural region in the image, where each edge Indicates that there is a direct adjacency relationship between two nodes. For each node in the graph , all have an observed eigenvector , the vector includes the main label score, local structure tension parameters and texture grayscale gradient, etc.; each node also corresponds to a label , indicating the type of garbage identified in the area (such as plastic bottles, metal cans, plastic bags, etc.).

[0040] In the conditional random field model, the observation feature set Represents each node in the graph structure The sum of the observation data of is a node The corresponding local image feature vector, including the main label score, regional structure tension parameter, local texture gradient value, etc., is a quantitative expression of each grid area in the image in spatial and semantic dimensions; and the label configuration set It represents the garbage category labels assigned to all nodes in the graph, such as "plastic bottles" and "metal cans", etc. and the corresponding correspondence; symbol Indicates the current node Is a set of graph nodes members, It is the complete set of all regional nodes in the entire graph structure, including all grid areas with significant structures in the image; the symbol Indicates that there is a pair of nodes in the graph structure and There is a spatial adjacency relationship between them, which constitutes an edge in the graph. The edge set It is the sum of direct connections between all regions in the graph structure, which is used to capture the mutual influence between adjacent regions in spatial structure and label reasoning; therefore, the term in the energy function For all nodes The label and its observed features are penalized for consistency, and Indicates the cumulative joint penalty for the inconsistency between label configuration and feature relationship of all adjacent node pairs in the graph. These two parts together constitute the total energy , determines the label configuration Relative to feature observation rationality and possibility.

[0041] The Conditional Random Field (CRF) model is used to make an overall judgment on the labels of all regions. The goal is to solve the problem of Under the condition of The conditional probability of , defined as follows: ; in: : The label configuration set of all regions in the entire image; : The set of observation features of all regional nodes; : Normalization factor, representing all possible label combinations The exponentially weighted sum of : The total energy of the label configuration, indicating its unreasonableness. The smaller it is, the more reasonable it is.

[0042] Total energy function The definition is as follows: ; in: : Single point potential function, measuring area Label Its observation characteristics The higher the score, the lower the cost, which is calculated as ,in is the rating value; : Point potential function, measuring the adjacent area 、 Label 、 The coordination degree is set based on the consistency of structural differences and labels: 0.3 for structural similarity and consistent labels, 1.5 for structural similarity and inconsistent labels, 2.0 for structural difference and consistent labels, and 0.3 for structural difference and inconsistent labels; : Indicates the numbers of two nodes that have an adjacency relationship; :respectively nodes 、 Observational characteristics of :respectively nodes 、 The label type.

[0043] Assume that the system extracts the following regional nodes and their observation data from the image: :score ,tension , texture ; :score ,tension , texture ; :score ,tension , texture ; :score ,tension , texture ; The single-point potential function is calculated as: , , , .

[0044] Consider 4 sets of label combinations to , the candidate labels are A=plastic bottle, B=metal can, C=plastic bag, and the adjacent edges are 、 , the energy of each combination is calculated as follows: combination : :Similar structure, consistent labels → ; :Structural differences, inconsistent labels → ; ; .

[0045] combination : :Similar structures, inconsistent labels → ; :Structural differences, inconsistent labels → ; ; .

[0046] combination : :Similar structure, consistent labels → :Structural differences, inconsistent labels → ; .

[0047] combination : :Similar structure, consistent labels → ; :Structural differences, consistent labels→ ; ; .

[0048] The normalization factor is: , The conditional probability is: , , , .

[0049] The final system is selected from the above four groups or (with the highest conditional probability) as the final label configuration: areas 1 and 2 are classified as "plastic bottles", area 3 as "metal cans", and area 4 as "plastic bags". The system maps this configuration with the hardware action instruction library, such as A1, B3, C2, etc., and outputs the image number, area index, label and action code for the garbage sorting equipment to perform separation operations.

[0050] The above are merely preferred embodiments of the present invention and do not limit the present invention in any other form. Any technician familiar with the profession may use the technical content disclosed above to change or modify it into an equivalent embodiment with equivalent changes and apply it to other fields. However, any simple modification, equivalent change and modification made to the above embodiment based on the technical essence of the present invention without departing from the content of the technical solution of the present invention shall still fall within the scope of protection of the technical solution of the present invention.

Claims

1. The solid waste intelligent sorting system based on image recognition is characterized by: The system comprises: The confidence score generation module obtains the edge intensity gradient features and grayscale co-occurrence contrast features in the solid garbage image frame sequence and performs a uniform amplitude adjustment. It then compares them with similar features counted in the image samples. Based on the comparison results, it generates a confidence value score for the corresponding candidate label for the current image frame and constructs a scoring path sequence. The main label identification module analyzes the scoring direction of each candidate label in the scoring path sequence and identifies the main label path, and outputs a main label mapping table; The structural anomaly detection module divides the solid waste image into multiple grid structures, extracts the tension index of each grid structure, and marks the areas with abnormal tension index fluctuations as potential structural anomaly areas; The topology repair module performs local repair and reconstruction on the intersection of the grid boundaries in the potential structural abnormality area to obtain a repaired topology image; The sorting and classification module combines the repaired topological image with the main label mapping table to classify and identify the solid waste image to obtain a solid waste sorting result.

2. The solid waste intelligent sorting system based on image recognition according to claim 1 is characterized in that: The scoring path sequence includes a label scoring value sequence, a scoring direction change sequence, and a label scoring stability index; the main label mapping table specifically includes a main label recognition result, an activation threshold parameter, and a label priority order; the potential structural abnormality area includes a boundary continuity mutation area, a tension abnormality concentration area, and a grid deformation expansion area; the repaired topological image specifically refers to a closed boundary path, a connection completion area, and a structural reconstruction fragment; the solid waste sorting result includes a main label classification identifier, an image structure status, and a sorting control instruction.

3. The solid waste intelligent sorting system based on image recognition according to claim 1 is characterized in that: The confidence score generation module includes: The image feature normalization submodule obtains solid waste images and performs frame operations, collecting edge intensity gradient features and grayscale co-occurrence contrast features of each frame in the image frame sequence, and maps the two types of features to the unit interval numerical space through the normalization method to generate unified dimension features; Feature label mapping submodule: Project the unified dimension feature as a two-dimensional coordinate point into the feature similarity space, calculate the Euclidean distance with the candidate label feature coordinates counted in the image sample, and determine the corresponding candidate label confidence score according to the inverse of the distance value; The scoring path construction submodule calls the candidate tag confidence score, arranges the scoring results of the candidate tags in each frame in sequence according to the image frame order, and connects the scoring points of the same candidate tag in each frame to obtain a scoring path sequence.

4. The solid waste intelligent sorting system based on image recognition according to claim 3 is characterized in that: The main tag identification module includes: The score change extraction submodule calls the score path sequence, calculates the score direction difference of each candidate tag in consecutive image frames, extracts the score jump frequency, score amplitude and direction continuity parameters corresponding to the tag path, and generates the candidate tag score change feature; The candidate tag screening submodule determines the consistency and fluctuation range of each candidate tag path in the scoring direction based on the candidate tag score change characteristics, screens the path with the most stable scoring direction and the smallest amplitude fluctuation range, and generates a main tag candidate path identifier; The main label output submodule sets the confidence activation threshold according to the variance of the main label candidate path identification score path, determines the main label path under the current image frame sequence, and establishes a mapping relationship between the label path and the confidence activation threshold to generate a main label mapping table.

5. The solid waste intelligent sorting system based on image recognition according to claim 4 is characterized in that: The structural anomaly detection module includes: The grid division submodule obtains the solid waste image area, divides each area into an equilateral triangle grid structure, sequentially numbers the grids to locate the intersection position relationship, and generates an image grid structure coordinate set; The tension extraction submodule calculates three types of indicators, namely, the side length change rate, the angle offset, and the diagonal length difference corresponding to the intersection position of each grid in the image grid structure coordinate set, summarizes them into unit tension description parameters, and generates grid tension index data; The abnormal area identification submodule judges the fluctuation direction and amplitude change of each tension index in the continuous grid structure based on the grid tension index data, selects areas with continuous index deviation and boundary mutation rate greater than the judgment standard, and marks them as potential structural abnormal areas.

6. The solid waste intelligent sorting system based on image recognition according to claim 5 is characterized in that: The topology repair module includes: The boundary extraction submodule extracts the coordinate sequence of the grid boundary intersection position in the potential structural anomaly area, identifies the connection loss and angle misalignment information between each intersection position, and generates a fracture boundary intersection coordinate set; The connection generation submodule sets interpolation rules based on the fracture boundary intersection coordinate set according to the connection angle deviation and the side length similarity, adds auxiliary intersection points at the missing connection positions, establishes auxiliary connection chains according to the shortest connection path, and generates a boundary reconstruction connection path; The structure output submodule combines and updates the boundary reconstruction connection path with the original grid structure of the image through a triangulation algorithm, performs closed repair processing on the boundary of the abnormal area, and obtains a repaired topological image.

7. The solid waste intelligent sorting system based on image recognition according to claim 6 is characterized in that: The sorting and classification module includes: The path matching submodule calls the repaired topology image and the main label mapping table, extracts the confidence scores corresponding to the structural connection paths and the main label paths, performs path structure consistency and score threshold judgment, and generates a main label matching judgment result; The label confirmation submodule selects the main labels with scores higher than the activation threshold based on the main label matching judgment result, confirms the garbage category corresponding to the target main label as the recognition output of the current image, and generates solid garbage identification label data; The result generation submodule calls the solid waste identification label data, uses the identification label as the classification basis to generate matching sorting instructions, performs image-level classification judgment and action binding through conditional random fields, and obtains the solid waste sorting results.

Images