Intelligent solid waste sorting system based on image recognition

Abstract

The present application relates to the technical field of image processing, in particular to a solid waste intelligent sorting system based on image recognition. In the present application, the edge gradient intensity and the gray level co-occurrence contrast in the image are uniformly mapped to the same scale space, and the confidence score path is constructed by combining the image frame sequence, which can improve the stability and distinguishability of label scoring in solid waste image recognition. Through the scoring direction and fluctuation analysis mechanism, the main trend label in the scoring path can be mined, which can enhance the main label judgment accuracy of image classification. Through grid division and tension index extraction, fine-grained recognition of structural abnormalities of garbage images is realized, so that abnormal structures such as deformation and fracture in the image can be marked and positioned. On this basis, the boundary structure of the image is updated through connection reconstruction and triangular subdivision, and the effective repair of the abnormal area is realized. Combined with the consistency evaluation of the main label path and the topological structure, it is helpful to improve the robustness of the recognition output and the structural integrity of the classification result.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of image processing, and in particular to an intelligent solid waste sorting system based on image recognition. BACKGROUND

[0002] The technical field of image processing mainly involves the acquisition, analysis, recognition and processing of images, covering image acquisition, preprocessing, feature extraction, image recognition and understanding, image enhancement, image segmentation, image compression and reconstruction, and other technical directions, with the goal of realizing intelligent analysis and effective utilization of image content.

[0003] Among them, the traditional solid waste intelligent sorting system based on image recognition refers to a system that realizes the judgment of waste types by collecting and recognizing solid waste images, and guides the sorting equipment to perform corresponding separation operations accordingly, mainly for how to accurately and timely automatically classify solid waste under complex composition and variable appearance conditions.

[0004] When the prior art determines the solid waste image in a complex environment, it relies on image acquisition and preliminary recognition to directly classify the types, lacks detection and processing capability for image structure abnormalities, and leads to label judgment errors in cases such as image boundary blur, distorted configuration, and target defects. When different label scores in consecutive image frames are unstable, it is difficult to establish a label evolution trend path, resulting in the inability of the recognition logic to continuously track effective label changes. For example, when there are obvious cracks or obstructions in the image, the system often cannot recognize and repair missing structures, resulting in classification interruption or misleading the sorting equipment to perform incorrect actions. In addition, under the interference of label signal noise, the existing system is difficult to confirm a stable label path, thereby affecting the final classification reliability and system response efficiency. SUMMARY

[0005] The purpose of the present application is to solve the shortcomings in the prior art, and to provide an intelligent solid waste sorting system based on image recognition.

[0006] In order to achieve the above-mentioned purpose, the present application adopts the following technical scheme: the intelligent solid waste sorting system based on image recognition comprises:

[0007] The confidence score generation module obtains the edge intensity gradient feature and the gray co-occurrence contrast feature in the solid waste image frame sequence and performs unified amplitude adjustment, compares with the same type of features statistically in the image sample, generates a confidence value score for the candidate label corresponding to the current image frame according to the comparison result, and constructs a score path sequence;

[0008] The main label recognition module analyzes the score direction of each candidate label in the score path sequence and identifies the main label path, and outputs a main label mapping table;

[0009] The structural anomaly detection module divides the solid waste image into a plurality of grid structures, extracts a tension index of each grid structure, and marks a region with a fluctuation anomaly of the tension index as a potential structural anomaly region;

[0010] The topology repair module locally repairs and reconstructs a grid boundary intersection position in the potential structural anomaly region to obtain a repaired topology image;

[0011] The sorting and classification module combines the repaired topology image and the main label mapping table to classify and identify the solid waste image, and obtains a solid waste sorting result.

[0012] As a further scheme of the present application, the score path sequence includes a label score value sequence, a score direction change sequence, and a label score stability index; the main label mapping table specifically refers to a main label identification result, an activation threshold parameter, and a label priority order; the potential structural anomaly region includes a boundary continuity mutation region, a tension anomaly concentration region, and a grid deformation expansion region; the repaired topology image specifically refers to a closed boundary path, a connection completion region, and a structure reconstruction segment; and the solid waste sorting result includes a main label classification identifier, an image structure state, and a sorting control instruction.

[0013] As a further scheme of the present application, the confidence score generation module includes:

[0014] The image feature normalization submodule acquires the solid waste image and performs a frame division operation, collects an edge intensity gradient feature and a gray co-occurrence contrast feature of each image in the image frame sequence, respectively maps the two types of features to a unit interval numerical space through a normalization method, and generates a unified dimension feature;

[0015] The feature label mapping submodule: projects the unified dimension feature as a two-dimensional coordinate point to a feature similarity space, performs an Euclidean distance calculation on a candidate label feature coordinate statistically collected in the image sample, and determines a corresponding candidate label confidence score according to an inverse distance value;

[0016] The score path construction submodule calls the candidate label confidence score, arranges score results of a candidate label in each frame in sequence according to an image frame order, respectively connects score points of the same candidate label on each frame, and obtains a score path sequence.

[0017] As a further scheme of the present application, the main label identification module includes:

[0018] The score change extraction submodule calls the score path sequence, calculates a score direction difference of each candidate label in consecutive image frames, extracts a score jump frequency, a score amplitude, and a direction continuity parameter corresponding to a label path, generates a candidate label score change feature, and inputs the candidate label score change feature into the label confidence score calculation submodule.

[0019] The candidate label screening submodule judges the consistency degree and the fluctuation range of each candidate label path in the scoring direction based on the candidate label score change feature, screens the path with the most stable scoring direction and the smallest amplitude fluctuation range, and generates a main label candidate path identifier;

[0020] The main label output submodule sets a confidence activation threshold according to the main label candidate path identifier score path variance, determines the main label path under the current image frame sequence, establishes a mapping relationship with the label path and the confidence activation threshold, and generates a main label mapping table.

[0021] As a further scheme of the present application, the structure anomaly detection module comprises:

[0022] The grid division submodule obtains a solid waste image region, divides each region into an equilateral triangle grid structure, sequentially numbers the grid positioning intersection position relationship, and generates an image grid structure coordinate set;

[0023] The tension extraction submodule calculates three types of indexes including the edge length change rate, the angle offset, and the diagonal length difference corresponding to the intersection position in each grid in the image grid structure coordinate set, aggregates the unit tension description parameters, generates grid tension index data, and outputs the grid tension index data.

[0024] The anomaly region identification submodule judges the fluctuation direction and amplitude change of each tension index in the continuous grid structure based on the grid tension index data, screens the region with the index continuous offset and the boundary mutation rate greater than the judgment standard, and marks it as a potential structure anomaly region.

[0025] As a further scheme of the present application, the topology repair module comprises:

[0026] The boundary extraction submodule extracts the coordinate sequence of the grid boundary intersection position in the potential structure anomaly region, identifies the connection loss and angle dislocation information between each intersection position, generates a broken boundary intersection coordinate set, and outputs the broken boundary intersection coordinate set.

[0027] The connection generation submodule sets an interpolation rule according to the connection angle deviation and the edge length similarity based on the broken boundary intersection coordinate set, sets an auxiliary intersection point at the missing connection position, establishes an auxiliary connection chain according to the shortest connection path, generates a boundary reconstruction connection path, and outputs the boundary reconstruction connection path.

[0028] The structure output submodule combines and updates the boundary reconstruction connection path and the original image grid structure through a triangulation algorithm, performs a closed repair processing on the anomaly region boundary, and obtains a repaired topology image.

[0029] As a further scheme of the present application, the sorting and classification module comprises:

[0030] The path matching sub-module calls the repaired topological image and the main label mapping table, extracts the confidence score corresponding to the structural connection path and the main label path, performs path structure consistency and score threshold judgment, and generates a main label matching determination result;

[0031] The label confirmation sub-module filters the main labels with scores higher than the activation threshold based on the main label matching determination result, confirms the garbage category corresponding to the target main label as the recognition output of the current image, and generates solid waste recognition label data;

[0032] The result generation sub-module calls the solid waste recognition label data, generates matched sorting instructions based on the recognition label as the classification basis, performs image-level classification judgment and action binding through conditional random fields, and obtains solid waste sorting results.

[0033] Compared with the prior art, the advantages and positive effects of the present application are:

[0034] In the present application, by uniformly mapping the edge gradient intensity and the gray level co-occurrence contrast in the image to the same scale space and combining the image frame sequence to construct the confidence score path, the stability and discrimination of the label score in solid waste image recognition can be improved. By analyzing the main trend label in the score path through the score direction and fluctuation analysis mechanism, the main label judgment accuracy of image classification can be enhanced. Through grid division and tension index extraction, the abnormal structure of the garbage image structure can be identified in detail, so that the abnormal structure such as deformation and fracture in the image can be marked and positioned. On this basis, the boundary structure of the image is updated through connection reconstruction and triangular partitioning, the abnormal area is effectively repaired, the consistency of the main label path and the topological structure is evaluated, which helps to improve the robustness of the recognition output and the structural integrity of the classification result. Finally, the sorting instruction with high matching degree is generated through the image-level recognition result, and the accuracy and response efficiency of the sorting action are enhanced. BRIEF DESCRIPTION OF DRAWINGS

[0035] Figure 1 The system flowchart of the present application;

[0036] Figure 2 The flowchart of the confidence score generation module of the present application;

[0037] Figure 3 The flowchart of the main label recognition module of the present application;

[0038] Figure 4 The flowchart of the structure abnormality detection module of the present application;

[0039] Figure 5 The flowchart of the topological repair module of the present application;

[0040] Figure 6 The flowchart of the sorting classification module of the present application. DETAILED DESCRIPTION

[0041] In order to make the purpose, technical solutions and advantages of the present invention more clearly understood, the present invention will be further described in detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present invention and are not intended to limit the present invention.

[0042] In the description of the present invention, it should be understood that the terms "length," "width," "up," "down," "front," "back," "left," "right," "vertical," "horizontal," "top," "bottom," "inside," "outside," and the like, indicating positions or relationships, are based on the positions or relationships shown in the accompanying drawings and are intended only to facilitate the description of the present invention and simplify the description. They do not indicate or imply that the devices or elements referred to must have a specific orientation, be constructed, or operate in a specific orientation. Therefore, they should not be construed as limiting the present invention. Furthermore, in the description of the present invention, "plurality" means two or more, unless otherwise expressly and specifically defined.

[0043] See also Figure 1 , the solid waste intelligent sorting system based on image recognition includes:

[0044] The confidence score generation module obtains the edge intensity gradient features and grayscale co-occurrence contrast features in the solid garbage image frame sequence and performs a uniform amplitude adjustment. It then compares them with similar features counted in the image samples. Based on the comparison results, it generates a confidence value score for the corresponding candidate label for the current image frame and constructs a scoring path sequence.

[0045] The main label identification module analyzes the scoring direction of each candidate label in the scoring path sequence and identifies the main label path, and outputs the main label mapping table;

[0046] The structural anomaly detection module divides the solid waste image into multiple grid structures, extracts the tension index of each grid structure, and marks the areas with abnormal tension index fluctuations as potential structural anomaly areas;

[0047] The topology repair module performs local repair and reconstruction on the intersection of grid boundaries in the potential structural anomaly area to obtain the repaired topology image;

[0048] The sorting and classification module combines the repaired topological image and the main label mapping table to classify and identify the solid waste image and obtain the solid waste sorting results;

[0049] The score path sequence includes a label score value sequence, a score direction change sequence and a label score stability index. The main label mapping table specifically refers to a main label identification result, an activation threshold parameter and a label priority order. The potential structural abnormal area includes a boundary continuity mutation area, a tension abnormal concentration area and a grid deformation expansion area. The repaired topological image specifically refers to a closed boundary path, a connection completion area and a structure reconstruction fragment. The solid waste sorting result includes a main label classification mark, an image structure state and a sorting control instruction.

[0050] Please refer to Figure 2 , the confidence score generation module includes:

[0051] The image feature normalization submodule acquires the solid waste image and performs a frame division operation. The edge intensity gradient features and the gray level co-occurrence contrast features of each image in the image frame sequence are collected. The two types of features are respectively mapped to the unit interval numerical space by a normalization method to generate uniform dimension features.

[0052] The acquired image frame sequence is obtained by performing a frame division operation on the solid waste image. The image frame sequence is collected from a fixed camera device installed above a waste sorting conveying belt. The device continuously collects images at a set time interval to construct a sequence. After each image is acquired, the image is first subjected to edge intensity gradient feature extraction. The Sobel operator is used to calculate the gradient of the pixels along the horizontal and vertical directions. Let the horizontal gradient of a pixel point in the image be , , and the vertical gradient be , , then the edge intensity gradient of the pixel point is expressed as:

[0053] ;

[0054] This operation traverses each pixel in the image to generate a gradient map as a feature representation of the image edge intensity. Then, the gray level co-occurrence contrast feature is extracted. The image is subjected to gray level quantization with a gray level of 3. The image is converted into a two-dimensional matrix composed of gray values , , . Then, the co-occurrence matrix is constructed by counting the co-occurrence of pixels in the horizontal direction. Let the obtained gray level co-occurrence matrix be , , , where and represent the gray value index, represents the probability of the pixel with a gray value of and the pixel with a gray value of occurring horizontally adjacent in the image. Then, the contrast is calculated according to the following formula: ​; are calculated as follows: , , , , , , , , . All non-zero terms are accumulated to obtain the gray contrast feature of the image , and finally the image edge intensity gradient feature value and the gray co-occurrence contrast feature value are normalized and mapped to the unit interval, respectively, and combined into a two-dimensional feature vector of the current image frame, forming a unified dimension feature of the solid waste image under the current frame.

[0055] The feature label mapping submodule: the unified dimension feature is projected as a two-dimensional coordinate point to the feature similarity space, and the Euclidean distance is calculated with the candidate label feature coordinates statistically in the image sample. The reciprocal of the distance value determines the confidence score of the corresponding candidate label;

[0056] The obtained unified dimension feature is projected as a two-dimensional coordinate point to the feature similarity space, which stores the statistical feature center coordinate points of each candidate label in the image sample library, wherein , respectively represent the normalized edge intensity gradient and gray co-occurrence contrast feature values of the image frame, and the two-dimensional feature point of the current frame is , and the feature center points of the three candidate labels A, B, and C in the system are respectively: label A is , label B is , and label C is , the Euclidean distances between the current frame and the three center points are calculated respectively, and the formula is

[0057] ;

[0058] wherein represents the feature distance between the current image frame and the label , represents the center coordinate of the label , and the data is substituted to obtain:

[0059] ;

[0060] ;

[0061] .

[0062] Subsequently, the confidence score is constructed based on the reciprocal value of each distance, and the formula is used; wherein, represents the label The confidence score of To avoid division by zero error for the constant, substitute the following into the calculation: , , The above three categories of candidate labels and their corresponding scores are recorded as the label score data under the current frame. The system repeats the above process on all image frames, builds a confidence score list for each candidate label in turn that expands over time, and finally outputs the candidate label confidence score.

[0063] The scoring path construction submodule calls the candidate label confidence score, arranges the scoring results of the candidate label in each frame in the order of the image frames, and connects the scoring points of the same candidate label in each frame to obtain the scoring path sequence;

[0064] The confidence score of the candidate label is called, and the corresponding candidate label scores in each frame image are arranged in chronological order to construct the scoring timeline corresponding to the label. During the scoring timeline construction process, the existing candidate label score data in the current frame is extracted one by one for all frame sequences in the order of frame numbers, and the score points are classified and sorted according to the label dimension. Under the condition that the label dimension remains unchanged, the score points of the same label in consecutive frames are connected in chronological order to form a scoring trajectory. The trajectory uses the frame sequence index as the horizontal axis and the score value as the vertical axis to continuously form a scoring trend curve. At the same time, the same operation is performed on all labels to generate multiple independent scoring paths. The paths have the same length and the same time base within the image frame sequence. The scoring path construction process keeps the label identifier and the score value bound unchanged. Ultimately, each path uniquely corresponds to a candidate label. After the system completes all scoring paths, it merges them into a scoring path sequence set. The set is indexed by label order to form a two-dimensional structure for subsequent main label recognition module call processing, and finally a scoring path sequence is obtained.

[0065] See also Figure 3 , the main tag recognition module includes:

[0066] The score change extraction submodule calls the score path sequence, calculates the score direction difference of each candidate label in consecutive image frames, extracts the score jump frequency, score amplitude and direction continuity parameters corresponding to the label path, and generates the candidate label score change feature;

[0067] The score path sequence is called, and the label score path is extracted from the target video or real-time monitoring image sequence by the image processing system, which reflects the confidence change of a candidate label between image frames. For example, in a vehicle detection scenario, a vehicle is continuously tracked and identified as the same ID label, and its score for each frame forms a path, such as [0.85, 0.87, 0.91, 0.88, 0.82]. The score difference between each two adjacent frames in the path is calculated to obtain the score direction difference. For example, the score difference between the first frame and the second frame is 0.02, and the calculation direction is positive. If the difference is negative, the direction is downward. The direction change trend of the entire sequence is calculated to identify the score direction continuity parameter. This parameter can be represented as the proportion of the number of times the score positive and negative direction changes. For example, if the direction changes 2 times in 5 frames, the direction continuity is 60%. The score jump frequency can be set by setting a jump threshold (for example, 0.05). The number of times the score difference exceeds the threshold is counted, such as the third frame in the above score, which increases by 0.04 and is not counted as a jump, and the fourth frame decreases by 0.03 and is still not counted as a jump, but the fifth frame decreases by 0.06 and is counted as a jump 1 time. The jump frequency is 1 / 4=25%. The score amplitude can be the absolute value of the maximum score difference, such as |0.91-0.82|=0.09. All these parameters constitute the score change characteristics of the candidate label.

[0068] The candidate label screening sub-module judges the consistency degree and fluctuation range of each candidate label path in the score direction based on the score change characteristics of the candidate label, screens the path with the most stable score direction and the smallest amplitude fluctuation range, and generates a main label candidate path identifier;

[0069] Based on the score change characteristics of the candidate label, the score direction consistency of each candidate label path is calculated. Specifically, the proportion of consistent direction is calculated, that is, the proportion of the number of frames with consistent direction in the continuous score sequence. For example, the score direction sequence is [+,+,+,-,-], the first 3 frames have consistent direction, and the proportion is 3 / 4=75%. The score amplitude fluctuation range is calculated, which can be represented by the difference between the maximum and minimum score values. For example, the score is [0.91, 0.87, 0.83, 0.85, 0.89], the maximum and minimum difference is 0.91-0.83=0.08. The threshold for high direction consistency is set to 0.8, and the threshold for small score fluctuation range is set to 0.05. If the score direction consistency proportion of the candidate label is greater than 0.8 and the fluctuation range is less than 0.05, it is determined that the path has the most stable score direction and the smallest amplitude fluctuation. For example, label A has a score consistency of 0.9 and a fluctuation of 0.03, so it is selected as the main label candidate path. Otherwise, if label B has a score consistency of 0.7 and a fluctuation of 0.06, it is excluded.

[0070] The main label output sub-module sets a confidence activation threshold according to a main label candidate path identification score path variance, determines a main label path under a current image frame sequence, and establishes a mapping relationship with a label path and a confidence activation threshold to generate a main label mapping table;

[0071] According to the main label candidate path identification score path variance, first, the variance of the score value sequence in the selected main label path is calculated. For example, the score sequence is [0.88, 0.87, 0.89, 0.91, 0.88], the mean value is 0.886, and the variance is the average value of the square of the difference between each score and the mean value, that is, [(0.88-0.886)²+(0.87-0.886)²+…] / 5≈0.00019. The confidence activation threshold can be set based on the experience value and the variance mapping. For example, the variance <0.0002 is set as a high confidence threshold of 0.85, the variance 0.0002~0.0005 is set as a medium confidence of 0.80, and the variance >0.0005 is set as a low confidence of 0.75. In the above example, the variance is 0.00019, and the corresponding threshold is 0.85. That is, the label path is identified as the main label path in the current frame sequence. Finally, the path and the corresponding confidence threshold are combined to form a mapping relationship, such as label ID_15→confidence 0.85.

[0072] Please refer to Figure 4 The structural anomaly detection module comprises:

[0073] The grid division sub-module obtains a solid waste image region, divides each region into an equilateral triangle grid structure, sequentially numbers the grid to locate the intersection position relationship, and generates an image grid structure coordinate set;

[0074] To obtain a solid waste image region, the boundary contour of the waste target is extracted through an image segmentation algorithm, and then the target region is cropped to limit the calculation region range. The region is divided into an equilateral triangle grid structure with a fixed side length (such as 20 pixels). In actual operation, OpenCV is used to draw each edge, and each triangular grid is numbered in the order from left to right and from top to bottom. The coordinates of each grid node are calculated by pixel position. For example, if the first grid side length is 20 pixels, the first node is (0, 0), the second node is (20, 0), and the third node is (10, 17). The triangular grid is constructed in this way and the coordinate relationship between the intersection points is recorded. The coordinate values of all nodes are combined into an image grid structure coordinate set, where each coordinate item is represented in a triple form, that is, (number, x coordinate, y coordinate). For example, the third point is (3, 40, 34). This structure supports subsequent tension extraction based on geometric relationships.

[0075] The tension extraction submodule calculates three types of indexes corresponding to the edge length change rate, the angle offset and the diagonal line length difference of the intersection position in each grid in the image grid structure coordinate set, and summarizes them as unit tension description parameters to generate grid tension index data;

[0076] The calculation of three types of indexes corresponding to the edge length change rate, the angle offset and the diagonal line length difference of the intersection position in each grid in the image grid structure coordinate set requires frame-by-frame comparison of each triangular grid in adjacent frames in the image sequence, extraction of the lengths of the three sides of the triangle and calculation of the change proportion thereof relative to the initial frame. For example, if the initial edge length is 20 pixels and the current frame edge length is 22 pixels, the change rate is (22-20) / 20=0.1, i.e. 10%, the angle offset is calculated from the three-point coordinates, for example, if the initial angle is 60 degrees and the current frame angle is 66 degrees, the offset is 6 degrees, and the diagonal line length difference can be calculated by connecting the lengths of the non-adjacent vertices of the triangle, for example, if the initial diagonal line is 28 pixels and the current frame is 30 pixels, the difference is 2 pixels. The three types of indexes are recorded according to the grid number and summarized to form a tension description parameter set, and the three parameters of each grid form a tension vector, for example, the tension description of grid A is (10%, 6 degrees, 2 pixels), and the grid tension parameter set is generated after all grid tension parameters are summarized, which is used for subsequent structure deformation analysis.

[0077] The abnormal region identification submodule judges the fluctuation direction and amplitude change of each tension index in the continuous grid structure based on the grid tension index data, filters the region where the index continuously deviates and the boundary mutation rate is greater than the judgment standard, and marks it as a potential structure abnormal region;

[0078] The judgment of the fluctuation direction and amplitude change of each tension index in the continuous grid structure based on the grid tension index data requires first-order difference analysis of the tension values of the same numbered grid on the time axis, the sign of the difference value of the tension values of the continuous frames is calculated to determine the fluctuation direction, for example, the tension of frame 1 is 0.1, the tension of frame 2 is 0.12, the direction is positive, the tension of frame 3 is 0.11, and the direction changes to negative, and the continuous same direction greater than 3 frames is determined as a trend fluctuation region. The fluctuation amplitude is divided into intervals, the high fluctuation interval of the edge length change rate is set to be greater than 0.2, the medium is 0.1-0.2, and the low is less than or equal to 0.1, the angle offset medium is 5-10 degrees, the high is greater than 10 degrees, the diagonal line difference medium is 1-3 pixels, and the high is greater than 3 pixels. According to the condition that each parameter meets the medium or high interval, it is determined as a potential abnormality, and a boundary mutation rate threshold is further set, for example, if the tension difference value of a grid and an adjacent grid is greater than the set value (for example, the tension edge length change rate difference is greater than 0.15), it is considered as a boundary mutation, and if the cumulative mutation rate exceeds 40% of the total number of continuous grids, it is considered as a judgment standard, and the region is marked as a potential structure abnormal region for further tracking or warning.

[0079] Please refer to Figure 5, the topology repair module includes:

[0080] The boundary extraction submodule extracts the coordinate sequence of the grid boundary intersection positions within the potential structural anomaly area, identifies the connection loss and angle misalignment information between each intersection position, and generates a fracture boundary intersection coordinate set;

[0081] To extract the coordinate sequence of the grid boundary intersection positions within the potential structural anomaly region, we first need to obtain the grid numbers marked as anomalies in the previous step. Then, based on the grid structure coordinate set, we extract the boundary intersection coordinates of these grids. For example, if the abnormal grids are numbered 10, 11, and 12 and are located at adjacent positions in the image area, we read the coordinates of the triangle vertices corresponding to the numbers and list the boundary vertex positions in order to form a coordinate sequence. Next, we traverse the adjacent intersection point pairs and check one by one whether there is a connection loss in the image, that is, whether the corresponding edge is blocked, broken, or missing. This can be determined by calculating the grayscale continuity between adjacent points. If the grayscale difference of an edge connection exceeds a set threshold (such as greater than 40), the connection is considered broken. At the same time, angular misalignment is detected in the geometric structure by comparing the difference between the angle of adjacent connection vectors and the standard angle in the ideal grid (such as 60 degrees). If the difference is greater than 10 degrees, it is considered an angular misalignment. This information is combined to form a broken boundary intersection coordinate set. Each record contains the coordinates of two intersection points, the connection status, and the angular deviation value.

[0082] The connection generation submodule sets interpolation rules based on the fracture boundary intersection coordinate set and the connection angle deviation and edge length similarity. It then adds auxiliary intersection points at the missing connection locations, establishes auxiliary connection chains according to the shortest connection path, and generates boundary reconstruction connection paths.

[0083] Based on the fracture boundary intersection coordinate set, the connection angle deviation and side length similarity are first judged for each set of fracture coordinate pairs, and the angle difference between the actual connection vector and the ideal direction is calculated. For example, if the deviation between the actual connection direction and the adjacent triangle side direction is 12 degrees, it is within the acceptable range (if the upper limit of the deviation is set to 15 degrees). The side length similarity is compared by comparing the distance between the two intersection points with the average side length of the region. For example, if the average side length is 20 pixels and the current distance is 22 pixels, the relative deviation is (22-20) / 20=0.1, that is, 10%. If the side length similarity threshold is set If the value is within 20%, it is considered a match. After the interpolation connection conditions are met, an auxiliary intersection point is inserted between the two intersection points. The coordinates of the interpolation point are obtained according to the midpoint of the line segment or piecewise linear interpolation. For example, interpolation between coordinates (100, 120) and (140, 130) gives the midpoint (120, 125). All interpolation points are connected to form an auxiliary connection chain. To reduce the path length, the group with the shortest total path length is selected from multiple connectable paths as the final connection chain. This connection chain combines the coordinates of the original point and the auxiliary point to generate a boundary reconstruction connection path.

[0084] The structure output sub-module combines and updates the boundary reconstruction connection path with the original grid structure of the image through a triangulation algorithm, closes and repairs the abnormal area boundary, and obtains a repaired topological image.

[0085] The triangulation algorithm combines and updates the boundary reconstruction connection path with the original grid structure of the image. First, the original grid structure node coordinates (for example, 120 points) in the image are combined with the auxiliary intersection points (for example, 12 points) newly added by the connection generation sub-module to generate a point set containing points, and the coordinates of each point are expressed in pixel units, for example , , and the like. All points are organized in a numbered index manner to prepare for subsequent triangulation.

[0086] In the specific triangulation process, the Delaunay algorithm is used, and the core judgment condition is: if a triangle is constructed with any three points , , , then it is judged whether the circumcircle thereof contains other points, the circumcenter of the circumcircle and the denominator item used for judgment are calculated, and the formula is as follows:

[0087] ;

[0088] Among them, the meanings of various parameters are as follows: : the horizontal and vertical coordinates of point ; : the horizontal and vertical coordinates of point ; : the horizontal and vertical coordinates of point ; : the common denominator item of the circumcircle formula of the triangle.

[0089] When , the circumcenter coordinate is calculated as follows:

[0090] ;

[0091] ;

[0092] In these expressions, the expression such as represents the square distance of point to the origin, which can be used to describe the position relationship between the point and the circumcenter.

[0093] Next, to verify whether the triangle meets the Delaunay condition, the distance of any other point to the circumcenter is calculated, and if If the distance between any vertex of the triangle and the center of the circle is greater than the radius of the circumscribed circle, the triangle is legal.

[0094] Let the three points be: , , .

[0095] Calculate: ;

[0096] ;

[0097] ;

[0098] Therefore, the center of the circle is , and the radius is the distance between any point (such as ) and the center of the circle

[0099] .

[0100] After completing the generation of all legal triangular units, the system enters the update and closure repair process of the image grid structure. This process needs to first identify the boundary grid units adjacent to the abnormal areas in the original image grid, extract their numbers and coordinates, and perform boundary docking operations with the newly added grid units through triangulation. The specific operation process is as follows: First, arrange all the vertices of the newly added triangular units in order, and connect the head and tail of the vertex sequence of the original grid units. If any two vertices are not connected by existing edges in the image, automatically add a connection edge and set its attribute to "auxiliary edge". This auxiliary edge will be marked as a fictional connection to avoid being mistaken as a real structure edge in subsequent analysis. Then, according to the angle and overlap ratio between all new edges and old grid edges, judge whether the boundary has formed a closed loop. If there is an opening (i.e., the edge is not completely closed), insert a connecting segment at both ends of the opening to form a closed loop with the smallest area, complete the closure repair operation. In the process of constructing the closed structure, the continuity and direction consistency of the edges need to be maintained to ensure that each triangle can form a face grid data structure through its three vertices and adjacent units. Finally, write the structure information of these newly added triangles into the topology structure diagram, including the coordinates of the three points, the sequence of edge connections, the triangle number, and the index of the repaired area to which it belongs. Mark the unit structure of the abnormal area in the original image grid as "replaced" state, indicating that the area has been repaired. Then, the system outputs the updated image grid structure, in which the original damaged part of the abnormal area has been replaced by the newly constructed triangular unit, forming a coherent, complete, and closed topological image.

[0101] Please refer to Figure 6 , the sorting and classification module includes:

[0102] The path matching sub-module calls the repaired topological image and the main label mapping table, extracts the confidence score corresponding to the structural connection path and the main label path, performs path structure consistency and score threshold judgment, and generates a main label matching judgment result;

[0103] The repaired topological image and the main label mapping table are called, all connection paths in the repaired image are first extracted, the path is composed of the aforementioned triangular subdivision and the connected structure generated by the closure repair, each path contains a group of continuous triangular units and the coordinate sequence of the vertices formed thereby, then the main label path information is called from the main label mapping table, the corresponding main label path contains the topological number sequence of the starting to the ending node and the associated confidence score, the system searches one by one in the topological graph structure to find whether there is a path that is consistent with the main label path structure in terms of node order, edge connection direction, and topological structure type (such as linear, tree-shaped, or grid-shaped), if the structural path is completely matched or can be considered as approximately matched within the tolerance range, then the score threshold judgment is performed, the score threshold required for the main label matching is set to 0.80, if the confidence score of the main label path is higher than the value, then it is judged as a matching success, if it is lower, then it is not matched, finally the main label matching judgment result is output, the matching label number and the judgment flag bit are recorded, which are used for subsequent label confirmation processing.

[0104] The label confirmation sub-module filters the main labels with scores higher than the activation threshold based on the main label matching judgment result, confirms the garbage category corresponding to the target main label as the recognition output of the current image, and generates solid waste recognition label data;

[0105] Based on the main label matching judgment result, all label paths that are judged as matching success are traversed and filtered to see whether their scores are higher than the activation threshold, the threshold comes from the score standard set by the main label output sub-module, for example, assuming that the activation threshold is set to 0.85, only the labels with scores equal to or higher than this value are retained, such as label ID_05 with a score of 0.89 and ID_07 with a score of 0.91 meet the conditions, while ID_12 with a score of 0.82 is excluded, the label content corresponding to the filtered label path is indexed and matched with the classification binding relationship in the garbage recognition database, the garbage type represented by the main label is confirmed, for example, ID_07 corresponds to the "plastic bottle" category and ID_05 corresponds to the "glass bottle" category, the target main label output is the path label with the highest score and passing the activation threshold filtering, the system recognizes it as the main garbage type contained in the current image, and generates solid waste recognition label data, the data structure includes label number, category name, score value, and path node index sequence, etc. information for subsequent sorting.

[0106] The result generation sub-module calls the solid waste recognition label data, generates matching sorting instructions based on the recognition label as the classification basis, performs image-level classification judgment and action binding through conditional random field, and obtains the solid waste sorting result.

[0107] Call the solid waste identification label data, take the main label of the repaired grid area in each image as the initial classification candidate, and build a graph structure model ,in An abstract graph model representing the entire image structure, Represents the set of all image grid region nodes (such as ), Represents the set of spatial adjacent edges between these nodes, each node Represents a spatially continuous structural region in the image, where each edge Indicates that there is a direct adjacency relationship between two nodes. For each node in the graph , all have an observed eigenvector , the vector includes the main label score, local structure tension parameters and texture grayscale gradient, etc.; each node also corresponds to a label , indicating the type of garbage identified in the area (such as plastic bottles, metal cans, plastic bags, etc.).

[0108] In the conditional random field model, the observation feature set Represents each node in the graph structure The sum of the observation data of is a node The corresponding local image feature vector, including the main label score, regional structure tension parameter, local texture gradient value, etc., is a quantitative expression of each grid area in the image in spatial and semantic dimensions; and the label configuration set It represents the garbage category labels assigned to all nodes in the graph, such as "plastic bottles" and "metal cans", etc. and the corresponding correspondence; symbol Indicates the current node is a set of graph nodes members, It is the complete set of all regional nodes in the entire graph structure, including all grid areas with significant structures in the image; the symbol Indicates that there is a pair of nodes in the graph structure and There is a spatial adjacency relationship between them, which constitutes an edge in the graph. The edge set It is the sum of direct connections between all regions in the graph structure, which is used to capture the mutual influence between adjacent regions in spatial structure and label reasoning; therefore, the term in the energy function For all nodes The label and its observed features are penalized for consistency, and represents the joint penalty accumulation on the inconsistency of label configuration and feature relation for all adjacent node pairs in the graph, and the two parts together constitute the total energy , determines the rationality and possibility of label configuration relative to feature observation .

[0109] The conditional random field (CRF) model is used to make a global judgment on the labels of all regions, and the goal is to solve the conditional probability of region label configuration under the condition of all observed features , defined as follows:

[0110] ;

[0111] wherein: : the label configuration set of all regions in the whole graph; : the observed feature set of all region nodes; : the normalization factor, representing the exponentially weighted sum of all possible label combinations ; : the total energy of label configuration, representing its degree of irrationality, the smaller the better.

[0112] The total energy function is defined as follows:

[0113] ;

[0114] wherein: : the single point potential function, measuring the consistency degree of region label and its observed feature , the higher the score, the lower the cost, calculated as , wherein is the score value; : the pair potential function, measuring the coordination degree of adjacent region , label , , combining the structure difference and label consistency setting, the structure is similar and the label is consistent, which is 0.3, the structure is similar and the label is inconsistent, which is 1.5, the structure is different and the label is consistent, which is 2.0, and the structure is different and the label is inconsistent, which is 0.3; : represents the two node numbers with adjacent relationship; : the observed features of nodes , respectively; : the label types of nodes , respectively.​

[0115] Let the system extract the following region nodes and their observation data from the image: : score , tension , texture ; : score , tension , texture ; : score , tension , texture ; : score , tension , texture ;

[0116] The single-point potential function is calculated as: , , , .

[0117] Consider 4 groups of label combinations to , candidate labels are A = plastic bottle, B = metal can, C = plastic bag, and the adjacent edges are , , and the energy of each combination is calculated as follows:

[0118] Combination :

[0119] : structural similarity, label consistent → ;

[0120] : structural difference, label inconsistent → ;

[0121] ;

[0122] .

[0123] Combination :

[0124] : structural similarity, label inconsistent → ;

[0125] : structural difference, label inconsistent → ;

[0126] ;

[0127] .

[0128] combination :

[0129] : structural similarity, label consistent

[0130] : structural difference, label inconsistent

[0131] ;

[0132] .

[0133] combination :

[0134] : structural similarity, label consistent ;

[0135] : structural difference, label consistent ;

[0136] ;

[0137] .

[0138] The normalization factor is: ,

[0139] The conditional probability is: , , , .

[0140] The final system selects from the above four groups or (the highest conditional probability) as the final label configuration: region 1 and 2 are classified as "plastic bottles", region 3 as "metal cans", and region 4 as "plastic bags". The system maps this configuration with the hardware action instruction library, such as A1, B3, C2, etc., and outputs the image number, region index, label and action code for the garbage sorting equipment to perform the separation operation.

[0141] The above merely describes the preferred embodiments of the present application, and is not intended to limit the present application in other forms. Any skilled person in the art can modify or change the disclosed technical content into equivalent embodiments with equivalent changes, and apply them to other fields. However, any simple modification, equivalent change and modification made to the above embodiments according to the technical essence of the present application, without departing from the technical solution content of the present application, still falls within the protection scope of the present application.

Claims

1. The solid waste intelligent sorting system based on image recognition is characterized by: The system comprises: The confidence score generation module obtains the edge intensity gradient features and grayscale co-occurrence contrast features in the solid garbage image frame sequence and performs a uniform amplitude adjustment. It then compares them with similar features counted in the image samples. Based on the comparison results, it generates a confidence value score for the corresponding candidate label for the current image frame and constructs a scoring path sequence. The main label identification module analyzes the scoring direction of each candidate label in the scoring path sequence and identifies the main label path, and outputs a main label mapping table; The structural anomaly detection module divides the solid waste image into multiple grid structures, extracts the tension index of each grid structure, and marks the areas with abnormal tension index fluctuations as potential structural anomaly areas; The topology repair module performs local repair and reconstruction on the intersection of the grid boundaries in the potential structural abnormality area to obtain a repaired topology image; The sorting and classification module combines the repaired topological image with the main label mapping table to classify and identify the solid waste image to obtain a solid waste sorting result.

2. The solid waste intelligent sorting system based on image recognition according to claim 1 is characterized in that: The scoring path sequence includes a label scoring value sequence, a scoring direction change sequence, and a label scoring stability index; the main label mapping table specifically includes a main label recognition result, an activation threshold parameter, and a label priority order; the potential structural abnormality area includes a boundary continuity mutation area, a tension abnormality concentration area, and a grid deformation expansion area; the repaired topological image specifically refers to a closed boundary path, a connection completion area, and a structural reconstruction fragment; the solid waste sorting result includes a main label classification identifier, an image structure status, and a sorting control instruction.

3. The solid waste intelligent sorting system based on image recognition according to claim 1 is characterized in that: The confidence score generation module includes: The image feature normalization submodule obtains solid waste images and performs frame operations, collecting edge intensity gradient features and grayscale co-occurrence contrast features of each frame in the image frame sequence, and maps the two types of features to the unit interval numerical space through the normalization method to generate unified dimension features; Feature label mapping submodule: Project the unified dimension feature as a two-dimensional coordinate point into the feature similarity space, calculate the Euclidean distance with the candidate label feature coordinates counted in the image sample, and determine the corresponding candidate label confidence score according to the inverse of the distance value; The scoring path construction submodule calls the candidate tag confidence score, arranges the scoring results of the candidate tags in each frame in sequence according to the image frame order, and connects the scoring points of the same candidate tag in each frame to obtain a scoring path sequence.

4. The solid waste intelligent sorting system based on image recognition according to claim 3 is characterized in that: The main tag identification module includes: The score change extraction submodule calls the score path sequence, calculates the score direction difference of each candidate tag in consecutive image frames, extracts the score jump frequency, score amplitude and direction continuity parameters corresponding to the tag path, and generates the candidate tag score change feature; The candidate tag screening submodule determines the consistency and fluctuation range of each candidate tag path in the scoring direction based on the candidate tag score change characteristics, screens the path with the most stable scoring direction and the smallest amplitude fluctuation range, and generates a main tag candidate path identifier; The main label output submodule sets the confidence activation threshold according to the variance of the main label candidate path identification score path, determines the main label path under the current image frame sequence, and establishes a mapping relationship between the label path and the confidence activation threshold to generate a main label mapping table.

5. The solid waste intelligent sorting system based on image recognition according to claim 4 is characterized in that: The structural anomaly detection module includes: The grid division submodule obtains the solid waste image area, divides each area into an equilateral triangle grid structure, sequentially numbers the grids to locate the intersection position relationship, and generates an image grid structure coordinate set; The tension extraction submodule calculates three types of indicators, namely, the side length change rate, the angle offset, and the diagonal length difference corresponding to the intersection position of each grid in the image grid structure coordinate set, summarizes them into unit tension description parameters, and generates grid tension index data; The abnormal area identification submodule judges the fluctuation direction and amplitude change of each tension index in the continuous grid structure based on the grid tension index data, selects areas with continuous index deviation and boundary mutation rate greater than the judgment standard, and marks them as potential structural abnormal areas.

6. The solid waste intelligent sorting system based on image recognition according to claim 5 is characterized in that: The topology repair module includes: The boundary extraction submodule extracts the coordinate sequence of the grid boundary intersection position in the potential structural anomaly area, identifies the connection loss and angle misalignment information between each intersection position, and generates a fracture boundary intersection coordinate set; The connection generation submodule sets interpolation rules based on the fracture boundary intersection coordinate set according to the connection angle deviation and the side length similarity, adds auxiliary intersection points at the missing connection positions, establishes auxiliary connection chains according to the shortest connection path, and generates a boundary reconstruction connection path; The structure output submodule combines and updates the boundary reconstruction connection path with the original grid structure of the image through a triangulation algorithm, performs closed repair processing on the boundary of the abnormal area, and obtains a repaired topological image.

7. The solid waste intelligent sorting system based on image recognition according to claim 6 is characterized in that: The sorting and classification module includes: The path matching submodule calls the repaired topology image and the main label mapping table, extracts the confidence scores corresponding to the structural connection paths and the main label paths, performs path structure consistency and score threshold judgment, and generates a main label matching judgment result; The label confirmation submodule selects the main labels with scores higher than the activation threshold based on the main label matching judgment result, confirms the garbage category corresponding to the target main label as the recognition output of the current image, and generates solid garbage identification label data; The result generation submodule calls the solid waste identification label data, uses the identification label as the classification basis to generate matching sorting instructions, performs image-level classification judgment and action binding through conditional random fields, and obtains the solid waste sorting results.

Images