Automatic solid waste sorting method and device applying machine vision

Abstract

The invention relates to the field of image recognition, in particular to an automatic solid waste sorting method and device applying machine vision. The method comprises the following steps that monitoring images of to-be-sorted areas in multiple directions are obtained; performing spatial geometric view angle deviation correction on the monitoring images of the to-be-sorted areas in the multiple directions, performing spatial panoramic image optimization, and constructing a panoramic image of the sorting area; performing frame-by-frame solid waste visual positioning scanning on the panoramic image of the sorting area, and performing three-dimensional morphological structure analysis so as to generate a three-dimensional morphological structure representation of each solid waste; performing solid type classification decision according to the three-dimensional morphological structure representation of each solid waste, and performing bounding box coding mapping to obtain a coding mapping bounding box of each solid waste; and performing equipment action sequence simulation on the coding mapping bounding box of each solid waste, and constructing a fine tuning action sequence. Efficient solid waste recognition is achieved, and the sorting efficiency is improved.

Description

Technical Field

[0001] The present invention relates to the field of image recognition, and in particular to a method and device for automatically sorting solid waste using machine vision. Background Art

[0002] With the increasing severity of global environmental problems, the effective management and recycling of solid waste has become an important issue for the sustainable development of societies in all countries. Especially in the context of accelerated industrialization and urbanization, the amount of solid waste generated is increasing. How to efficiently and accurately classify and treat solid waste has become an important means to achieve resource recovery and reduce environmental pollution. Traditional solid waste sorting methods usually rely on manual operations. Although preliminary sorting work can be completed in some cases, there are problems such as high labor intensity, low sorting efficiency, and easy errors. Especially in large-scale production or processing scenarios, manual sorting cannot meet the needs of efficient, accurate and continuous operations.

[0003] With the continuous development of machine vision technology, intelligent and automated solid waste sorting systems have gradually become a hot topic in research and application. Machine vision technology can simulate the human eye's ability to recognize objects. Through image acquisition, processing and analysis, it can identify the type, shape, color and other characteristic information of solid waste, thereby realizing automatic classification of waste. Compared with traditional manual sorting methods, the solid waste sorting system based on machine vision has the advantages of high efficiency, accuracy and automation. It can maintain a high level of work efficiency in a high-speed and continuous production environment, reduce labor costs and reduce error rates. However, solid waste is of various types and shapes, and the surface of the waste is often contaminated, damaged or obstructed. This makes waste sorting based on machine vision face many challenges. There is often inaccurate object recognition, resulting in low solid waste sorting efficiency.

[0004] Therefore, studying an efficient, intelligent and accurate automatic sorting method of solid waste based on machine vision has become an important technical need in the current fields of environmental protection and resource recycling. Summary of the invention

[0005] In order to solve the above technical problems, the present invention proposes a method and device for automatic sorting of solid waste using machine vision to solve at least one of the above technical problems.

[0006] To achieve the above object, the present invention provides a method for automatic sorting of solid waste using machine vision, comprising the following steps: Step S1: acquiring monitoring images of the area to be sorted in multiple directions; performing spatial geometric perspective deviation correction on the monitoring images of the area to be sorted in multiple directions, and performing spatial panoramic image optimization to construct a panoramic image of the sorting area; Step S2: performing visual positioning scanning of solid waste frame by frame on the panoramic image of the sorting area, and performing three-dimensional morphological structure analysis, thereby generating a three-dimensional morphological structure representation of each solid waste; Step S3: making a solid type classification decision based on the three-dimensional morphological structure representation of each solid waste, and performing bounding box coding mapping to obtain a coding mapping bounding box of each solid waste; Step S4: simulate the equipment action sequence for each coded mapping boundary box of solid waste, and then fine-tune the action parameters to construct a fine-tuned action sequence; Step S5: Performing instant automatic sorting based on the fine-tuning action sequence, and collecting sorting images at multiple time points; performing temporal sorting efficiency trend evolution on the sorting images at multiple time points, and generating sorting efficiency trend evolution features; Step S6: Dynamically optimize the sorting path planning and adaptively enhance the transfer learning of the sorting efficiency trend evolution characteristics, so as to build an adaptive intelligent sorting model to perform automatic sorting of solid waste.

[0007] The present invention improves image quality by correcting spatial geometric viewing angle deviation and optimizing panoramic images, which is convenient for subsequent processing. Constructing a panoramic image of the sorting area helps provide an overall perspective, laying the foundation for subsequent solid waste identification and sorting. Through frame-by-frame solid waste visual positioning scanning and three-dimensional morphological structure analysis, the morphological characteristics of each solid waste are accurately obtained, and the generation of three-dimensional morphological structure representation is helpful for subsequent classification and sorting decisions, improving the accuracy and efficiency of automatic sorting. Solid type classification is carried out according to the three-dimensional morphological structure representation to accurately identify different types of solid waste. Bounding box coding mapping provides an accurate mark for the location of each solid waste, providing a basis for subsequent action simulation and sorting. Equipment action sequence simulation and fine-tuning optimize the sorting process The actions and parameters in the sorting process can improve the accuracy of automatic sorting. Building a fine-tuning action sequence can help optimize the operating procedures of robots or equipment and improve sorting efficiency and accuracy. Analyzing the temporal evolution characteristics of sorting images at multiple time points can help evaluate the changing trend of sorting efficiency, identify problems and improvement directions in a timely manner, and generate sorting efficiency trend evolution characteristics to help optimize sorting strategies and improve overall operational efficiency. Dynamic sorting path planning optimization and adaptive reinforcement transfer learning can improve the intelligence level and adaptability of automatic sorting systems. Building an adaptive intelligent sorting model can help achieve efficient automatic sorting of solid waste, improve production efficiency and reduce labor costs.

[0008] Preferably, step S1 comprises the following steps: Step S11: Acquire monitoring images of the area to be sorted in multiple directions based on a high-resolution panoramic camera; Step S12: Optimizing the image contrast of the monitoring images of the to-be-sorted areas in multiple directions to construct a multi-directional contrast-balanced monitoring image; Step S13: performing image feature point recognition on the multi-directional contrast equalization monitoring image, and marking a plurality of image feature points; Step S14: performing multi-directional precision matching on multiple image feature points, thereby obtaining multi-angle matching data for each feature point; Step S15: performing spatial geometric perspective deviation correction based on the multi-angle matching data of each feature point, thereby obtaining a spatial geometric perspective deviation corrected image; Step S16: performing spatial panoramic image optimization on the spatial angle deviation corrected image to construct a panoramic image of the sorting area.

[0009] The present invention uses a high-resolution panoramic camera to obtain images to improve the clarity and detail capture ability of the image, which is helpful for accurately identifying solid waste. The monitoring images in multiple directions provide comprehensive coverage, which is helpful for a comprehensive understanding of the situation in the sorting area. The image contrast is optimized to enhance the details and contrast in the image, which is helpful for improving the recognition accuracy of solid waste. The multi-directional contrast-balanced monitoring images are helpful for unifying the visual effects of images in different directions and providing consistent input for subsequent processing. The identification and marking of image feature points are helpful for determining key positions and features in the image and providing basic data for subsequent matching and correction. The marking of image feature points helps the algorithm to better understand the image content and improve the accuracy and efficiency of subsequent processing. Precisely matching feature points and obtaining matching data of feature points at different angles is helpful for accurately correcting spatial geometric perspectives. Multi-angle matching data provides rich information and more accurately restores the position and shape of objects in space. Through spatial geometric perspective deviation correction, the perspective deformation and distortion in the image are eliminated, and the accuracy and visualization of the image are improved. The corrected image is helpful for subsequent processing and analysis, providing a more accurate data basis for automatic sorting. Constructing a panoramic image of the sorting area provides an overall perspective and scene, which helps to fully understand the situation in the sorting area and provide basic data for subsequent automatic sorting of solid waste. The optimization of spatial panoramic images improves image quality and provides better visual support for subsequent analysis and operations.

[0010] Preferably, the specific steps of step S12 are: Performing image region division on monitoring images of the to-be-sorted areas in multiple directions to obtain multiple image regions; Calculating original grayscale pixel values ​​of multiple image regions to generate original grayscale pixel values ​​of each region; Perform global average calculation on the original grayscale pixel value of each area to obtain the average grayscale pixel value of the image; Perform pixel linear stretching enhancement processing on the average grayscale pixel value of the image to generate an enhanced grayscale pixel value; Stretching pixels in a plurality of image regions according to enhanced grayscale pixel values ​​to generate a brightness enhanced monitoring image; Perform grayscale pixel frequency distribution analysis based on the original grayscale pixel value of each region to generate image grayscale pixel frequency distribution data; Performing cumulative distribution function calculation on the image grayscale pixel frequency distribution data to extract the cumulative distribution function of the image; Perform pixel contrast optimization processing on the cumulative distribution function of the image to obtain a contrast optimized cumulative distribution function; The contrast optimization cumulative distribution function is used to perform contrast equalization on the brightness enhanced monitoring image, thereby constructing a multi-directional contrast balanced monitoring image.

[0011] The present invention helps to decompose the image into smaller processable units by dividing the image area, thereby improving the accuracy and efficiency of subsequent processing. The division of each image area can make the processing more targeted and help to adopt personalized processing methods for different parts. Calculating the original grayscale pixel value helps to understand the grayscale distribution of each area and provide basic data for subsequent processing. The calculation of these values ​​helps to evaluate the brightness distribution of the image and provide a basis for subsequent enhancement and optimization. Calculating the average grayscale pixel value of the image helps to understand the overall brightness level and provide a benchmark for subsequent enhancement processing. This step helps to adjust the brightness balance of the image and improve the visual quality and analyzability of the image. The pixel linear stretching enhancement processing expands the grayscale range of the image and enhances the contrast and visual effect of the image. This processing method It helps to highlight the details and features in the image and improve the visualization of the image. Analyzing the grayscale pixel frequency distribution helps to understand the distribution of each grayscale level in the image and provide guidance for subsequent processing. By analyzing the pixel frequency, the contrast and brightness of the image can be better adjusted to improve the quality and analyzability of the image. Calculating the cumulative distribution function helps to describe the overall contrast of the image and provide an important basis for subsequent contrast optimization. The calculation of the cumulative distribution function helps to understand the overall grayscale distribution of the image and provide an effective adjustment direction for subsequent processing. Contrast optimization processing improves the contrast of the image, enhances the details and features in the image, and improves the visualization of the image. The optimized contrast helps to highlight the important information in the image and provides better visual support for subsequent processing and analysis.

[0012] Preferably, the specific steps of step S2 are: Step S21: performing a solid waste visual positioning scan frame by frame on the panoramic image of the sorting area, and marking each solid waste positioning data; Step S22: identifying the boundary contour of the solid waste according to each solid waste location data, and extracting the boundary contour line of each solid waste; Step S23: fine-grained segmentation of the panoramic image of the sorting area is performed based on the boundary contour line of each solid waste to generate an image boundary frame of each solid waste; Step S24: performing object geometry analysis on each solid waste image boundary frame to generate solid waste geometry features; Step S25: performing object texture detail recognition on each solid waste image boundary frame to obtain solid waste texture detail data; Step S26: Performing a three-dimensional morphological structure analysis on the solid waste texture detail data and the solid waste geometric morphological features, thereby generating a three-dimensional morphological structure representation of each solid waste.

[0013] The present invention helps to accurately locate each object by scanning and marking the position of solid waste frame by frame, providing accurate data for subsequent processing, and can effectively identify the location information of solid waste, providing key data support for subsequent steps. Identifying the boundary contour of solid waste helps to accurately extract the shape and edge information of the object, preparing for subsequent analysis. The identification of the boundary contour can help to accurately locate the shape and size of solid waste, providing an important basis for subsequent processing. Generating the boundary box of solid waste through fine-grained segmentation helps to accurately frame the position range of each object, providing data support for subsequent analysis. The image boundary box of each solid waste provides the accurate position and boundary information of the object in the image, providing a basis for subsequent processing. ,Object geometric morphology analysis can extract the shape characteristics of solid waste, such as aspect ratio, area, etc., which provides an important basis for subsequent classification and sorting. ,Through geometric morphological feature analysis, a deeper understanding of the morphological characteristics of solid waste is ,conducive to further processing and classification. ,Identifying the texture details of solid waste helps to distinguish different types of objects and improve ,the accuracy of classification. ,The acquisition of texture detail data helps to distinguish objects with similar shapes but different ,textures, and provides more information for automatic sorting. ,Through three-dimensional morphological structure analysis, the geometric morphology and texture characteristics of ,solid waste are comprehensively considered to provide a more comprehensive object description, ,which helps to have a deeper understanding of the characteristics of solid waste and provide a more accurate ,recognition and classification basis for the automatic sorting system.

[0014] Preferably, step S3 specifically comprises the following steps: Step S31: making solid type classification decisions based on the three-dimensional morphological structure representation of each solid waste, and obtaining type data of each solid waste; Step S32: performing sorting sequence analysis on the type data of each solid waste to generate sorting sequence data of each solid waste; Step S33: performing sorting sequence coding on the sorting sequence data of each solid waste to obtain a solid waste sequence code; Step S34: performing bounding box coding mapping on the image bounding box of each solid waste based on the solid waste sequential coding to obtain a coded mapping bounding box of each solid waste.

[0015] The present invention makes classification decisions through the three-dimensional morphological structure representation of solid waste, which helps to accurately distinguish different types of solid waste, realize automatic classification of solid waste, and provide correct type data for subsequent sorting and processing. Analyzing the sorting order of solid waste helps to determine the priority and order of each object in the processing process. By analyzing the sorting order, the sorting efficiency is improved to ensure that the solid waste is correctly processed and classified. Coding the sorting order helps the system to accurately record the processing order of each solid waste, which is convenient for subsequent tracking and management. The sorting order coding helps to improve the traceability and management efficiency of the automatic sorting system. The bounding box coding mapping based on the solid waste order coding helps to associate each solid waste with its processing order, improve the accuracy and efficiency of solid waste processing, and ensure that each object is sorted and processed in the correct order.

[0016] Preferably, the specific steps of step S4 are: Step S41: performing equipment action sequence simulation on the code mapping boundary box of each solid waste to obtain action sequence simulation data of the automatic sorting equipment; Step S42: Dynamically sorting movement tracking is performed on the action sequence simulation data of the automatic sorting device to extract the dynamic sorting movement trajectory; Step S43: Analyze the solid waste position change of the dynamic sorting movement trajectory to generate solid waste position change data; Step S44: Calculating the equipment sorting accuracy for the solid waste position change data to obtain the equipment sorting accuracy; Step S45: identifying the action sequence deviation according to the device sorting accuracy to obtain the action sequence deviation value; Step S46: performing deviation compensation calculation on the action sequence deviation value to generate action sequence deviation compensation data; Step S47: fine-tuning the action parameters of the action sequence simulation data of the automatic sorting device based on the action sequence deviation compensation data to construct a fine-tuning action sequence.

[0017] The present invention generates action sequence simulation data of automatic sorting equipment by simulating the equipment action sequence of the coding mapping boundary box of solid waste, which is helpful to determine the operation order and mode of the equipment in the processing flow; dynamically sorting and moving tracking is performed on the action sequence simulation data of the equipment to extract the movement trajectory of the solid waste in the processing process, which is helpful to track and monitor the position change of the solid waste; the dynamic sorting movement trajectory is analyzed to generate solid waste position change data, which helps to understand the position change of the solid waste in the processing process and provide data support for subsequent accuracy calculation; the accuracy of equipment sorting is obtained by calculating the solid waste position change data, which is helpful to evaluate the performance and accuracy of the automatic sorting equipment; action sequence deviation identification is performed according to the equipment sorting accuracy to determine the deviation in the equipment operation, which provides a basis for further optimizing the equipment action; the action sequence deviation value is calculated and compensated, which is helpful to reduce the error in the equipment action and improve the accuracy and efficiency of sorting; the action sequence simulation data of the equipment is fine-tuned based on the deviation compensation data to construct a more accurate action sequence and improve the performance and accuracy of the automatic sorting equipment.

[0018] Preferably, the specific steps of step S5 are: Step S51: performing instant automatic sorting based on the fine-tuning action sequence, and collecting sorting images at multiple time points; Step S52: performing sorting status analysis on the multi-time point sorting images to extract the sorting status parameters of the automation equipment; Step S53: performing a quantitative evaluation of the sorting efficiency of the automated equipment sorting state parameters to generate a quantitative evaluation value of the sorting efficiency; Step S54: performing time-series sorting efficiency trend evolution on the sorting efficiency quantitative evaluation value to generate sorting efficiency trend evolution characteristics.

[0019] The present invention realizes real-time monitoring and feedback by performing instant automatic sorting based on fine-tuning action sequences and collecting sorting images at multiple time points. This helps to verify the effect of fine-tuning the action sequence and provides real-time sorting image data for subsequent data analysis. The multi-time point sorting images are analyzed to extract the sorting state parameters of the automated equipment, such as sorting speed, accuracy, etc. This helps to understand the performance of the equipment in actual operation and provide data support for subsequent evaluation. By quantitatively evaluating the sorting state parameters of the automated equipment, the sorting efficiency of the equipment can be accurately evaluated. This helps to identify existing problems and put forward improvement suggestions to improve the sorting efficiency. The quantitative evaluation values ​​of the sorting efficiency are analyzed in time series to generate trend evolution characteristics of the sorting efficiency. This helps to understand the changing trend of the equipment sorting efficiency and provides a reference for long-term performance monitoring and optimization.

[0020] Preferably, the specific steps of step S6 are: Step S61: performing dynamic sorting path planning optimization on the sorting efficiency trend evolution characteristics to obtain a dynamic sorting optimization path; Step S62: optimizing the whole sorting process of the fine-tuning action sequence based on the dynamic sorting optimization path to generate a global optimization action sequence; Step S63: Perform adaptive reinforcement transfer learning on the global optimization action sequence to build an adaptive intelligent sorting model to perform automatic sorting operations for solid waste.

[0021] The present invention helps to adjust the sorting path according to the real-time situation and improve efficiency by analyzing the evolution characteristics of the sorting efficiency trend and performing dynamic path planning optimization. A more efficient solid waste sorting process is achieved through dynamic path planning optimization, which reduces time and resource waste. Full-process optimization of fine-tuning action sequences based on dynamic sorting optimization paths helps to improve the efficiency and accuracy of the entire sorting process. The optimized action sequence can ensure that solid waste is accurately and quickly classified and processed, and improve the overall performance of the automatic sorting system. An adaptive intelligent sorting model is constructed through adaptive reinforcement transfer learning to continuously optimize system performance and adapt to changing environments and needs. Transfer learning helps the model learn and improve sorting strategies in different environments, thereby improving the accuracy and efficiency of automatic sorting of solid waste.

[0022] In this specification, a solid waste automatic sorting device using machine vision is provided, which is used to perform the solid waste automatic sorting method using machine vision as described above, including: The geometric perspective correction module is used to obtain monitoring images of the area to be sorted in multiple directions; perform spatial geometric perspective deviation correction on the monitoring images of the area to be sorted in multiple directions, and perform spatial panoramic image optimization to construct a panoramic image of the sorting area; The three-dimensional morphological structure module is used to perform visual positioning scanning of solid waste frame by frame in the panoramic image of the sorting area and perform three-dimensional morphological structure analysis to generate a three-dimensional morphological structure representation of each solid waste; A coding mapping module is used to make solid type classification decisions based on the three-dimensional morphological structure representation of each solid waste, and perform bounding box coding mapping to obtain a coding mapping bounding box for each solid waste; The action parameter fine-tuning module is used to simulate the equipment action sequence for each solid waste coding mapping boundary box, and then fine-tune the action parameters to construct a fine-tuning action sequence; The sorting efficiency trend module is used to perform instant automatic sorting based on fine-tuning action sequences and collect sorting images at multiple time points; it performs temporal sorting efficiency trend evolution on the sorting images at multiple time points and generates sorting efficiency trend evolution features; The sorting path optimization module is used to perform dynamic sorting path planning optimization and adaptive reinforcement transfer learning on the sorting efficiency trend evolution characteristics, so as to build an adaptive intelligent sorting model to perform automatic solid waste sorting operations.

[0023] The present invention helps to eliminate perspective distortion in images and improve sorting accuracy by acquiring monitoring images of the area to be sorted in multiple orientations and performing geometric perspective deviation correction, constructs a panoramic image of the sorting area to provide a more comprehensive field of view for subsequent solid waste identification and sorting, and increases the global perception capability of the system. Through frame-by-frame visual positioning scanning and three-dimensional morphological structure analysis of solid waste, the morphological characteristics of each solid waste are accurately acquired, providing important data support for subsequent classification and sorting. The generation of three-dimensional morphological structure representation helps the system to accurately identify and classify solid waste, improves sorting accuracy and efficiency, and makes classification decisions and bounding box coding mapping based on the three-dimensional morphological structure representation of solid waste to achieve accurate solid type classification and position marking. The coding mapping bounding box provides accurate Accurate location information provides an important basis for subsequent processing and sorting. Through equipment action sequence simulation and action parameter fine-tuning, the solid waste processing action is optimized to improve the accuracy and efficiency of sorting. Building a fine-tuning action sequence helps to ensure that each solid waste is properly processed and reduce processing errors and losses. Analysis of the evolution characteristics of the time-series sorting efficiency trend helps the system understand the changing rules of sorting efficiency, optimize the sorting strategy and improve the overall efficiency. Through instant automatic sorting and analysis of efficiency trends, the system adjusts the strategy in real time to improve sorting efficiency and accuracy. Dynamic sorting path planning optimization and adaptive reinforcement transfer learning help to build an intelligent sorting model and improve system adaptability and performance. The adaptive intelligent sorting model adjusts the sorting path according to the real-time situation to improve the efficiency and accuracy of automatic sorting of solid waste. BRIEF DESCRIPTION OF THE DRAWINGS

[0024] Figure 1 A schematic flow chart of the steps of a method for automatic sorting of solid waste using machine vision according to the present invention; Figure 2 Detailed implementation flow chart of step S1; Figure 3 Detailed implementation flow chart of step S2; Figure 4 Detailed implementation flow chart of step S3. DETAILED DESCRIPTION

[0025] It should be understood that the specific embodiments described herein are only used to explain the present invention, and are not used to limit the present invention.

[0026] The present application example provides a method and device for automatic sorting of solid waste using machine vision. The execution subject of the method and device for automatic sorting of solid waste using machine vision includes but is not limited to: mechanical equipment, data processing platform, cloud server node, network upload device, etc. equipped with the system can be regarded as the general computing node of the present application, and the data processing platform includes but is not limited to: at least one of an audio image management system, an information management system, and a cloud data management system.

[0027] See also Figures 1 to 4 The present invention provides a method for automatically sorting solid waste using machine vision, and the method for automatically sorting solid waste using machine vision comprises the following steps: Step S1: acquiring monitoring images of the area to be sorted in multiple directions; performing spatial geometric perspective deviation correction on the monitoring images of the area to be sorted in multiple directions, and performing spatial panoramic image optimization to construct a panoramic image of the sorting area; Step S2: performing visual positioning scanning of solid waste frame by frame on the panoramic image of the sorting area, and performing three-dimensional morphological structure analysis, thereby generating a three-dimensional morphological structure representation of each solid waste; Step S3: making a solid type classification decision based on the three-dimensional morphological structure representation of each solid waste, and performing bounding box coding mapping to obtain a coding mapping bounding box of each solid waste; Step S4: simulate the equipment action sequence for each coded mapping boundary box of solid waste, and then fine-tune the action parameters to construct a fine-tuned action sequence; Step S5: Performing instant automatic sorting based on the fine-tuning action sequence, and collecting sorting images at multiple time points; performing temporal sorting efficiency trend evolution on the sorting images at multiple time points, and generating sorting efficiency trend evolution features; Step S6: Dynamically optimize the sorting path planning and adaptively enhance the transfer learning of the sorting efficiency trend evolution characteristics, so as to build an adaptive intelligent sorting model to perform automatic sorting of solid waste.

[0028] The present invention improves image quality by correcting spatial geometric viewing angle deviation and optimizing panoramic images, which is convenient for subsequent processing. Constructing a panoramic image of the sorting area helps provide an overall perspective, laying the foundation for subsequent solid waste identification and sorting. Through frame-by-frame solid waste visual positioning scanning and three-dimensional morphological structure analysis, the morphological characteristics of each solid waste are accurately obtained, and the generation of three-dimensional morphological structure representation is helpful for subsequent classification and sorting decisions, improving the accuracy and efficiency of automatic sorting. Solid type classification is carried out according to the three-dimensional morphological structure representation to accurately identify different types of solid waste. Bounding box coding mapping provides an accurate mark for the location of each solid waste, providing a basis for subsequent action simulation and sorting. Equipment action sequence simulation and fine-tuning optimize the sorting process The actions and parameters in the sorting process can improve the accuracy of automatic sorting. Building a fine-tuning action sequence can help optimize the operating procedures of robots or equipment and improve sorting efficiency and accuracy. Analyzing the temporal evolution characteristics of sorting images at multiple time points can help evaluate the changing trend of sorting efficiency, identify problems and improvement directions in a timely manner, and generate sorting efficiency trend evolution characteristics to help optimize sorting strategies and improve overall operational efficiency. Dynamic sorting path planning optimization and adaptive reinforcement transfer learning can improve the intelligence level and adaptability of automatic sorting systems. Building an adaptive intelligent sorting model can help achieve efficient automatic sorting of solid waste, improve production efficiency and reduce labor costs.

[0029] In the embodiment of the present invention, refer to Figure 1 , is a schematic flow chart of the steps of a method for automatic sorting of solid waste using machine vision of the present invention. In this example, the steps of the method for automatic sorting of solid waste using machine vision include: Step S1: acquiring monitoring images of the area to be sorted in multiple directions; performing spatial geometric perspective deviation correction on the monitoring images of the area to be sorted in multiple directions, and performing spatial panoramic image optimization to construct a panoramic image of the sorting area; In this embodiment, multiple high-resolution cameras are configured to ensure that all directions (such as the front, side and top) of the area to be sorted are covered, the stability of the camera is ensured, and the image quality is prevented from being degraded due to vibration. The camera is started to collect images within a predetermined time interval to ensure that clear images can be obtained under different lighting and environmental conditions. An automated script is used to control the operation of each camera to ensure the synchronization of image collection time. The collected images are stored locally or in cloud storage to ensure that the image files have good naming rules for subsequent processing (for example, using timestamps and camera numbers). A calibration board (such as a chessboard) is used to calibrate the camera to obtain the intrinsic parameters (focal length, optical center position, distortion coefficient) and extrinsic parameters (position and direction) of each camera. A camera calibration algorithm (such as Zhang's calibration method) is applied to calculate the geometric parameters of each camera. The image distortion correction function in a computer vision library such as OpenCV is used to perform geometric correction on the image obtained by each camera to ensure that the shape and size of the object in the image are consistent with the actual object. Eliminate the deviation caused by lens distortion, identify the same feature points in each image through feature matching algorithms (such as SIFT and ORB), calculate the transformation matrix between perspectives, align images from different perspectives to ensure that they are aligned in space to form a common reference frame, use image stitching algorithms (such as RANSAC algorithm combined with image registration) to stitch the corrected images to generate a preliminary panoramic image, deal with overlapping areas that appear during the stitching process, use weighted averaging and other methods to smooth the stitching gaps and improve image quality, post-process the generated panoramic image, apply image enhancement techniques (such as contrast adjustment and brightness balance) to improve image visibility, use denoising algorithms (such as Gaussian filtering and median filtering) to improve image clarity, ensure that the details of the sorting area are clearly visible, save the optimized panoramic image to the database, ensure that it has a good format (such as JPEG, PNG) and resolution for subsequent analysis and use, and add relevant metadata (such as acquisition time and camera position) to the panoramic image for subsequent retrieval and management.

[0030] Step S2: performing visual positioning scanning of solid waste frame by frame on the panoramic image of the sorting area, and performing three-dimensional morphological structure analysis, thereby generating a three-dimensional morphological structure representation of each solid waste; In this embodiment, the panoramic image is preprocessed, including denoising (using Gaussian filtering or median filtering) and contrast enhancement (for example, using histogram equalization) to improve the visibility of solid waste, converting the image into a grayscale image to reduce computational complexity and facilitate subsequent feature extraction, using an edge detection algorithm (such as Canny edge detection) to extract edge information in the image to identify the outline of the solid waste, applying morphological operations (such as dilation and erosion) to further clarify the edges and eliminate small noise points, using an object detection algorithm (such as YOLO, SSD) to detect solid waste in the image, identifying the position and bounding box of each object, post-processing the detection results, filtering out low-confidence detection results, ensuring that only accurate object positioning information is retained, and recording the position information of each solid waste (such as bounding box coordinates, object ID, etc.) in a data structure for subsequent analysis and processing. If the camera used supports depth perception (such as a stereo camera or a structured light camera), the position information of each solid waste is obtained. The depth information of the object is used to construct a depth map. If depth perception is not supported, the depth is estimated by combining monocular vision with the known size of the object (such as through the perspective principle and camera internal parameter inversion). A 3D reconstruction algorithm (such as voxel method, point cloud method or multi-view stereo vision) is used to combine the two-dimensional boundary contour and depth information of the solid waste to generate a three-dimensional model of each object. The reconstructed three-dimensional model is smoothed to eliminate the noise and irregularities generated in the reconstruction process. The geometric features of each three-dimensional model (such as volume, surface area, shape complexity, etc.) are calculated, and these features are recorded as the morphological structure representation of the object. The morphological analysis method (such as principal component analysis) is used to further analyze the morphological features of the object to extract the main structural features. The three-dimensional model of each solid waste and its morphological features are saved in the database to ensure that the data structure is clear (such as including object ID, shape features, size information, etc.), and a report or visualization interface is generated to display the three-dimensional morphological structure of each solid waste, which is convenient for subsequent analysis and decision-making.

[0031] Step S3: making a solid type classification decision based on the three-dimensional morphological structure representation of each solid waste, and performing bounding box coding mapping to obtain a coding mapping bounding box of each solid waste; In this embodiment, key morphological features such as volume, surface area, aspect ratio, shape complexity, etc. are extracted from the three-dimensional model of each solid waste, and features meaningful to classification decisions are selected, including geometric features, color histograms, and texture features. The extracted feature data are standardized to eliminate the influence of different feature dimensions to ensure the effectiveness of the classification algorithm. Dimensionality reduction is performed using methods such as principal component analysis (PCA) to reduce the complexity of the feature space while retaining the main information. Suitable classification algorithms such as support vector machines (SVMs), random forests, and deep learning (such as convolutional neural networks) are selected. Based on the size of the data set and the complexity of the features, if there is a labeled training data set, a classification model is constructed using supervised learning; if not, consider using unsupervised learning methods (such as K-Means clustering) for preliminary classification. Use the extracted feature data set to train the selected classification model. For supervised learning, the data set needs to be divided into a training set and a test set to ensure the accuracy and generalization ability of the training. The model parameters are optimized through methods such as cross-validation to ensure the best performance of the classification model. The three-dimensional morphological structural features of each solid waste are input into the trained classification model to classify the solid type. The classification results of each solid waste are recorded for subsequent processing. A coding system is designed to ensure that each solid type has a unique code. For example, alphanumeric combinations are used to represent different waste types (such as "P1" for plastic, "M1" for metal, etc.). Based on the previous visual positioning results, a corresponding bounding box is generated for each solid waste. The coordinates of the bounding box should be based on the position and size of the object in the panoramic image. The calculated bounding box coordinates are used to generate the corresponding The bounding box graphics are drawn to ensure that they accurately surround the target object. The classification results of each solid waste are combined with its bounding box information to form a coded mapping bounding box. The unique code, bounding box coordinates and object type of each solid waste are recorded. The coded mapping results are stored in the database, and a suitable data structure (such as object ID, code, bounding box coordinates) is designed for subsequent retrieval and use. The coded mapping bounding box results of each solid waste are output to a standard format (such as JSON, XML or CSV) to facilitate subsequent data processing and analysis. A visual report of the sorting results is generated to display the classification of each solid waste and its corresponding bounding box. The performance of the classification model is verified, and the classification accuracy, recall rate and F1 score are calculated to evaluate the reliability of the model. According to the verification results, the model is adjusted and optimized to ensure the accuracy of the classification effect.

[0032] Step S4: simulate the equipment action sequence for each coded mapping boundary box of solid waste, and then fine-tune the action parameters to construct a fine-tuned action sequence; In this embodiment, ensure that the bounding box information of each solid waste code mapping has been correctly recorded, including the coordinates of the bounding box, object type and unique code. According to the function and operation manual of the equipment, design a preliminary action sequence, including key actions such as grabbing, lifting, moving and placing. Each action should have clear parameters, such as starting position, target position, action speed and grabbing force, etc. Use simulation software (such as Gazebo, MATLAB Build a virtual sorting environment using Simulink and Unity to ensure that it can simulate the operation of real equipment, import the coded mapping bounding box data to ensure that the model can accurately reflect the actual position and status of the object, execute the designed action sequence in the simulation environment, observe the impact of each action on solid waste, and record key data in the simulation process, including action execution time, success rate, and any problems that arise (such as grasping failure or misoperation). According to the results of the simulation run, set fine-tuning goals, for example, improve the grasping success rate, reduce the action execution time, or improve the sorting efficiency. Identify the types of parameters that need to be adjusted, such as grasping angle, grasping force, moving speed, etc. Apply parameter optimization algorithms (such as grid search, random search, or genetic algorithm) to systematically adjust the action parameters, select a series of parameter combinations, and gradually test their effects in the simulation environment to find the best parameter configuration. Run the fine-tuned action sequence, observe and Record the execution effect of each action, compare the data before and after fine-tuning, analyze whether the effect of fine-tuning has achieved the expected goal, further adjust the parameters according to the test results, optimize the action sequence, form a cyclic iterative optimization process, record the results of each fine-tuning, ensure that the performance of different parameter configurations can be tracked, integrate the action sequence that has undergone multiple fine-tuning into a final version, ensure that the execution order and parameter settings of each action are reasonable, ensure that the fine-tuned action sequence can be efficiently executed on the real device, record the final fine-tuned action sequence in a document, including a detailed description of each action and its parameter settings, store the fine-tuned action sequence in the control system of the automatic sorting equipment, ensure that it can be called in actual operation, test and run it on the actual equipment, verify the effectiveness of the fine-tuned action sequence, observe its performance in the actual environment, and make necessary final adjustments based on the feedback from the actual operation to ensure that the operation of the equipment achieves the best effect.

[0033] Step S5: Performing instant automatic sorting based on the fine-tuning action sequence, and collecting sorting images at multiple time points; performing temporal sorting efficiency trend evolution on the sorting images at multiple time points, and generating sorting efficiency trend evolution features; In this embodiment, it is ensured that the automatic sorting equipment has loaded the fine-tuned action sequence, and a pre-operation test is performed to confirm that the equipment status is normal, and sensors (such as lidar, camera) are configured to ensure real-time monitoring of the sorting area, and the sorting operation is started in the control system. The automated equipment performs the sorting task according to the fine-tuned action sequence. The equipment starts to receive solid waste and classifies and processes it according to the coded mapping information. The image acquisition device is triggered at key nodes in the sorting process (such as grabbing, moving and placing) to record the sorting status at each time point, and the image acquisition frequency is set to ensure that images at multiple time points can be captured during the sorting process (for example, one image is captured every 1 second). The collected multi-time point sorting images are pre-processed, including denoising, contrast enhancement, etc., to improve the accuracy of subsequent analysis, and the images are converted into grayscale images or binary images to simplify the analysis process. Computer vision technology (such as image segmentation and feature extraction) is used to extract sorting status information from each image, including the type, quantity and location of each solid waste. , record the sorting success rate, failure rate and other related indicators at each time point, calculate the sorting efficiency indicators according to the extracted sorting status information, such as: the number of objects sorted per hour (Throughput), the correct sorting rate (the number of correctly sorted objects / the total number of sorted objects), the average sorting time (the average sorting time for each object), organize the calculated sorting efficiency indicators into time series data, ensure the consistency of data format and the accuracy of timestamps, create a data framework, record the sorting efficiency indicators at each time point for subsequent analysis, apply time series analysis methods (such as moving average, exponential smoothing method) to perform trend analysis on sorting efficiency data, identify the pattern of efficiency change, generate efficiency trend chart, show the dynamic characteristics of sorting efficiency over time, extract the evolution characteristics of sorting efficiency trend, including efficiency fluctuation amplitude, upward / downward trend and key time nodes, generate a sorting efficiency analysis report, including data visualization (such as line chart, bar chart, etc.) and key analysis results, so that management can formulate improvement strategies.

[0034] Step S6: Dynamically optimize the sorting path planning and adaptively enhance the transfer learning of the sorting efficiency trend evolution characteristics, so as to build an adaptive intelligent sorting model to perform automatic sorting of solid waste.

[0035] In this embodiment, the sorting efficiency trend evolution feature data generated in the previous steps are collected and sorted, including efficiency fluctuations, trend changes, time nodes and other information, and the sorting efficiency features are deeply analyzed using statistical analysis methods (such as regression analysis and correlation analysis) to identify key factors affecting the sorting efficiency, generate feature reports, and clarify which features have a significant impact on the improvement of sorting efficiency. A suitable path planning algorithm (such as A* algorithm, Dijkstra algorithm, RRT) is selected to build a dynamic sorting path model, and the sorting area is divided into multiple grids or areas to ensure that the path planning can cover all objects to be sorted. According to the sorting efficiency trend characteristics, the path planning model is adjusted in real time to ensure that the optimal path is selected under different workloads and material types. A dynamic optimization algorithm (such as genetic algorithm, particle swarm optimization) is applied to optimize the path in real time to ensure the flexibility and efficiency of path selection. The optimized sorting path is verified in a simulation environment to ensure that it can be effectively executed in actual operations. Feedback data after the path execution is collected to further adjust and optimize the path planning, determine the framework of the adaptive intelligent sorting model, and select a suitable reinforcement learning algorithm (such as Deep Q-Learning, Proximal PolicyOptimization) is used as the basis, and the state space (environmental state, object state), action space (equipment operation instructions) and reward mechanism (based on feedback on sorting efficiency and accuracy) are designed. The existing sorting task data (such as the execution results of fine-tuning action sequences) are used as the basis for transfer learning to pre-train the model. The model is adjusted using transfer learning techniques (such as fine-tuning) to enable it to quickly adapt to the new environment. The adaptive intelligent sorting model is trained in a simulated environment, and the strategy is optimized through a continuous trial and error process. The sorting effect of each time is recorded, and the model parameters are adjusted using the feedback mechanism of the reinforcement learning algorithm to improve the sorting efficiency and accuracy. The model is evaluated in an actual sorting environment, and the performance of the model in real tasks is monitored to ensure that it can adapt to different solid waste types and working conditions. The performance indicators of the model, such as sorting accuracy, speed and efficiency, are calculated to evaluate its effectiveness. The trained adaptive intelligent sorting model is deployed to the control system of the automatic sorting equipment to ensure that it can perform sorting tasks in real time. Data in the real environment is regularly collected, and the model is updated and retrained online to ensure that the model can continuously adapt to new sorting conditions.

[0036] In this embodiment, refer to Figure 2 , is a flowchart of detailed implementation steps of step S1. In this embodiment, the detailed implementation steps of step S1 include: Step S11: Acquire monitoring images of the area to be sorted in multiple directions based on a high-resolution panoramic camera; Step S12: Optimizing the image contrast of the monitoring images of the to-be-sorted areas in multiple directions to construct a multi-directional contrast-balanced monitoring image; Step S13: performing image feature point recognition on the multi-directional contrast equalization monitoring image, and marking a plurality of image feature points; Step S14: performing multi-directional precision matching on multiple image feature points, thereby obtaining multi-angle matching data for each feature point; Step S15: performing spatial geometric perspective deviation correction based on the multi-angle matching data of each feature point, thereby obtaining a spatial geometric perspective deviation corrected image; Step S16: performing spatial panoramic image optimization on the spatial angle deviation corrected image to construct a panoramic image of the sorting area.

[0037] In this embodiment, an on-site assessment is conducted in the sorting area to determine the key locations to be monitored. These locations should cover the entire sorting area to ensure that there are no blind spots. High-resolution panoramic cameras are installed at multiple predetermined locations to ensure that the camera fields of view overlap for subsequent image processing. The camera height and angle are ensured to capture the best view. Camera parameters (such as resolution and frame rate) are configured to obtain clear images. After the settings are completed, the monitoring system is started, and the images of the sorting area captured by each camera are regularly saved. The monitoring images of the sorting area obtained from each camera are loaded using an image processing tool for subsequent contrast optimization. , According to the characteristics of the image, select a suitable contrast optimization algorithm, such as histogram equalization or adaptive histogram equalization. These methods enhance the contrast of the image and make the details clearer and more discernible. Apply the selected contrast optimization algorithm to each monitoring image to generate a monitoring image with multi-directional contrast balance. This process significantly improves the details and visualization effects in the image. Select a suitable feature point detection algorithm, such as SIFT, SURF or ORB, to identify the key feature points in the image. These feature points should have good repeatability and stability. Apply the selected feature point detection algorithm to each contrast-optimized monitoring image to identify and mark multiple image feature points. These feature points will become the basis for the subsequent matching process. Select a suitable feature matching algorithm, such as FLANN or BFMatcher. These algorithms can effectively find the correspondence between feature points between different images, match the feature points in each pair of images, and generate multi-angle matching data for each feature point. The purpose of this step is to find the corresponding feature points of the same object at different perspectives to provide a basis for subsequent geometric correction. According to the matched feature points, calculate the geometric transformation between different perspectives, including translation and rotation. Use the RANSAC algorithm to effectively remove mismatches in matching, thereby improving the accuracy of the calculation. Apply the calculated geometric transformation to each image to correct its perspective deviation and generate a spatial geometric perspective deviation corrected image. This step ensures that images of different perspectives can be aligned in the same coordinate system. Use the panoramic stitching algorithm to stitch multiple corrected images into a complete panoramic image. This process requires processing the overlapping areas between images to ensure the smoothness and seamless connection of the stitching. Post-process the stitched panoramic image, including color correction and image smoothing, to improve the overall visual effect and readability of the information. Save the optimized sorting area panoramic image to the specified file path to ensure the safe storage of image data for subsequent analysis and use.

[0038] In this embodiment, the specific steps of step S12 are: Performing image region division on monitoring images of the to-be-sorted areas in multiple directions to obtain multiple image regions; Calculating original grayscale pixel values ​​of multiple image regions to generate original grayscale pixel values ​​of each region; Perform global average calculation on the original grayscale pixel value of each area to obtain the average grayscale pixel value of the image; Perform pixel linear stretching enhancement processing on the average grayscale pixel value of the image to generate an enhanced grayscale pixel value; Stretching pixels in a plurality of image regions according to enhanced grayscale pixel values ​​to generate a brightness enhanced monitoring image; Perform grayscale pixel frequency distribution analysis based on the original grayscale pixel value of each region to generate image grayscale pixel frequency distribution data; Performing cumulative distribution function calculation on the image grayscale pixel frequency distribution data to extract the cumulative distribution function of the image; Perform pixel contrast optimization processing on the cumulative distribution function of the image to obtain a contrast optimized cumulative distribution function; The contrast optimization cumulative distribution function is used to perform contrast equalization on the brightness enhanced monitoring image, thereby constructing a multi-directional contrast balanced monitoring image.

[0039] In this embodiment, according to the characteristics of the area to be sorted, an appropriate image segmentation strategy is selected, such as uniform division or division based on specific features (such as color or texture), and the overall image is divided into multiple small areas to ensure that each area contains the object to be sorted. This is achieved by setting a grid or using a region growing method. For each divided image area, its original grayscale pixel value is calculated, which is usually achieved by converting an RGB image into a grayscale image. The grayscale value of each area is summed and divided by the total number of pixels in the area to obtain the average grayscale pixel value of the area. The average grayscale value of each area is stored in a list or data table for subsequent use. A linear stretching method is used, usually by determining the minimum and maximum grayscale values ​​of the image, linearly mapping them to a new range (such as 0 to 255), applying a linear stretching formula to the grayscale value of each area, and generating an enhanced grayscale pixel value. This process ensures that the contrast of the image is improved. According to the enhanced grayscale pixel value, a brightness value is assigned to each area of ​​the original image to generate a monitoring image with enhanced brightness. All brightness-enhanced areas are synthesized into a complete monitoring image for subsequent analysis. The original grayscale pixel values ​​of each area are counted, and the frequency of occurrence of each grayscale value is calculated to obtain the grayscale pixel frequency distribution data of each area. The grayscale frequency distribution data is stored in a data structure for subsequent cumulative distribution calculation. The grayscale pixel frequency distribution is cumulatively calculated to generate a cumulative distribution function (CDF) for each grayscale value, which represents the proportion of pixels less than or equal to the grayscale value. The calculated cumulative distribution function data is stored in an appropriate data structure for subsequent processing. A suitable contrast optimization algorithm is selected, which is usually a contrast adjustment based on the cumulative distribution function. The brightness-enhanced monitoring image is processed using the optimized cumulative distribution function calculated to improve the overall contrast of the image. The brightness-enhanced monitoring image is histogram-equalized according to the contrast-optimized cumulative distribution function to generate a contrast-equalized image. The generated multi-directional contrast-equalized monitoring image is saved to a storage device for subsequent use and analysis.

[0040] In this embodiment, refer to Figure 3 , is a flowchart of detailed implementation steps of step S2. In this embodiment, the detailed implementation steps of step S2 include: Step S21: performing a solid waste visual positioning scan frame by frame on the panoramic image of the sorting area, and marking each solid waste positioning data; Step S22: identifying the boundary contour of the solid waste according to each solid waste location data, and extracting the boundary contour line of each solid waste; Step S23: fine-grained segmentation of the panoramic image of the sorting area is performed based on the boundary contour line of each solid waste to generate an image boundary frame of each solid waste; Step S24: performing object geometry analysis on each solid waste image boundary frame to generate solid waste geometry features; Step S25: performing object texture detail recognition on each solid waste image boundary frame to obtain solid waste texture detail data; Step S26: Performing a three-dimensional morphological structure analysis on the solid waste texture detail data and the solid waste geometric morphological features, thereby generating a three-dimensional morphological structure representation of each solid waste.

[0041] In this embodiment, a suitable target detection algorithm is selected, such as YOLO (You Only Look Once) or SSD (Single Shot MultiBox Detector), for real-time detection of solid waste in the image, solid waste detection is performed on each frame of the image, the detected position and category of the solid waste are marked as positioning data, the coordinates and size of each solid waste are recorded, an edge detection algorithm (such as Canny edge detection) is used to identify the boundary contour of the solid waste, edge detection is applied to each solid waste positioning image area, its boundary contour line is extracted, and boundary data is generated, and a watershed algorithm or a deep learning-based segmentation network (such as Mask R-CNN) is used for fine-grained image segmentation to ensure that the boundary of each solid waste is clearly visible. Based on the extracted boundary contour, an image bounding box of each solid waste is generated, and the coordinate information of the bounding box is recorded. A geometric morphological analysis method, such as a shape descriptor (such as Hu moment), is used to calculate the geometric features of each solid waste, including area, perimeter, aspect ratio, etc. A geometric analysis is performed on the bounding box of each solid waste to extract its morphological features and record these feature data. A texture analysis algorithm, such as gray-level co-occurrence matrix (GLCM) or LBP (local binary pattern) is used to analyze the texture features of the solid waste. Texture features are extracted from the bounding box of each solid waste to generate texture detail data of the solid waste. A three-dimensional reconstruction technology, such as structured light or stereo vision method, is used to combine geometric morphology and texture data to generate a three-dimensional model of the solid waste. According to the extracted geometric morphological features and texture detail data, a three-dimensional morphological structure representation of each solid waste is constructed to form a visual three-dimensional model.

[0042] In this embodiment, refer to Figure 4 , is a flowchart of detailed implementation steps of step S3. In this embodiment, the detailed implementation steps of step S3 include: Step S31: making solid type classification decisions based on the three-dimensional morphological structure representation of each solid waste, and obtaining type data of each solid waste; Step S32: performing sorting sequence analysis on the type data of each solid waste to generate sorting sequence data of each solid waste; Step S33: performing sorting sequence coding on the sorting sequence data of each solid waste to obtain a solid waste sequence code; Step S34: performing bounding box coding mapping on the image bounding box of each solid waste based on the solid waste sequential coding to obtain a coded mapping bounding box of each solid waste.

[0043] In this embodiment, according to the three-dimensional morphological structure data of solid waste, a suitable machine learning or deep learning classification algorithm is selected, such as support vector machine (SVM), random forest, or convolutional neural network (CNN) to collect and annotate the three-dimensional morphological data of solid waste of different types, to form a training set and a test set, so as to facilitate the training and verification of the model, the training data is used to train the selected classification model, and the model parameters are optimized to improve the classification accuracy, the three-dimensional morphological structure data of each solid waste is input into the trained model, and the type classification is performed to generate the type data of each solid waste, and according to the type and characteristics of the solid waste, a corresponding sorting strategy is formulated, and the priority, processing time, category and other factors are considered, and the type data of each solid waste is analyzed to generate The sorting order data of each solid waste is usually sorted by priority, and the rules for sorting order coding are determined, such as using simple digital codes (such as 1, 2, 3) or more complex coding methods (such as a combination of letters and numbers). According to the formulated coding scheme, the sorting order data of each solid waste is encoded to generate a sequential code for the solid waste, and the encoded sorting order data is saved to the database to ensure the persistence and accessibility of the data. The image bounding box data of each solid waste is collected, including the coordinates and size information of the bounding box. According to the sequential code of the solid waste, the image bounding box of each solid waste is encoded and mapped to generate a coded mapping bounding box, and the generated coded mapping bounding box data is recorded in the database to facilitate subsequent sorting operations and management.

[0044] In this embodiment, step S4 includes the following steps: Step S41: performing equipment action sequence simulation on the code mapping boundary box of each solid waste to obtain action sequence simulation data of the automatic sorting equipment; Step S42: Dynamically sorting movement tracking is performed on the action sequence simulation data of the automatic sorting device to extract the dynamic sorting movement trajectory; Step S43: Analyze the solid waste position change of the dynamic sorting movement trajectory to generate solid waste position change data; Step S44: Calculating the equipment sorting accuracy for the solid waste position change data to obtain the equipment sorting accuracy; Step S45: identifying the action sequence deviation according to the device sorting accuracy to obtain the action sequence deviation value; Step S46: performing deviation compensation calculation on the action sequence deviation value to generate action sequence deviation compensation data; Step S47: fine-tuning the action parameters of the action sequence simulation data of the automatic sorting device based on the action sequence deviation compensation data to construct a fine-tuning action sequence.

[0045] In this embodiment, the coded information of solid waste, including type, size and weight, is collected, and the bounding box data of the waste is obtained by using a sensor (such as a laser scanner) to form the basis of a 3D model. The bounding box data is converted into a 3D model using computer vision tools (such as OpenCV and Blender), and the model is optimized to ensure that it can run effectively in the simulation software. According to the operating manual of the automatic sorting equipment, an action sequence including basic actions such as grabbing, moving, rotating and placing is designed. Each action needs to set specific parameters, such as the height of grabbing, the angle of placing, etc., and the 3D model is simulated in a simulation environment (such as Gazebo, MATLAB). Input the designed action sequence into Simulink, run the simulation, observe the execution of each action and its impact on the location of solid waste, record the time delay, action success rate and error type of each step, deploy sensors (such as lidar, camera) on the sorting equipment, capture the motion data of the equipment in real time, set the data collection frequency to ensure the real-time and accuracy of the data, use data processing algorithms such as Kalman filter to smooth and filter sensor data to reduce noise, extract the motion trajectory including the starting point, end point and path according to the timestamp and coordinate data, and use visualization tools (such as Matplotlib, Plotly) to draw the extracted trajectory in three-dimensional space to ensure that the visualization results can clearly show the motion path of the equipment and the sorting of solid waste. According to the sorting situation, the dynamic sorting movement trajectory is compared with the initial position of the solid waste frame by frame, the displacement of each piece of waste is identified, the movement of each solid waste object is marked for subsequent analysis, and the displacement vector of each solid waste object, including the moving distance (Euclidean distance) and direction (angle), is calculated. The difference between the final position and the expected position of each object is recorded, and the position change data is stored in the database. A suitable data table structure is designed, including fields such as object ID, initial position, end position, displacement, etc., to ensure the scalability of data storage for subsequent large-scale data analysis, and to clarify the calculation standards for sorting accuracy, such as the correct sorting rate (the number of correctly sorted wastes / the total number of sortings) and the wrong sorting rate (the number of wrongly sorted wastes / the total number of sortings).

[0046] Use statistical analysis tools (such as Python's Pandas library) to analyze the solid waste location change data, calculate the sorting accuracy index, and generate statistical reports including charts and key data. Generate a detailed report on sorting accuracy, including data visualization (such as bar charts, pie charts, etc.) and analysis results to help management understand equipment performance. Set standards for deviation identification based on the calculation results of sorting accuracy. For example, if the success rate of an action is lower than the threshold, it is marked as a deviation. Compare the expected action sequence with the actual action sequence, gradually identify the deviation, record the execution time, location and status information of each action, determine the source of the deviation, and record the identified action sequence deviation value in the database. Design a suitable data structure (such as action ID, deviation type, deviation value, etc.), design a compensation algorithm, including a PID controller or a machine learning model, adjust the action parameters according to the deviation value, and set compensation rules. For example, when the deviation exceeds a certain range, automatically adjust the gripping force and angle. According to real-time data feedback, dynamically adjust the action parameters to ensure that each operation can correct the previous deviation. Perform real-time monitoring to quickly respond to different sorting situations, generate records containing compensation data, and ensure that the data format is standardized for subsequent use.

[0047] Apply the compensation data to the control system of the equipment to ensure that it is effective in actual operation. Based on the compensation data, fine-tune the action sequence, update the time, speed, grasping angle and other parameters of each action, ensure that each parameter adjustment has a clear basis, and record the adjustment process and results. Test the fine-tuned action sequence again through simulation software, observe its performance under different conditions, conduct multiple tests to ensure that the new sequence is stable and efficient in actual operation, update the fine-tuned action sequence data to the control system of the automatic sorting equipment, ensure that the equipment can use the new strategy in actual work, formulate a monitoring plan, regularly evaluate the performance of the new sequence, and ensure continuous optimization and improvement.

[0048] In this embodiment, step S5 includes the following steps: Step S51: performing instant automatic sorting based on the fine-tuning action sequence, and collecting sorting images at multiple time points; Step S52: performing sorting status analysis on the multi-time point sorting images to extract the sorting status parameters of the automation equipment; Step S53: performing a quantitative evaluation of the sorting efficiency of the automated equipment sorting state parameters to generate a quantitative evaluation value of the sorting efficiency; Step S54: performing time-series sorting efficiency trend evolution on the sorting efficiency quantitative evaluation value to generate sorting efficiency trend evolution characteristics.

[0049] In this embodiment, it is ensured that the automatic sorting equipment has loaded the fine-tuned action sequence, and a preliminary test is performed to verify that the system is operating normally. An image acquisition device (such as a high-resolution camera) is configured to ensure that it covers the sorting area without blind spots. The automatic sorting program is started in the control system, and real-time sorting operations are performed according to the fine-tuned action sequence. The equipment begins to receive solid waste materials and classifies them according to the preset sorting logic. The image acquisition device is triggered at each key time point of sorting (such as when the material approaches the sorting point, when the material sorting is completed, etc.), and the images in the sorting process are recorded. The frequency of image acquisition is set, for example, one picture is collected per second to ensure that dynamic changes can be captured. The collected images are stored in the cloud or local database, and the image file naming is ensured to be standardized to facilitate subsequent retrieval and analysis. The collected multi-time point sorting images are preprocessed, including noise removal, contrast enhancement, brightness adjustment, etc., to improve the accuracy of subsequent analysis.

[0050] Use image processing tools (such as OpenCV) to convert the image format into a format suitable for analysis, apply computer vision algorithms (such as object recognition and edge detection) to extract sorting state parameters from the image, such as: the type and number of sorted objects, the execution status of the sorting action (success, failure), and the working status of the equipment (such as running speed and grasping strength). Use deep learning models (such as YOLO and Faster R-CNN) to perform real-time object detection to ensure high accuracy of state extraction. Record the extracted sorting state parameters (such as object type, sorting status, timestamp, etc.) in the database for subsequent analysis and clarify the quantitative indicators of sorting efficiency, such as: the number of objects sorted per hour (Throughput), the correct sorting rate (the number of correctly sorted objects / the total number of sorted objects), and the sorting time (the average sorting time for each object). Use analysis tools (such as Python's Pandas and NumPy) to perform statistical analysis on the sorting state parameters, calculate the sorting efficiency indicators, and generate a report containing detailed data and charts for each indicator. Generate a quantitative evaluation value of the sorting efficiency based on the calculation results to form a standardized evaluation report for subsequent comparison and optimization.

[0051] The quantitative evaluation values ​​of sorting efficiency are sorted in chronological order to form a time series data set, which is convenient for analyzing trend changes. It ensures that the data format is consistent and the timestamp is accurate for subsequent analysis. Time series analysis methods (such as moving average and exponential smoothing) are used to perform trend analysis on sorting efficiency data, identify potential patterns and periodic changes in efficiency, so as to find out the factors affecting sorting efficiency and generate characteristic data of sorting efficiency trend evolution, including: time nodes when efficiency rises or falls, amplitude and frequency of efficiency fluctuations. Visualization tools (such as Matplotlib and Seaborn) are used to draw sorting efficiency trend graphs to show the dynamic characteristics of efficiency changes.

[0052] In this embodiment, the specific steps of step S6 are: Step S61: performing dynamic sorting path planning optimization on the sorting efficiency trend evolution characteristics to obtain a dynamic sorting optimization path; Step S62: optimizing the whole sorting process of the fine-tuning action sequence based on the dynamic sorting optimization path to generate a global optimization action sequence; Step S63: Perform adaptive reinforcement transfer learning on the global optimization action sequence to build an adaptive intelligent sorting model to perform automatic sorting operations for solid waste.

[0053] In this embodiment, sorting efficiency trend evolution characteristic data are collected, including sorting efficiency indicators and sorting path information in different time periods, and key influencing factors such as object type, sorting position, equipment status, etc. are determined. A path planning algorithm (such as A* algorithm, Dijkstra algorithm or RRT algorithm) is used to build a preliminary sorting path model, and the sorting area is divided into multiple areas. The starting position and target position of the object are considered to ensure that the path is as short as possible and efficient. According to the sorting efficiency trend characteristics, the path planning model is adjusted in real time. For example, if the sorting efficiency of a certain path decreases, the system will automatically select other better paths. Optimization techniques (such as genetic algorithms and particle swarm optimization) are used to iteratively optimize the dynamic path to ensure the adaptability of path selection under different conditions. A dynamic sorting optimization path is generated and recorded in a database for subsequent reference and application. The optimized path is displayed through a visualization tool to ensure that the operator can intuitively understand it.

[0054] Clarify the goals of full-process optimization, such as improving overall sorting efficiency, reducing energy consumption and shortening sorting time, determine the action sequence that needs to be optimized, including grabbing, moving, placing, etc., analyze and fine-tune the execution order and timing of each action in the action sequence based on the dynamic sorting optimization path, apply scheduling algorithms (such as greedy algorithms, dynamic programming) to globally optimize the action sequence, ensure the coordination and coherence between each action, generate a globally optimized action sequence, record the parameters of each action (such as execution time, force, position, etc.), and ensure the reasonable connection between actions, store the globally optimized action sequence in the control system for calling in actual operations, determine the framework of the adaptive intelligent sorting model, select suitable reinforcement learning algorithms (such as Deep Q-Learning, Proximal Policy Optimization), design state space, action space and reward mechanism so that the model can learn how to make optimal decisions in sorting tasks.

[0055] The global optimization action sequence is used as training data, and the model is pre-trained using transfer learning methods (such as fine-tuning). The training model adapts to the new environment and quickly adjusts the strategy to deal with unseen sorting situations. The model is trained in a simulated environment, and the strategy is optimized through a continuous trial and error process. The sorting effect of each time is recorded, the model performance is evaluated regularly, and the performance of the model in the real sorting scenario is tested using the validation set to ensure that it has high accuracy and efficiency. The trained adaptive intelligent sorting model is deployed to the control system of the automatic sorting equipment for on-site testing. The performance of the model in actual sorting operations is monitored in real time to ensure that it can automatically adjust the strategy to adapt to different sorting tasks.

[0056] In this embodiment, a solid waste automatic sorting device using machine vision is provided, which is used to perform the solid waste automatic sorting method using machine vision as described above, including: The geometric perspective correction module is used to obtain monitoring images of the area to be sorted in multiple directions; perform spatial geometric perspective deviation correction on the monitoring images of the area to be sorted in multiple directions, and perform spatial panoramic image optimization to construct a panoramic image of the sorting area; The three-dimensional morphological structure module is used to perform visual positioning scanning of solid waste frame by frame in the panoramic image of the sorting area and perform three-dimensional morphological structure analysis to generate a three-dimensional morphological structure representation of each solid waste; A coding mapping module is used to make solid type classification decisions based on the three-dimensional morphological structure representation of each solid waste, and perform bounding box coding mapping to obtain a coding mapping bounding box for each solid waste; The action parameter fine-tuning module is used to simulate the equipment action sequence for each solid waste coding mapping boundary box, and then fine-tune the action parameters to construct a fine-tuning action sequence; The sorting efficiency trend module is used to perform instant automatic sorting based on fine-tuning action sequences and collect sorting images at multiple time points; it performs temporal sorting efficiency trend evolution on the sorting images at multiple time points and generates sorting efficiency trend evolution features; The sorting path optimization module is used to perform dynamic sorting path planning optimization and adaptive reinforcement transfer learning on the sorting efficiency trend evolution characteristics, so as to build an adaptive intelligent sorting model to perform automatic solid waste sorting operations.

[0057] The present invention helps to eliminate perspective distortion in images and improve sorting accuracy by acquiring monitoring images of the area to be sorted in multiple orientations and performing geometric perspective deviation correction, constructs a panoramic image of the sorting area to provide a more comprehensive field of view for subsequent solid waste identification and sorting, and increases the global perception capability of the system. Through frame-by-frame visual positioning scanning and three-dimensional morphological structure analysis of solid waste, the morphological characteristics of each solid waste are accurately acquired, providing important data support for subsequent classification and sorting. The generation of three-dimensional morphological structure representation helps the system to accurately identify and classify solid waste, improves sorting accuracy and efficiency, and makes classification decisions and bounding box coding mapping based on the three-dimensional morphological structure representation of solid waste to achieve accurate solid type classification and position marking. The coding mapping bounding box provides accurate Accurate location information provides an important basis for subsequent processing and sorting. Through equipment action sequence simulation and action parameter fine-tuning, the solid waste processing action is optimized to improve the accuracy and efficiency of sorting. Building a fine-tuning action sequence helps to ensure that each solid waste is properly processed and reduce processing errors and losses. Analysis of the evolution characteristics of the time-series sorting efficiency trend helps the system understand the changing rules of sorting efficiency, optimize the sorting strategy and improve the overall efficiency. Through instant automatic sorting and analysis of efficiency trends, the system adjusts the strategy in real time to improve sorting efficiency and accuracy. Dynamic sorting path planning optimization and adaptive reinforcement transfer learning help to build an intelligent sorting model and improve system adaptability and performance. The adaptive intelligent sorting model adjusts the sorting path according to the real-time situation to improve the efficiency and accuracy of automatic sorting of solid waste.

[0058] Therefore, the embodiments should be regarded as illustrative and non-restrictive from all points, and the scope of the present invention is limited by the appended claims rather than the above description, and it is therefore intended that all changes falling within the meaning and range of equivalent elements of the application documents are included in the present invention.

[0059] The above is only a specific embodiment of the present invention, so that those skilled in the art can understand or implement the present invention. Various modifications to these embodiments will be apparent to those skilled in the art, and the general principles defined herein may be implemented in other embodiments without departing from the spirit or scope of the present invention. Therefore, the present invention will not be limited to the embodiments shown herein, but should conform to the widest scope consistent with the principles and novel features invented herein.

Claims

1. A method for automatic sorting of solid waste using machine vision, characterized in that: The following steps are involved: Step S1: Acquire monitoring images of the area to be sorted in multiple directions; Correct the spatial geometric perspective deviation of the monitoring images of the sorting area in multiple directions, optimize the spatial panoramic image, and construct a panoramic image of the sorting area; Step S2: performing visual positioning scanning of solid waste frame by frame on the panoramic image of the sorting area, and performing three-dimensional morphological structure analysis, thereby generating a three-dimensional morphological structure representation of each solid waste; Step S3: making a solid type classification decision based on the three-dimensional morphological structure representation of each solid waste, and performing bounding box coding mapping to obtain a coding mapping bounding box of each solid waste; Step S4: simulate the equipment action sequence for each coded mapping boundary box of solid waste, and then fine-tune the action parameters to construct a fine-tuned action sequence; Step S5: Performing instant automatic sorting based on the fine-tuning action sequence, and collecting sorting images at multiple time points; Perform temporal sorting efficiency trend evolution on multi-time point sorting images to generate sorting efficiency trend evolution features; Step S6: Dynamically optimize the sorting path planning and adaptively enhance the transfer learning of the sorting efficiency trend evolution characteristics, so as to build an adaptive intelligent sorting model to perform automatic sorting of solid waste.

2. The method for automatic sorting of solid waste using machine vision according to claim 1, characterized in that: The specific steps of step S1 are: Step S11: Acquire monitoring images of the area to be sorted in multiple directions based on a high-resolution panoramic camera; Step S12: Optimizing the image contrast of the monitoring images of the to-be-sorted areas in multiple directions to construct a multi-directional contrast-balanced monitoring image; Step S13: performing image feature point recognition on the multi-directional contrast equalization monitoring image, and marking a plurality of image feature points; Step S14: performing multi-directional precision matching on multiple image feature points, thereby obtaining multi-angle matching data for each feature point; Step S15: performing spatial geometric perspective deviation correction based on the multi-angle matching data of each feature point, thereby obtaining a spatial geometric perspective deviation corrected image; Step S16: performing spatial panoramic image optimization on the spatial angle deviation corrected image to construct a panoramic image of the sorting area.

3. The method for automatic sorting of solid waste using machine vision according to claim 2, characterized in that: The specific steps of step S12 are: Performing image region division on monitoring images of the to-be-sorted areas in multiple directions to obtain multiple image regions; Calculating original grayscale pixel values ​​of multiple image regions to generate original grayscale pixel values ​​of each region; Perform global average calculation on the original grayscale pixel value of each area to obtain the average grayscale pixel value of the image; Perform pixel linear stretching enhancement processing on the average grayscale pixel value of the image to generate an enhanced grayscale pixel value; Stretching pixels in a plurality of image regions according to enhanced grayscale pixel values ​​to generate a brightness enhanced monitoring image; Perform grayscale pixel frequency distribution analysis based on the original grayscale pixel value of each region to generate image grayscale pixel frequency distribution data; Performing cumulative distribution function calculation on the image grayscale pixel frequency distribution data to extract the cumulative distribution function of the image; Perform pixel contrast optimization processing on the cumulative distribution function of the image to obtain a contrast optimized cumulative distribution function; The contrast optimization cumulative distribution function is used to perform contrast equalization on the brightness enhanced monitoring image, thereby constructing a multi-directional contrast balanced monitoring image.

4. The method for automatic sorting of solid waste using machine vision according to claim 1, characterized in that: The specific steps of step S2 are: Step S21: performing a solid waste visual positioning scan frame by frame on the panoramic image of the sorting area, and marking each solid waste positioning data; Step S22: identifying the boundary contour of the solid waste according to each solid waste location data, and extracting the boundary contour line of each solid waste; Step S23: fine-grained segmentation of the panoramic image of the sorting area is performed based on the boundary contour line of each solid waste to generate an image boundary frame of each solid waste; Step S24: performing object geometry analysis on each solid waste image boundary frame to generate solid waste geometry features; Step S25: performing object texture detail recognition on each solid waste image boundary frame to obtain solid waste texture detail data; Step S26: Performing a three-dimensional morphological structure analysis on the solid waste texture detail data and the solid waste geometric morphological features, thereby generating a three-dimensional morphological structure representation of each solid waste.

5. The method for automatic sorting of solid waste using machine vision according to claim 1, characterized in that: The specific steps of step S3 are: Step S31: making solid type classification decisions based on the three-dimensional morphological structure representation of each solid waste, and obtaining type data of each solid waste; Step S32: performing sorting sequence analysis on the type data of each solid waste to generate sorting sequence data of each solid waste; Step S33: performing sorting sequence coding on the sorting sequence data of each solid waste to obtain a solid waste sequence code; Step S34: performing bounding box coding mapping on the image bounding box of each solid waste based on the solid waste sequential coding to obtain a coded mapping bounding box of each solid waste.

6. The method for automatic sorting of solid waste using machine vision according to claim 1, characterized in that: The specific steps of step S4 are: Step S41: performing equipment action sequence simulation on the code mapping boundary box of each solid waste to obtain action sequence simulation data of the automatic sorting equipment; Step S42: Dynamically sorting movement tracking is performed on the action sequence simulation data of the automatic sorting device to extract the dynamic sorting movement trajectory; Step S43: Analyze the solid waste position change of the dynamic sorting movement trajectory to generate solid waste position change data; Step S44: Calculating the equipment sorting accuracy for the solid waste position change data to obtain the equipment sorting accuracy; Step S45: identifying the action sequence deviation according to the device sorting accuracy to obtain the action sequence deviation value; Step S46: performing deviation compensation calculation on the action sequence deviation value to generate action sequence deviation compensation data; Step S47: fine-tuning the action parameters of the action sequence simulation data of the automatic sorting device based on the action sequence deviation compensation data to construct a fine-tuning action sequence.

7. The method for automatic sorting of solid waste using machine vision according to claim 1, characterized in that: The specific steps of step S5 are: Step S51: performing instant automatic sorting based on the fine-tuning action sequence, and collecting sorting images at multiple time points; Step S52: performing sorting status analysis on the multi-time point sorting images to extract the sorting status parameters of the automation equipment; Step S53: performing a quantitative evaluation of the sorting efficiency of the automated equipment sorting state parameters to generate a quantitative evaluation value of the sorting efficiency; Step S54: performing time-series sorting efficiency trend evolution on the sorting efficiency quantitative evaluation value to generate sorting efficiency trend evolution characteristics.

8. The method for automatic sorting of solid waste using machine vision according to claim 1, characterized in that: The specific steps of step S6 are: Step S61: performing dynamic sorting path planning optimization on the sorting efficiency trend evolution characteristics to obtain a dynamic sorting optimization path; Step S62: optimizing the whole sorting process of the fine-tuning action sequence based on the dynamic sorting optimization path to generate a global optimization action sequence; Step S63: Perform adaptive reinforcement transfer learning on the global optimization action sequence to build an adaptive intelligent sorting model to perform automatic sorting operations for solid waste.

9. An automatic solid waste sorting device using machine vision, characterized in that: The method for automatically sorting solid waste using machine vision as claimed in claim 1 comprises: The geometric perspective correction module is used to obtain monitoring images of the area to be sorted in multiple directions; perform spatial geometric perspective deviation correction on the monitoring images of the area to be sorted in multiple directions, and perform spatial panoramic image optimization to construct a panoramic image of the sorting area; The three-dimensional morphological structure module is used to perform visual positioning scanning of solid waste frame by frame in the panoramic image of the sorting area and perform three-dimensional morphological structure analysis to generate a three-dimensional morphological structure representation of each solid waste; A coding mapping module is used to make solid type classification decisions based on the three-dimensional morphological structure representation of each solid waste, and perform bounding box coding mapping to obtain a coding mapping bounding box for each solid waste; The action parameter fine-tuning module is used to simulate the equipment action sequence for each solid waste coding mapping boundary box, and then fine-tune the action parameters to construct a fine-tuning action sequence; The sorting efficiency trend module is used to perform instant automatic sorting based on fine-tuning action sequences and collect sorting images at multiple time points; it performs temporal sorting efficiency trend evolution on the sorting images at multiple time points and generates sorting efficiency trend evolution features; The sorting path optimization module is used to perform dynamic sorting path planning optimization and adaptive reinforcement transfer learning on the sorting efficiency trend evolution characteristics, so as to build an adaptive intelligent sorting model to perform automatic solid waste sorting operations.

Images