A sensor coupling-based machine vision garbage recognition and positioning technology

Abstract

The application discloses a kind of machine vision garbage identification positioning technology based on sensor coupling including object image information, infrared spectrum information and 3D profile information by sensor acquisition;Image information is transmitted into YOLOv5 target identification network, and dataset type is established;Infrared spectrum information is transmitted to principal component analysis (PCA) and support vector machine (SVM) model, and dataset type is established;YOLOv5 identified target class is coupled with the object class identified by infrared spectrum;The two-dimensional coordinates of object obtained by YOLOv5 are coupled with the three-dimensional coordinates obtained by sensor;The target class obtained by coupling is transmitted to parallel machine arm;Coupling obtained coordinates are transmitted to PLC, and positioning is carried out by the servo control module of PLC;Parallel machine arm guides mechanical arm to grasp by controller, and selects placement position according to target class;The present application uses multiple sensor coupling, so that different sensors complement each other, for only image can be distinguished class, can guarantee identification accuracy.

Description

Technical Field

[0001] This invention relates to the field of machine vision technology, and more particularly to a machine vision-based garbage identification and location technology based on sensor coupling. Background Technology

[0002] Current technologies primarily rely on manual processes for waste sorting, which is costly, inefficient, and negatively impacts worker health. While some utilize machine vision technology, it only relies on single-image recognition for object identification and location. Single-image recognition cannot distinguish between objects with similar shapes but different materials, thus failing to address the intricate sorting of high-value-added products during recycling. Single-image infrared recognition is less accurate and efficient than image recognition and is easily affected by external environmental interference; its accuracy and positioning still have room for improvement. This patent proposes a faster sorting speed compared to manual methods, with an estimated capacity of 5400 pieces / hour per production line. It is also more environmentally friendly and can effectively alleviate the labor shortage problem in labor-intensive industries. Compared to the current integration of machine vision and waste sorting, this technology explores ways to improve recognition and positioning accuracy through multi-sensor coupling. Summary of the Invention

[0003] The purpose of this section is to outline some aspects of embodiments of the present invention and to briefly describe some preferred embodiments. Simplifications or omissions may be made in this section, as well as in the abstract and title of this application, to avoid obscuring the purpose of these documents; however, such simplifications or omissions should not be construed as limiting the scope of the invention.

[0004] In view of the problems existing in the current sensor-coupled machine vision garbage identification and location technology, this invention is proposed.

[0005] Therefore, the purpose of this invention is to provide a machine vision-based waste identification and positioning technology based on sensor coupling, which improves the accuracy of identification and positioning through the coupling of multiple sensors.

[0006] To solve the above-mentioned technical problems, the present invention provides the following technical solution: a machine vision garbage identification and positioning technology based on sensor coupling, which collects object image information, infrared spectral information and 3D contour information through sensors;

[0007] Image information is fed into the YOLOv5 object recognition network to create a dataset type;

[0008] Infrared spectral information is transmitted to principal component analysis (PCA) and support vector machine (SVM) models to establish dataset types;

[0009] Couple the target categories identified by YOLOv5 with the object categories identified by infrared spectroscopy;

[0010] Couple the two-dimensional coordinates of the object obtained by YOLOv5 with the three-dimensional coordinates obtained by the sensor;

[0011] The target category obtained through coupling is transmitted to the parallel robotic arm;

[0012] The coordinates obtained from the coupling are transmitted to the PLC, and the positioning is performed by the PLC's servo control module.

[0013] The parallel robotic arm is guided by a controller to grasp objects and select the placement location based on the type of the target.

[0014] As a preferred embodiment of the sensor-coupled machine vision-based waste identification and positioning technology described in this invention, the coordinate system involved in sensor calibration includes four planar coordinate systems, namely: pixel planar coordinate system. Image physical coordinate system (image plane coordinate system) Camera coordinate system and world coordinate system Pixel coordinates The image plane coordinates can be obtained using the following formula:

[0015]

[0016] in, , , , All of these are set parameters. , It represents the actual size of a pixel on the image sensor and connects the pixel coordinate system and the real size coordinate system; , It is the center of the image plane, and the intrinsic and extrinsic parameters can be obtained in the end; establish a pixel plane coordinate system. When using a checkerboard pattern as the calibration object, the corresponding values ​​of the checkerboard side length and the number of pixels of the corresponding side in the image are calculated during the calibration process to complete the establishment of the coordinate system.

[0017] As a preferred embodiment of the machine vision-based waste identification and location technology based on sensor coupling described in this invention, the above formula can be converted into matrix form as follows:

[0018]

[0019] The camera coordinate system is the world coordinate system after rotation and translation, which can be obtained by rotation matrix R and translation matrix R.

[0020] The following relationship is obtained by shifting the matrix T:

[0021]

[0022] Based on the principles of camera imaging, we can derive the following formula for the side lengths of similar triangles:

[0023]

[0024] The final formula can be obtained as follows:

[0025] .

[0026] As a preferred embodiment of the machine vision-based waste identification and localization technology based on sensor coupling described in this invention, the following applies: straight lines in the edge portion of the image acquired by the camera may be distorted into curves; the radial distortion mathematical model is:

[0027]

[0028] In the formula, , , Indicates the radial distortion coefficients of each order; It is an ideal, distortion-free coordinate system (image coordinate system); These are the coordinates of the pixels in the distorted image; This represents the distance between the target coordinates and the origin, i.e. ;

[0029] The mathematical model for tangential distortion is:

[0030]

[0031] In the formula , Indicates the tangential distortion coefficients of each order; This represents the distance between the target coordinates and the origin, i.e. ;

[0032] merge:

[0033]

[0034] Ultimately, five distortion parameters can be obtained. , , , , By calibrating the intrinsic and distortion parameters of the industrial camera, the acquired image can be corrected by performing distortion correction processing.

[0035] As a preferred embodiment of the machine vision-based waste identification and positioning technology based on sensor coupling described in this invention, the following steps are taken: The obtained reflected signal is corrected using a reference spectrum. First, whiteboard sampling is performed at a probe height h=1m. A whole white ceramic tile is laid under the probe for continuous sampling. Then, the probe is wrapped with black cotton cloth to collect data from the blackboard. Subsequently, spectral correction is performed using the following formula:

[0036]

[0037] In the formula, The signal strength is the signal intensity after correction to the reference spectrum at the i-th wavelength. The signal strength of the i-th wavelength of the original reflected signal; Let be the signal intensity of the i-th wavelength in the diffuse reflectance spectrum of the whiteboard; The signal intensity of the dark current background spectrum at the i-th wavelength.

[0038] As a preferred embodiment of the sensor-coupled machine vision-based garbage identification and localization technology described in this invention, the YOLOv5 object detection model first obtains anchor boxes through K-nearest neighbor (KNN) clustering, and then predicts accurate target boxes through target box regression. YOLOv5 uses the following formula:

[0039]

[0040]

[0041]

[0042]

[0043] in, , , , These are all parameters that need to be iterated during the backpropagation process. It is the sigmoid activation function. , The center of the prediction box coordinate, and These represent the length and width of a single grid cell, respectively. , For the width and length of the prediction box, , These represent the width and length of the anchor boxes obtained from clustering, respectively.

[0044] The loss function (L) of YOLOv5 is the confidence loss ( ), classification loss ( ), bounding box loss ( The weighted sum of the three parts:

[0045]

[0046] Confidence loss and classification loss are defined using binary cross-entropy, as shown in the following formula:

[0047]

[0048]

[0049] The bounding box loss is calculated using CIOU, as shown in the following formula:

[0050]

[0051] Where IOU is the intersection-union ratio of the predicted bounding box and the ground truth, and b and These are the coordinates of the center point of the predicted bounding box and the center point of the ground truth bounding box, respectively. and Here, represents the width and height of the ground truth bounding box, w and h represent the width and height of the predicted bounding box, v is the aspect ratio consistency parameter, and α is the balance parameter. Their expressions are as follows:

[0052]

[0053]

[0054] During training, after the image is fed into the improved YOLOv5 network model, it first undergoes KNN clustering preprocessing, then feature extraction is performed in the backbone network, and finally prediction is performed at three scales in the head part. The prediction results are calculated using the loss function, the gradient is returned, and the network weights are updated. During the prediction process, the image first undergoes KNN clustering preprocessing, is divided into three scales in the head, and finally is fed into the backbone network for multi-scale coordinate and category prediction.

[0055] As a preferred embodiment of the sensor-coupled machine vision-based garbage identification and localization technology described in this invention, the following steps are performed: spectral information is preprocessed before being transmitted to the PCA and SVM models, using SG filtering for smoothing (7 windows, 2 polynomial order), linear detrending for baseline correction, and standardization using standard normal transformation (SVN); then, PCA data dimensionality reduction is performed using the Python function sklearn.decomposition.PCA, with 6-8 principal components being preferred, the specific choice depending on the scenario; finally, SVM category prediction is performed using the Python function sklearn.svm.SVC.

[0056] As a preferred embodiment of the machine vision garbage identification and positioning technology based on sensor coupling described in this invention, the target category and the object category are coupled, and the category predicted by the YOLOv5 network model and the category predicted by the spectral method are jointly judged, specifically in a cautious parallel mode.

[0057] As a preferred embodiment of the machine vision garbage identification and positioning technology based on sensor coupling described in this invention, the two-dimensional coordinates are coupled with the three-dimensional coordinates. The three-dimensional coordinates obtained by the laser sensor are used as the reference. The two-dimensional coordinates obtained by the target recognition algorithm are mainly used to verify the consistency of the object. That is, if the x and y coordinates of the three-dimensional coordinates obtained by the laser are significantly different from the coordinates given by YOLOv5, the algorithm skips or reports an error; if the difference is not significant, the three-dimensional coordinates are transmitted to the robotic arm gripper.

[0058] As a preferred embodiment of the machine vision-based waste identification and positioning technology based on sensor coupling described in this invention, the communication between the industrial control computer and the PLC is achieved through UDP Ethernet communication; and the transmission between the PLC and the frequency converter is achieved through CC-Link connection.

[0059] The beneficial effects of this invention are:

[0060] Compared to manual sorting, this patented technology offers faster sorting speeds, with an estimated capacity of 5400 pieces / hour per production line. It is also more environmentally friendly and can effectively alleviate the labor shortage problem in labor-intensive industries in the Yangtze River Delta region. Compared to existing applications combining machine vision and waste sorting, this technology uses multi-sensor coupling, allowing different sensors to complement each other's strengths. For categories that can be identified solely by image data, it ensures high accuracy. For categories requiring further material differentiation, the combined image and infrared analysis is sufficient for most identification scenarios, demonstrating greater application potential. Furthermore, the combined use of coordinate information from laser sensors and image-based analysis results in more precise positioning. Attached Figure Description

[0061] To more clearly illustrate the technical solutions of the embodiments of the present invention, the accompanying drawings used in the description of the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort. Wherein:

[0062] Figure 1 This is a schematic diagram of the YOLOv5 network structure for the machine vision-based garbage identification and location technology based on sensor coupling, as presented in this invention.

[0063] Figure 2 This is a schematic diagram of the network structure of the machine vision garbage identification and localization technology based on sensor coupling of the present invention, which incorporates a channel attention mechanism.

[0064] Figure 3 This is a schematic diagram of the process of the machine vision garbage identification and positioning technology based on sensor coupling of the present invention. Detailed Implementation

[0065] To make the above-mentioned objects, features and advantages of the present invention more apparent and understandable, the specific embodiments of the present invention will be described in detail below with reference to the accompanying drawings.

[0066] Many specific details are set forth in the following description in order to provide a full understanding of the invention. However, the invention may also be practiced in other ways different from those described herein, and those skilled in the art can make similar extensions without departing from the spirit of the invention. Therefore, the invention is not limited to the specific embodiments disclosed below.

[0067] Secondly, the term "one embodiment" or "embodiment" as used herein refers to a specific feature, structure, or characteristic that may be included in at least one implementation of the present invention. The phrase "in one embodiment" appearing in different places in this specification does not necessarily refer to the same embodiment, nor is it a single or selective embodiment that is mutually exclusive with other embodiments.

[0068] Secondly, the present invention is described in detail with reference to the schematic diagrams. When detailing the embodiments of the present invention, for ease of explanation, the cross-sectional views illustrating the device structure may be partially enlarged, not according to the usual scale. Furthermore, the schematic diagrams are merely examples and should not limit the scope of protection of the present invention. In addition, actual fabrication should include three-dimensional spatial dimensions of length, width, and depth.

[0069] Example 1

[0070] Reference Figure 1 , 23. This is the first embodiment of the present invention, which provides a machine vision-based waste identification and positioning technology based on sensor coupling. It acquires image information, infrared spectral information, and 3D contour information of objects through a CCD camera, a near-infrared sensor, and a line laser sensor. This information is acquired by a PLC and sent to a host computer for further processing. The CCD camera needs to be calibrated to improve positioning accuracy. The specific method is as follows:

[0071] Camera calibration involves four planar coordinate systems: pixel plane coordinate system, pixel plane coordinate system, and pixel plane coordinate system. Image physical coordinate system (image plane coordinate system) Camera coordinate system and world coordinate system .

[0072] Furthermore, pixel coordinates The image plane coordinates can be obtained using the following formula:

[0073]

[0074] in, , , , All of these are set parameters. , It represents the actual size of a pixel on the image sensor and connects the pixel coordinate system and the real size coordinate system; , It is the center of the image plane, and the intrinsic and extrinsic parameters can be obtained in the end.

[0075] Establish pixel plane coordinate system When using a checkerboard pattern as the calibration object, the corresponding values ​​of the checkerboard side length and the number of pixels of the corresponding side in the image are calculated during the calibration process to complete the establishment of the coordinate system.

[0076] The above formula can be converted into matrix form using linear algebra:

[0077]

[0078] Camera coordinate system and world coordinate system The relationship between the camera coordinate system and the world coordinate system after rotation and translation can be obtained using the rotation matrix R and the translation matrix T as follows:

[0079]

[0080] Camera coordinate system and world coordinate system The relationship between the sides of similar triangles can be derived based on the principles of camera imaging:

[0081]

[0082] The final formula can be obtained as follows:

[0083] .

[0084] Then, the two-dimensional planar data of the high-pressure valve and the low-pressure valve are output. The two-dimensional planar data mainly includes two-dimensional center coordinate data and two-dimensional contour data.

[0085] Due to manufacturing errors in the camera lens itself and deviations in the lens assembly process, images acquired by machine vision systems are distorted, resulting in differences between the acquired images and the actual images. In practical applications, industrial cameras typically use a pinhole camera model to correct these distortions.

[0086] Furthermore, lens distortion can be categorized into radial distortion, tangential distortion, and thin prism distortion. Among these, radial distortion and tangential distortion have a greater impact on the projected image; therefore, this application primarily considers radial distortion and tangential distortion.

[0087] Radial distortion mainly occurs at the edges of the camera's field of view, while the degree of radial distortion is weaker in the center of the image. Therefore, straight lines in the edge areas of the image captured by the camera may be distorted into curves. The mathematical model for radial distortion is:

[0088]

[0089] In the formula, , , Indicates the radial distortion coefficients of each order; It is an ideal, distortion-free coordinate system (image coordinate system); These are the coordinates of the pixels in the distorted image; This represents the distance between the target coordinates and the origin, i.e. ;

[0090] Tangential distortion is mainly caused by processing and installation errors during the manufacturing process. Installation errors cause the lens plane to be non-parallel to the camera plane, resulting in distortion in the acquired image. The mathematical model for tangential distortion is as follows:

[0091]

[0092] In the formula , Indicates the tangential distortion coefficients of each order; This represents the distance between the target coordinates and the origin, i.e. ;

[0093] The two sets of mathematical models for radial and tangential distortion are combined:

[0094]

[0095] Ultimately, five distortion parameters can be obtained. , , , , By calibrating the intrinsic and distortion parameters of the industrial camera, the acquired image can be corrected by performing distortion correction processing.

[0096] The spectrum acquired by the near-infrared sensor needs to be calibrated. The specific steps are as follows:

[0097] To eliminate the influence of measurements taken by different instruments or at different times, the obtained reflection signal must be corrected using reference spectra. The two reference spectra are the white board spectrum obtained by the light source shining on the diffuse white board through the probe, and the background spectrum, i.e., dark current, collected by the spectrometer when the probe is in a closed state.

[0098] First, whiteboard sampling was performed at a probe height of h = 1m. A whole white ceramic tile was laid under the probe, and continuous sampling was conducted. Then, the probe was wrapped with black cotton cloth, and the blackboard was sampled. Spectral correction was then performed using the following formula.

[0099]

[0100] In the formula, —Signal strength after correction to the i-th wavelength reference spectrum;

[0101] —The signal strength of the i-th wavelength of the original reflected signal;

[0102] —The signal intensity at the i-th wavelength of the diffuse reflectance spectrum of the whiteboard;

[0103] —The signal intensity of the dark current background spectrum at the i-th wavelength.

[0104] The image information captured by the CCD camera is transmitted into the improved YOLOv5 target recognition network, which provides the target category and two-dimensional location. Specifically:

[0105] See YOLOv5 structure. Figure 1The YOLOv5 network architecture consists of a backbone and a head. The backbone comprises Focus, Conv, BSCP (Bottlenack CSP), and SPP (Spatial Pyramid Pooling) modules. The Focus module slices the input feature map into four parts spatially, reducing complexity and making the network more lightweight. Conv and BSCP are different types of feature extraction modules; the difference is that BSCP includes residual extraction, while Conv is a simple convolution with activation. The SPP module upsamples the input feature map by 5x, 9x, and 13x, respectively, which to some extent introduces a spatial attention mechanism. The head includes two consecutive convolutional upsampling operations, allowing the network to output at three scales, improving its predictive ability for targets at different scales.

[0106] The YOLOv5 object detection model first obtains anchor boxes through K-nearest neighbor (KNN) clustering, and then predicts the accurate object boxes through object box regression. YOLOv5 uses the following formula:

[0107]

[0108]

[0109]

[0110]

[0111] in, , , , These are all parameters that need to be iterated during the backpropagation process. It is the sigmoid activation function. , The center of the prediction box coordinate, and These represent the length and width of a single grid cell, respectively. , For the width and length of the prediction box, , These represent the width and length of the anchor boxes obtained from clustering, respectively.

[0112] The loss function (L) of YOLOv5 is the confidence loss ( ), classification loss ( ), bounding box loss ( The weighted sum of the three parts:

[0113]

[0114] Confidence loss and classification loss are defined using binary cross-entropy, as shown in the following formula:

[0115]

[0116]

[0117] The bounding box loss is calculated using CIOU, as shown in the following formula:

[0118]

[0119] Where IOU is the intersection-union ratio of the predicted bounding box and the ground truth, and b and These are the coordinates of the center point of the predicted bounding box and the center point of the ground truth bounding box, respectively. and Here, represents the width and height of the ground truth bounding box, w and h represent the width and height of the predicted bounding box, v is the aspect ratio consistency parameter, and α is the balance parameter. Their expressions are as follows:

[0120]

[0121]

[0122] For an improved version of YOLOv5 that introduces a channel attention mechanism, see [link to YOLOv5]. Figure 2 The channel attention mechanism adds weights to each channel and updates these weights during backpropagation, allowing the differences between channels to be noticed. This channel attention mechanism module is abbreviated as the SE module. In this model, the SE module is placed in two locations. Existing tests show that the model with the channel attention mechanism exhibits significantly improved prediction robustness. During training, images are fed into the improved YOLOv5 network model, undergo KNN clustering preprocessing, feature extraction in the backbone network, and finally, prediction at three scales in the head region. The prediction results are calculated using a loss function, the gradient is returned, and the network weights are updated. During prediction, the image first undergoes KNN clustering preprocessing, is divided into three scales in the head region, and is finally fed into the backbone network for multi-scale coordinate and category prediction.

[0123] The spectral information collected by the near-infrared sensor is transmitted into the principal component analysis (PCA) and support vector machine (SVM) models. PCA is responsible for data dimensionality reduction, while SVM is responsible for predicting the target category.

[0124] Preprocessing: This experiment uses SG filtering for smoothing (7 windows, 2 polynomials), linear detrending for baseline correction, and standard normal transformation (SNV) for standardization.

[0125] PCA data dimensionality reduction: This is achieved using the Python function sklearn.decomposition.PCA. Setting the number of principal components to 6-8 is optimal, with the specific choice depending on the scenario.

[0126] SVM class prediction: Implemented using the Python function sklearn.svm.SVC.

[0127] The target category identified by the improved YOLOv5 is coupled with the object category obtained by the spectral method in step 1, as specifically implemented as follows:

[0128] The categories predicted by the YOLOv5 network model are jointly determined with those predicted by the spectral method. This is a cautious parallel approach, as illustrated in the following example:

[0129] Example 1: If the YOLOv5 prediction result is "plastic bottle" and the spectral prediction result is "PP", then the target is identified as a PP plastic bottle.

[0130] Example 2: YOLOv5 predicts that the target is an aluminum can, and the spectral prediction result is PP. Obviously, the aluminum can should be made of metal, so it is judged as invalid. This target is not classified and flows to the back end of the conveyor belt for manual sorting or is recycled back to the starting point for re-judgment.

[0131] The improved YOLOv5-generated 2D object coordinates are coupled with the 3D coordinates obtained from the laser sensor in step 1. This step uses the 3D coordinates obtained from the laser sensor as a reference, while the 2D coordinates obtained by the target recognition algorithm are mainly used to verify the consistency of the object. That is, if the x and y coordinates of the 3D coordinates obtained by the laser differ significantly from the coordinates given by YOLOv5, the process is skipped or an error is reported; otherwise, the 3D coordinates are transmitted to the robotic arm gripper.

[0132] The target category obtained through coupling is transmitted to the parallel robotic arm. The coupled coordinates are transmitted to the PLC, and the PLC's servo control module performs positioning. Communication between the industrial computer and the PLC is achieved via UDP Ethernet communication. Transmission between the PLC and the frequency converter is achieved via CC-Link connection. The parallel robotic arm, guided by the controller, performs grasping and selects the placement position according to the target category.

[0133] It should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention and are not intended to limit it. Although the present invention has been described in detail with reference to preferred embodiments, those skilled in the art should understand that modifications or equivalent substitutions can be made to the technical solutions of the present invention without departing from the spirit and scope of the technical solutions of the present invention, and all such modifications or substitutions should be covered within the scope of the claims of the present invention.

Claims

1. A machine vision-based waste identification and location technology based on sensor coupling, characterized in that: include, The sensor collects object image information, infrared spectrum information, and 3D contour information; Image information is fed into the YOLOv5 target recognition network to establish a dataset type; the YOLOv5 target recognition network is improved by introducing a channel attention mechanism, which adds weight to each channel and updates the weight during backpropagation. Infrared spectral information is transmitted to principal component analysis (PCA) and support vector machine (SVM) models to establish dataset types. Couple the target categories identified by YOLOv5 with the object categories identified by infrared spectroscopy; The two-dimensional coordinates of the object obtained by YOLOv5 are coupled with the three-dimensional coordinates obtained by the sensor. The three-dimensional coordinates obtained by the laser sensor are used as the reference. The two-dimensional coordinates obtained by the target recognition algorithm are mainly used to verify the consistency of the object. That is, if the x and y coordinates of the three-dimensional coordinates obtained by the laser are significantly different from the coordinates given by YOLOv5, the algorithm skips or reports an error; if the difference is not significant, the three-dimensional coordinates are transmitted to the robotic arm gripper. The target category obtained through coupling is transmitted to the parallel robotic arm; The coordinates obtained from the coupling are transmitted to the PLC, and the positioning is performed by the PLC's servo control module. The parallel robotic arm is guided by a controller to grasp objects and select the placement location based on the type of the target.

2. The machine vision-based waste identification and positioning technology based on sensor coupling as described in claim 1, characterized in that: Sensor calibration involves four planar coordinate systems: pixel plane coordinate system, pixel plane coordinate system, and pixel plane coordinate system. Image physical coordinate system Camera coordinate system and world coordinate system Pixel coordinates The image plane coordinates are derived using the following formula: in, , , , All of these are set parameters. , Indicates the actual pixel on the image sensor Size is the coordinate system that connects the pixel coordinate system and the actual size coordinate system; , The image plane center is used to ultimately obtain intrinsic and extrinsic parameters; a pixel plane coordinate system is established. When using a checkerboard pattern as the calibration object, the corresponding values ​​of the checkerboard side length and the number of pixels of the corresponding side in the image are calculated during the calibration process to complete the establishment of the coordinate system.

3. The machine vision-based waste identification and positioning technology based on sensor coupling as described in claim 2, characterized in that: The formula can be converted into matrix form as follows: The camera coordinate system is the world coordinate system after rotation and translation. The following relationship is obtained through the rotation matrix R and the translation matrix T: Based on the camera imaging principle, the relationship between the side lengths of similar triangles is obtained: The final formula is as follows: 。 4. The machine vision-based waste identification and positioning technology based on sensor coupling as described in claim 3, characterized in that: In images captured by a camera, straight lines at the edges may be distorted into curves; the mathematical model for radial distortion is: In the formula, , , Indicates the radial distortion coefficients of each order; These are ideal, distortion-free coordinates; These are the coordinates of the pixels in the distorted image; This represents the distance between the target coordinates and the origin, i.e. ; The mathematical model for tangential distortion is: In the formula , Indicates the tangential distortion coefficients of each order; This represents the distance between the target coordinates and the origin, i.e. ; merge: Five distortion parameters were finally obtained. , , , , By calibrating the intrinsic and distortion parameters of the industrial camera, distortion correction processing is performed on the acquired images to obtain the corrected images.

5. The machine vision-based waste identification and positioning technology based on sensor coupling as described in claim 4, characterized in that: The obtained reflected signal is corrected using a reference spectrum. First, whiteboard sampling is performed at a probe height h=1m. A whole white ceramic tile is laid under the probe for continuous sampling. Then, the probe is wrapped with black cotton cloth, and blackboard data is collected. Spectral correction is then performed using the following formula: In the formula, The signal strength is the signal intensity after correction to the reference spectrum at the i-th wavelength. The signal strength of the i-th wavelength of the original reflected signal; Let be the signal intensity at the i-th wavelength of the diffuse reflectance spectrum of the whiteboard; Let be the signal intensity of the dark current background spectrum at the i-th wavelength.

6. The machine vision-based waste identification and positioning technology based on sensor coupling as described in claim 5, characterized in that: The YOLOv5 object detection model first obtains anchor boxes through K-nearest neighbor (KNN) clustering, and then predicts the accurate object boxes through object box regression. YOLOv5 uses the following formula: in, , , , These are all parameters that need to be iterated during the backpropagation process. It is the sigmoid activation function. , The center of the prediction box coordinate, and These represent the length and width of a single grid, respectively. , To predict the width and length of the bounding box, , The widths of the anchor boxes obtained from clustering are respectively Degree and length; The loss function L in YOLOv5 is the confidence loss. Classification loss Bounding box loss Weighted sum of three parts: Confidence loss and classification loss are defined using binary cross-entropy, as shown in the following formula: The bounding box loss is calculated using CIOU, as shown in the following formula: in, The intersection-union ratio (IoU) of the predicted bounding box and the ground truth. and These are the coordinates of the center point of the predicted bounding box and the center point of the ground truth bounding box, respectively. and These are the width and height of the ground truth bounding box. and This represents the width and height of the predicted bounding box. It is a parameter for aspect ratio consistency. These are the balancing parameters, and their expressions are as follows: During training, after the image is fed into the improved YOLOv5 network model, it first undergoes KNN clustering preprocessing, then feature extraction is performed in the backbone network, and finally prediction is performed at three scales in the head part. The prediction results are calculated using the loss function, the gradient is returned, and the network weights are updated. During the prediction process, the image first undergoes KNN clustering preprocessing, is divided into three scales in the head, and finally is fed into the backbone network for multi-scale coordinate and category prediction.

7. The machine vision-based waste identification and positioning technology based on sensor coupling as described in claim 6, characterized in that: Before transmitting spectral information to the PCA and SVM models, preprocessing is performed. SG filtering is used for smoothing with a window size of 7 and a polynomial order of 2. Linear detrending is used for baseline correction, and standard normal transformation (SNV) is used for standardization. Then, PCA data dimensionality reduction is performed using the Python function sklearn.decomposition.PCA, with 6-8 principal components being optimal, depending on the scenario. Finally, SVM class prediction is performed using the Python function sklearn.svm.SVC.

8. The machine vision-based waste identification and positioning technology based on sensor coupling as described in claim 7, characterized in that: The target category and the object category are coupled, and the category predicted by the YOLOv5 network model is jointly judged with the category predicted by the spectral method. Specifically, it is a cautious parallel mode.

9. The machine vision-based waste identification and positioning technology based on sensor coupling as described in claim 8, characterized in that: Communication between the industrial computer and the PLC is achieved via UDP Ethernet communication; transmission between the PLC and the frequency converter is achieved via CC-Link connection.

Images