Spraying and sucking integrated cotton impurity detection and removal method and system based on machine vision

Abstract

The invention discloses a spraying and sucking integrated cotton impurity detection and removal method and system based on machine vision, and the method and system achieve the precise positioning and spraying and sucking cooperative removal of impurities through multispectral and RGB image fusion, deep learning feature extraction and dynamic trajectory prediction. The feature fusion effect is improved by adopting a channel attention mechanism, and the system delay is compensated in real time in combination with the FPGA, so that the injection action and the impurity position are ensured to be synchronous. The cleaning efficiency of complex impurities is remarkably improved, damage to cotton fibers is reduced, and the method is suitable for automatic upgrading of cotton processing.

Description

Technical Field

[0001] The present invention relates to the technical field of machine vision, and particularly to a method and system for detecting and removing cotton impurities by an integrated spraying and suction method based on machine vision. Background Art

[0002] During the processes of cotton picking, transportation, and primary processing, various impurities are often mixed in, including soil, stems and leaves, seed husks, small stones, plastic fragments, etc. These impurities not only reduce the quality of cotton, but may also cause damage to equipment during processing and even affect the textile performance of subsequent products. Therefore, quickly and accurately detecting and removing impurities in cotton has become a key technical requirement in the cotton processing industry.

[0003] Traditional cotton impurity removal methods mainly rely on mechanical screening and manual picking, but these methods have obvious limitations: Mechanical screening mainly separates based on the physical properties of impurities (such as density, size, etc.), and it is difficult to handle impurities with irregular shapes or characteristics similar to cotton, which results in low detection accuracy; The manual picking method has a too high labor intensity, is time-consuming and laborious, and its efficiency cannot meet the requirements in large-scale cotton processing; At the same time, the mechanical cleaning process may cause damage to cotton fibers, increase raw material losses, and affect product quality, and current cleaning equipment has poor flexibility in dealing with various types of complex impurities and is difficult to efficiently handle different types of impurities. Therefore, machine vision technology has been gradually applied to cotton impurity detection, but there are still the following technical bottlenecks: (1) Single-modal limitations: RGB cameras are easily interfered by changes in light and cannot distinguish impurities with similar colors; Multispectral cameras have low resolution and it is difficult to locate tiny impurities; (2) Dynamic synchronization problem: The high-speed transmission of cotton causes the real-time offset of impurity positions, and traditional control algorithms are difficult to achieve millisecond-level synchronization between the spraying and suction device and the target position; (3) Insufficient feature fusion: Existing methods simply stitch multi-modal features without considering the complementarity of spectral and texture information, resulting in poor robustness of the detection model. Summary of the Invention

[0004] The purpose of the present invention is to provide a method and system for detecting and removing cotton impurities by an integrated spraying and suction method based on machine vision in view of the deficiencies of the existing technology.

[0005] Technical Solution: The technical solution adopted by the present invention to solve the problem is to provide a method for detecting and removing cotton impurities by an integrated spraying and suction method based on machine vision. RGB cameras and multispectral cameras on both sides of the detection area input images into the image processing and analysis module. Among them, the RGB camera is used to obtain high-resolution images of the cotton surface, and the multispectral camera captures the reflection characteristics under different spectra, and comprehensively analyzes the characteristic differences between cotton and impurities (such as plastic film, leaves). And trigger the integrated spraying and suction device according to the impurity position to accurately remove the impurities. The method includes the following steps:

[0006] S1: Transport the cotton to the detection area through the fan conveying system;

[0007] S2: Synchronously collect image data using an RGB camera and a multispectral camera;

[0008] S3: Transmit the image data to the image processing and analysis module for image preprocessing;

[0009] S4: Adopt a multi-modal fusion deep learning algorithm to extract the features of the two types of images respectively, and use the feature fusion module to generate a unified feature map;

[0010] S5: Based on the deep learning detection algorithm Faster R-CNN for the fused features, quickly locate the position of impurities in the image and mark them with bounding boxes;

[0011] S6: Utilize the high-speed transmission characteristics, combine the image acquisition frequency and the impurity detection results, use the FPGA algorithm to calculate the position trajectories of the cotton and impurities in real time, and adopt a dynamic tracking algorithm to achieve spray-suction collaborative control and delay compensation;

[0012] S7: Use a spraying device to quickly remove the lighter impurities with high-pressure air flow, and an adsorption device to accurately remove the heavier or strongly adherent impurities, monitor the removal effect in real time, feedback the residual impurity detection information to the control system, and dynamically adjust the spray-suction parameters.

[0013] Further, in S3, geometric alignment is performed on the RGB image and the multispectral image, the pixel points are made to correspond one by one, a polynomial model is used for lens distortion correction, and the internal and external parameters of the camera are calibrated using a calibration board; the image is smoothed by a Gaussian filter, the multispectral data is processed using median filtering, the illumination of the RGB and multispectral images is equalized using the histogram equalization algorithm, the multispectral data is normalized, and the band intensity is adjusted to the range of 0-1; the principal component analysis method is used to concentrate the main information in the multispectral data into a few principal components.

[0014] Further, in S4, it further includes:

[0015] RGB feature extraction: Adopt a lightweight ResNet50 network as the backbone network, combine with a multi-scale feature pyramid structure (FPN), and fuse the feature maps of different resolutions layer by layer through the following formula:

[0016] P i =Conv(Upsample(P i+1 )+Conv(C i ))

[0017] where, P i+1 is the high-level feature map, C iis the feature map of the current layer, Upsample represents the bilinear upsampling operation, and Conv represents the 1×1 convolution to adjust the number of channels;

[0018] Spectral feature extraction: A 3D convolutional neural network is adopted. The convolutional kernel slides in the spectral dimension to capture the spectral differences in different bands, and the multi-band information is aggregated through the fully connected layer of the network to generate a comprehensive spectral feature representation of cotton and impurities.

[0019] Furthermore, the specific implementation of feature fusion in S4 includes:

[0020] Feature concatenation and dimensionality reduction: The RGB feature map and the spectral feature map are concatenated channel by channel according to the pixel positions to form a high-dimensional joint feature tensor, and the channel dimension is compressed through a 1×1 convolutional layer;

[0021] Channel attention weighting: The SE module is used to assign channel weights to the fused feature map, and the calculation method is:

[0022]

[0023] where z c is the global pooling feature of the c-th channel, H and W are the spatial dimensions of the feature map, and X c,i,j is the value of the feature map at the position (i, j);

[0024] Cross-attention correlation: The correlation relationship between RGB and spectral features is established through the cross-attention mechanism. The attention weights are used to guide the spectral features to focus on the key regions of the RGB features, and at the same time, the key band information of the spectral features is used to adjust the attention points of the RGB features in the reverse direction, and finally a unified multi-modal feature map is generated. Furthermore, the specific implementation of the suction and spraying collaborative control and delay compensation in S6 includes:

[0025] Total delay time calculation: The full-link delay of the system is quantified by the following formula:

[0026] T total = T image + T processing + T actuation

[0027] In the formula, T image represents the image acquisition time, T processing represents the image processing and trajectory prediction time, and T actuation represents the signal transmission and action execution time;

[0028] Trigger timing correction: According to the predicted impurity position xpredxpred and the total delay time, the trigger moment of the suction and spraying device is adjusted, and the formula is as follows:

[0029]

[0030] Suction and spraying synchronous control:

[0031] Time synchronization: A synchronization signal is generated by the FPGA, and the trigger interval between the spraying valve and the suction valve is ≤ 3 ms;

[0032] Spatial synchronization: Based on the servo motor, the spatial positions of the nozzle and the suction nozzle are adjusted in real time to align them with the center point of the impurity;

[0033] Communication protocol and interface: A high-speed LVDS interface is used to transmit trigger signals, action intensity signals, and feedback signals. The clocks of each module are aligned through the timestamp mechanism to ensure that the suction and spraying actions are synchronously executed within milliseconds.

[0034] A suction and spraying integrated cotton impurity detection and removal system based on machine vision according to the present invention is used to implement the suction and spraying integrated cotton impurity detection and removal method as described above, and includes a housing, an image acquisition module, an image processing and analysis module, a suction and spraying integrated removal device, and a human-machine interaction interface; the image acquisition module includes: an RGB camera fixed to the first camera bracket, a multispectral camera fixed to the second camera bracket, an RGB camera fixed to the third camera bracket, and a multispectral camera fixed to the fourth camera bracket; several LED light sources and several camera brackets are alternately installed on the inner top surface of the housing; the image acquisition module and the cotton conveying module are located inside the housing, and the image processing and analysis module is installed on the side of the housing; the cotton conveying module includes a black rubber pipe; the image processing and analysis module further includes an industrial computer, and the industrial computer is electrically connected to the RGB camera, the multispectral camera, the RGB camera, the multispectral camera, and several LED light sources.

[0035] Further, the housing is made of opaque aluminum alloy material, and the interior is coated with black paint with high absorption rate. The LED light sources are respectively fixed on the upper right side and the lower left side of the camera on the camera bracket, and are installed by rotating reversely about 5 degrees respectively in the direction perpendicular to the black rubber pipe with the camera as the center.

[0036] Beneficial effects: Compared with the prior art, the present invention has the following advantages:

[0037] (1) The present invention combines an RGB camera and a multispectral camera, which can simultaneously obtain the color, texture, and spectral information of cotton and impurities, effectively solve the detection limitations of complex backgrounds or specific impurities (such as plastic films) in a single modality, fuse RGB and multispectral features through deep learning, introduce channel attention and cross-attention mechanisms, and realize the effective integration of multimodal information, greatly improving the detection accuracy and robustness for different types of impurities;

[0038] (2) The present invention applies the optimized Faster R-CNN detection algorithm, combines a high-quality region proposal network and a multi-task learning framework, can quickly locate impurity targets and generate accurate bounding boxes, and has both detection speed and accuracy. The deep learning model adopts a multi-scale feature extraction and lightweight optimization strategy to reduce the computational burden, can be adapted to low-power embedded hardware devices, and reduces the system operation cost;

[0039] (3) The present invention utilizes the Kalman filter and acceleration compensation algorithm, combines a system delay correction mechanism, realizes accurate trajectory prediction of fast-moving impurities, ensures that the actions of the suction and spraying device are synchronized with the target position, and greatly improves the removal success rate. Through fully automated machine vision detection and suction and spraying linkage control, the manual participation is reduced, the operation difficulty and labor cost are lowered, and at the same time, the work efficiency and stability are improved. BRIEF DESCRIPTION OF THE DRAWINGS

[0040] Figure 1 is the algorithm flow chart of the embedded FPGA of the present invention;

[0041] Figure 2 is the overall structure schematic diagram of the present invention;

[0042] Figure 3 is the internal structure schematic diagram of the present invention. DETAILED DESCRIPTION OF THE EMBODIMENTS

[0043] The present invention will be further clarified below in conjunction with the drawings and specific embodiments. These embodiments are implemented on the premise of the technical solution of the present invention. It should be understood that these embodiments are only used to illustrate the present invention and not to limit the scope of the present invention.

[0044] As shown in the figure, a suction-jet integrated cotton impurity detection and removal system based on machine vision in this embodiment includes a cotton conveying module, an image acquisition module, an image processing and analysis module, a suction-jet integrated removal device, and a human-machine interaction interface; first LED light sources 41, second LED light sources 42, third LED light sources 43, fourth LED light sources 44, fifth LED light sources 45, sixth LED light sources 46, seventh LED light sources 47, and eighth LED light sources 48 are installed on both sides of the internal pipeline of the housing 1; the image acquisition module and the cotton conveying module are located inside the housing 1, and the image processing and analysis module is installed on the side of the housing. The image acquisition module includes: an RGB camera 51 fixed to the first camera bracket 61, a multispectral camera 52 fixed to the second camera bracket 62, an RGB camera 53 fixed to the third camera bracket 63; a multispectral camera 54 fixed to the fourth camera bracket 64; the cotton conveying module includes a black pipeline 2; an industrial computer 3 of the image processing and analysis module, and this industrial computer 3 is connected to the RGB camera 51, the multispectral camera 52, the RGB camera 53, the multispectral camera 54, the first LED light source 41, the second LED light source 42, the third LED light source 43, the fourth LED light source 44, the fifth LED light source 45, the sixth LED light source 46, the seventh LED light source 47, and the eighth LED light source 48. It also includes a cotton conveying channel 2, an automatic spray valve module 6, and an automatic suction valve module 7.

[0045] The first LED light source 41 is installed on the upper right side of the camera bracket 61 at an angle of 15 degrees to the vertical direction of the pipeline center; the third LED light source 43 is installed on the upper right side of the camera bracket 62 at an angle of 15 degrees to the vertical direction of the pipeline center; the second LED light source 42 is installed on the lower left side of the camera bracket 61 at an angle of 165 degrees to the vertical direction of the pipeline center; the fourth LED light source 44 is installed on the lower left side of the camera bracket 62 at an angle of 165 degrees to the vertical direction of the pipeline center; the fifth LED light source 45 is installed on the upper right side of the camera bracket 63 at an angle of 15 degrees to the vertical direction of the pipeline center; the seventh LED light source 47 is installed on the upper right side of the camera bracket 64 at an angle of 15 degrees to the vertical direction of the pipeline center; the sixth LED light source 46 is installed on the lower left side of the camera bracket 63 at an angle of 165 degrees to the vertical direction of the pipeline center; the eighth LED light source 48 is installed on the lower left side of the camera bracket 64 at an angle of 165 degrees to the vertical direction of the pipeline center;

[0046] The overall housing 1 is made of aluminum, which is convenient for the internal camera equipment to dissipate heat while isolating external light and interference signals. The black pipeline 2 used for the cotton conveying channel is made of rubber material, which can absorb and reduce the interference of external light sources on the inside of the pipeline, and a diffuser plate is installed on the top of the pipeline to enhance the light diffusion effect.

[0047] The two sides of the black pipeline 2 are provided with air holes, which can prevent the interior of the housing 1 from vibrating violently due to excessive wind speed. At the same time, the camera brackets 61, 62, 63, and 64 are not directly welded to the housing 1 to reduce the impact of vibration on image acquisition.

[0048] The embedded computing module is responsible for performing key tasks such as image processing, trajectory prediction, latency correction, and control signal generation.

[0049] The above device is used to implement the following method for detecting and removing cotton impurities by integrated spraying and suction based on machine vision. The method specifically includes the following steps:

[0050] S1: Convey cotton along the pipeline at high speed to the detection area through the fan conveying system.

[0051] S2: Use an RGB camera and a multispectral camera to synchronously capture cotton images.

[0052] S3: Transmit the collected image data to the image processing and analysis module for image preprocessing

[0053] S4: Adopt a multimodal fusion deep learning algorithm to extract the features of the two types of images respectively, and use the feature fusion module to generate a unified feature map.

[0054] S5: Based on the deep learning detection algorithm Faster R-CNN for fused features, quickly locate the position of impurities in the image and mark them with bounding boxes.

[0055] S6: Utilize the high-speed transmission characteristics, combine the image acquisition frequency and the impurity detection results, and use the FPGA algorithm to calculate the position trajectories of cotton and impurities in real time. The dynamic tracking algorithm ensures that the spraying and suction devices execute accurately and synchronously.

[0056] S7: The spraying device uses high-pressure air flow to quickly remove the lighter impurities; the adsorption device precisely removes the heavier or strongly adherent impurities. Monitor the removal effect in real time, feedback the residual impurity detection information to the control system, adjust the spraying and suction parameters (such as jet pressure, adsorption strength), optimize the system performance and improve the removal efficiency.

[0057] In step S1, "Convey cotton along the pipeline at high speed to the detection area through the fan conveying system.", the specific implementation method is as follows:

[0058] S11: The fan conveying system consists of a high-speed fan, a conveying pipeline, a detection window, etc., and pushes the cotton to move forward at high speed in a suspended state in the pipeline through high-pressure air flow.

[0059] In step S2, the statement "Use an RGB camera and a multispectral camera to synchronously capture cotton images.", the specific implementation method is as follows:

[0060] S21: Use LabVIEW to control the RGB camera and the hyperspectral camera simultaneously, and collect target cotton images in an external trigger mode.

[0061] In step S3, "Transmit the collected image data to the image processing and analysis module for image preprocessing", and the specific implementation method is as follows:

[0062] S31: Geometrically align the RGB image and the hyperspectral image to ensure one-to-one correspondence of pixel points, perform lens distortion correction using a polynomial model, and complete the calibration of the internal and external parameters of the camera using a calibration board.

[0063] S32: Smooth the image through a Gaussian filter, process the hyperspectral data using median filtering, use the histogram equalization algorithm to solve the problem of uneven illumination of the RGB and hyperspectral images, perform normalization processing on the hyperspectral data, and adjust the band intensity to the range of 0-1. Using the principal component analysis (PCA) method, concentrate the main information in the hyperspectral data into a few principal components.

[0064] In step S4, "Use the multi-modal fusion deep learning algorithm to extract the features of the two types of images respectively, and use the feature fusion module to generate a unified feature map", and the specific implementation method is as follows:

[0065] S41: Use the convolutional neural network CNN to extract the color and texture features of the RGB image. The network structure selects the lightweight ResNet50, and then introduces the multi-scale feature pyramid structure FPN to extract features at different resolutions. The formula is as follows:

[0066] P i =Conv(Upsample(P i+1 )+Conv(C i ))

[0067] In the formula, Upsample(P i+1 ) means upsampling the higher-level feature P i+1 to the resolution of the current scale C i , Conv(C i ) means adjusting the channels of the current layer feature map C i through a 1×1 convolution, and Conv is used to fuse the features from the upper layer and the features of the current layer.

[0068] S42: Use a 3D convolutional neural network to extract spectral features. Use the convolutional kernel to slide in the spectral dimension to capture the spectral differences of different bands, and aggregate the multi-band information through the fully connected layer of the network to generate a comprehensive spectral feature representation of cotton and impurities.

[0069] S43: Concatenate the extracted RGB and spectral features pixel by pixel to form a high-dimensional joint feature tensor. In the concatenated feature map, use 1×1 convolution to reduce the feature dimension. During the fusion process, introduce the channel attention mechanism SE module to assign weights to each feature channel. The formula is as follows:

[0070]

[0071] S44: Use the cross-attention mechanism to establish an association between the RGB and multispectral features, respond to the important regions in the RGB features, and guide the multispectral features to focus on the corresponding regions. In the reverse direction, adjust the focus of the RGB features through the key band information in the spectral features.

[0072] S45: Generate a unified multimodal feature map, retaining the multi-dimensional characteristics such as the color, texture, and spectrum of cotton and impurities, and providing input for subsequent object detection and classification.

[0073] The "deep learning detection algorithm Faster R-CNN based on fused features" described in step S5, which quickly locates the position of impurities in the image and marks them with bounding boxes, is specifically implemented as follows:

[0074] S51: Label the category of each detection target, output the coordinates of the upper left and lower right corners of each detection target, and define its position range. Attach a confidence score to each bounding box, indicating the likelihood that the target belongs to this category. Display the detection results superimposed on the original image through the visualization interface for easy manual monitoring and verification.

[0075] The "using the high-speed transmission characteristics, combining the image acquisition frequency and the impurity detection results, and using the FPGA algorithm to calculate the position trajectories of cotton and impurities in real time, and the dynamic tracking algorithm ensures accurate synchronous execution of the suction and spraying device" described in step S6 is specifically implemented as follows:

[0076] S61: Combine the camera frame rate and the detection results of S5, map the impurity positions in the image to the actual space coordinates, and use the calibration parameters to achieve the conversion from pixel coordinates to actual physical coordinates. Align with the system's time synchronization module by adding a timestamp to each frame of image data.

[0077] S62: Send the bounding boxes, target categories, and confidences output by Faster R-CNN to the control module through a high-speed communication interface (such as Ethernet or CAN bus). Perform real-time compensation for the time delay in data transmission.

[0078] S63: Combine with the conveyor belt speed to estimate the relative movement speed and direction of the detected cotton and impurities in real time. Use the Kalman filter algorithm to predict the trajectory of the target, and combine the position and speed of the previous frame to estimate the accurate position of the target at the next moment. Introduce an acceleration compensation model for fast-moving impurities, and use the Hungarian algorithm to achieve the matching and tracking of cotton and impurity targets based on the detected bounding box coordinates. When the target is lost in a certain frame of the image, restore its position based on trajectory prediction.

[0079] S64: Match the predicted impurity trajectory with the execution parameters of the suction and spraying device (such as nozzle position and suction range) to generate a real-time synchronization signal. According to the trajectory prediction result, control the suction and spraying device to accurately trigger the removal action when the impurity reaches the specified position. Quantify the entire process delay from image acquisition to suction and spraying execution, and adjust the trigger timing in real time. Compensate for the hardware execution error by dynamically adjusting the response time of the suction and spraying device.

[0080] After the image is preprocessed, use an embedded GPU to run Faster R-CNN for impurity detection, classification, and position localization. The FPGA is used for low-latency image preprocessing, Kalman filtering, and acceleration of the trajectory prediction algorithm. Based on the detected impurity position, perform Kalman filtering through the FPGA, input the predicted position and speed of the previous frame, combine with the current observation value, output the optimized current position estimate, and at the same time combine the acceleration compensation formula to predict the future trajectory position of the impurity, and then adjust the trajectory prediction result according to the system delay.

[0081] The integrated suction and spraying device realizes the precise removal of cotton impurities through the combined actions of compressed gas spraying and strong adsorption. The core components of the spraying module are high-speed solenoid valves and multi-nozzle arrays. The high-speed solenoid valves are used to control the opening and closing of the nozzles, and the response time is less than 5 ms. The multi-nozzle array enables each nozzle to be independently controlled to form a multi-point spraying ability. The core components of the adsorption module are high-speed fans and adjustable suction nozzles. The high-speed fans generate negative pressure airflows to adsorb impurities, and the suction nozzles are equipped with flexible robotic arms or track systems to be synchronized with the impurity position in real time. The nozzles and suction nozzles are arranged in a coaxial structure, and the directions of the spraying airflow and the adsorption airflow are the same. The actions of the nozzles and suction nozzles are driven by the trajectory prediction data, and the embedded module generates trigger signals. The synchronous signal embedded module uses a high-speed communication protocol to synchronously control the actions of the nozzles and suction nozzles. The nozzles and suction nozzles are respectively driven by independent servo controllers, but are synchronously controlled by the central control module. The spraying action of the nozzles is triggered first, and the adsorption action of the suction nozzles follows immediately, with an interval time less than 5 ms.

[0082] The FPGA receives data from the trajectory prediction module, including the impurity center position (x, y), the impurity movement speeds vx, vy, and the predicted positions (xpred, ypred).

[0083] The triggering logic of the spray valve is as follows: The triggering condition is that when the impurity position enters the spray valve triggering area, the FPGA sends an opening signal to the solenoid valve. The formula is as follows:

[0084]

[0085] The calculation formula for the spray air flow intensity Pspray is as follows:

[0086] Pspray = f(impurity weight, impurity area)

[0087] The calculation formula for the spray duration Tspray is as follows:

[0088]

[0089] The triggering logic of the suction valve is as follows: When the impurity position enters the suction valve triggering area, the FPGA sends an opening signal to the solenoid valve of the suction nozzle or the fan control module. The formula is as follows:

[0090]

[0091] The calculation formula for the suction force magnitude Fsunction is as follows:

[0092] Fsunction = f(impurity weight, impurity form)

[0093] The delay compensation is based on the image acquisition and processing time τprocess, and the system execution time τexecute, to predict the time tpred when the impurity reaches the triggering area. The formula is as follows:

[0094]

[0095] By adjusting the time point of the trigger signal, ensure that the actions of the spray valve and the suction valve are synchronized with the impurity.

[0096] According to the calculated triggering conditions and parameters, the FPGA generates control signals. The control signal formula for the spray valve is as follows:

[0097] Signal_spray = {Tstart, Tend, Pspray}

[0098] The signal directly drives the solenoid valve through PWM (pulse width modulation) or digital level. The control signal formula for the suction valve is as follows:

[0099] Signal_sunction = {Tstart, Tend, Fsuction}

[0100] The signal is used to control the opening and closing state of the suction nozzle and the suction force intensity of the fan.

[0101] The synchronous control of the spray valve and the suction valve is divided into time synchronization and space synchronization. The actions of the spray valve and the suction valve need to be time-synchronized according to the actual movement trajectory of the impurities and the system delay. The total delay time needs to be calculated, and the formula is as follows:

[0102] T total = T image + T processing + T actuation

[0103] In the formula, T image represents the image acquisition time. T processing represents the image processing and trajectory prediction time. T actuation represents the signal transmission and action execution time.

[0104] Then, adjust the trigger signal according to the delay time to synchronize the actions of the spray valve and the suction valve to the actual position of the impurities. The calculation formula for the trigger time after compensation is as follows:

[0105]

[0106] The FPGA adjusts the action position of the spraying and suction device according to the real-time trajectory prediction result, so that the nozzle and the suction nozzle are aligned with the center point of the impurities. The spatial position calibration is completed by controlling the position movement of the spraying and suction device with a servo motor.

[0107] The synchronization signal protocol is used to transmit control and feedback information between the FPGA, the drive circuit, and the spraying and suction device to ensure the coordinated operation of each part. The signal types designed in the protocol include trigger signals, action intensity signals, and feedback signals. The trigger signal mainly appears in the digital signal format and is used to indicate the opening and closing states of the spray valve and the suction valve; the action intensity signal mainly appears in the analog signal or PWM signal and is used to control the nozzle air pressure and the suction nozzle suction force; the feedback signal includes pressure values, switch states, etc., and is used to monitor the working states of the spray valve and the suction valve in real time. Use a high-speed LVDS interface to achieve efficient communication, and adopt a timestamp synchronization mechanism to ensure that the actions of the spray valve and the suction valve are consistent within milliseconds.

Claims

1. A method for detecting and removing impurities in a spray-suction integrated cotton machine based on machine vision, characterized in that: The following steps are involved: S1: The cotton is transported to the detection area through the fan conveying system; S2: Use RGB camera and multispectral camera to synchronously collect image data; S3: transmitting the image data to the image processing and analysis module for image preprocessing; S4: Use multimodal fusion deep learning algorithm to extract features of the two images respectively, and use feature fusion module to generate a unified feature map; S5: Faster R-CNN, a deep learning detection algorithm based on fusion features, quickly locates the position of impurities in the image and marks them with bounding boxes; S6: Taking advantage of the high-speed transmission characteristics, combined with the image acquisition frequency and impurity detection results, the FPGA algorithm is used to calculate the position trajectory of cotton and impurities in real time, and the dynamic tracking algorithm is used to achieve spray-suction coordinated control and delay compensation; S7: Use an injection device to quickly remove lighter impurities with high-pressure airflow, and an adsorption device to accurately remove heavier or strongly adherent impurities. Monitor the removal effect in real time, feed back the residual impurity detection information to the control system, and dynamically adjust the injection and suction parameters.

2. The method for detecting and removing impurities in cotton by spraying and sucking in one body based on machine vision according to claim 1, characterized in that: In S3, the RGB image and the multispectral image are geometrically aligned, the pixels are matched one by one, a polynomial model is used to correct the lens distortion, and a calibration plate is used to calibrate the internal and external parameters of the camera; The image is smoothed by a Gaussian filter, the multispectral data is processed using a median filter, the illumination of RGB and multispectral images is equalized using a histogram equalization algorithm, the multispectral data is normalized, and the band intensity is adjusted to the range of 0-1; the principal component analysis method is used to concentrate the main information in the multispectral data into a few principal components.

3. The method for detecting and removing impurities in cotton by spraying and sucking in one body based on machine vision according to claim 1, characterized in that: The S4 further comprises: RGB feature extraction: A lightweight ResNet50 network is used as the backbone network, combined with a multi-scale feature pyramid structure (FPN), and feature maps of different resolutions are fused layer by layer using the following formula: P i =Conv(Upsample(P i+1 )+Conv(C i )) Among them, P i+1 is the high-level feature map, C i is the feature map of the current layer, Upsample represents the bilinear upsampling operation, and Conv represents the 1×1 convolution to adjust the number of channels; Spectral feature extraction: A 3D convolutional neural network is used to slide the convolution kernel on the spectral dimension to capture the spectral differences of different bands. Multi-band information is aggregated through the fully connected layer of the network to generate a comprehensive spectral feature representation of cotton and impurities.

4. The method for detecting and removing impurities in cotton by spraying and sucking in an integrated manner based on machine vision according to claim 1 or 3, characterized in that: The specific implementation of feature fusion in S4 includes: Feature concatenation and dimensionality reduction: The RGB feature map and the spectral feature map are concatenated channel by channel according to the pixel position to form a high-dimensional joint feature tensor, and the channel dimension is compressed through a 1×1 convolution layer; Channel attention weighting: The SE module is used to assign channel weights to the fused feature map. The calculation method is: Among them, z c is the global pooling feature of the cc-th channel, H and W are the spatial dimensions of the feature map, X c,i,j is the value of the feature map at position (i, j); Cross-attention association: The association relationship between RGB and spectral features is established through the cross-attention mechanism, and the attention weight is used to guide the spectral features to focus on the key areas of the RGB features. At the same time, the focus of the RGB features is adjusted in reverse through the key band information of the spectral features, and finally a unified multimodal feature map is generated.

5. The method for detecting and removing impurities in cotton by spraying and sucking in one body based on machine vision according to claim 1, characterized in that: The specific implementation of the ejection-suction coordinated control and delay compensation in S6 includes: Total delay time calculation: The system full link delay is quantified by the following formula: T total =T image +T processing +T actuation Where, T image represents the image acquisition time, T processing represents the image processing and trajectory prediction time, T actuation Indicates signal transmission and action execution time; Trigger timing correction: According to the predicted impurity position xpredxpred and the total delay time, adjust the triggering time of the spray suction device. The formula is as follows: Synchronous control of injection and suction: Time synchronization: Generate synchronization signals through FPGA, and the trigger interval between the spray valve and the suction valve is ≤3ms; Spatial synchronization: Based on the servo motor, the spatial position of the nozzle and the suction nozzle is adjusted in real time to align them with the center point of the impurity; Communication protocol and interface: High-speed LVDS interface is used to transmit trigger signals, action intensity signals and feedback signals. The clocks of each module are aligned through the timestamp mechanism to ensure that the spraying and suction actions are executed synchronously within milliseconds.

6. A spray-suction integrated cotton impurity detection and removal system based on machine vision, characterized in that: Used to implement the method for detecting and removing cotton impurities based on machine vision by an integrated spray-suction type as claimed in any one of claims 1 to 5, comprising a housing (1), an image acquisition module, an image processing and analysis module, an integrated spray-suction type removal device, and a human-computer interaction interface; The image acquisition module comprises: an RGB camera (51) fixed to a first camera bracket (61), a multispectral camera (52) fixed to a second camera bracket (62), an RGB camera (53) fixed to a third camera bracket (63); and a multispectral camera (54) fixed to a fourth camera bracket (64). A plurality of LED light sources and a plurality of camera brackets are alternately mounted on the top surface of the inner part of the shell (1). The image acquisition module and the cotton transmission module are located inside the shell (1), and the image processing and analysis module is mounted on the side of the shell. The cotton transmission module comprises a black rubber pipe (2). The image processing and analysis module further comprises an industrial computer (3), and the industrial computer (3) is electrically connected to the RGB camera (51), the multispectral camera (52), the RGB camera (53), the multispectral camera (54), and the plurality of LED light sources.

7. The machine vision-based spray-suction integrated cotton impurity detection and removal system according to claim 6, characterized in that: The shell (1) is made of an opaque aluminum alloy material, and the interior is painted with a high-absorption black paint. The LED light sources are respectively fixed to the upper right side and the lower left side of the camera of the camera bracket, and are installed with the camera as the center and perpendicular to the direction of the black rubber pipe (2) and rotated about 5 degrees in the opposite direction.

Images