Nut automatic conveying machine operation control method and system based on visual detection

Abstract

This invention relates to the field of automated production equipment technology, and discloses a method and system for controlling the operation of an automatic nut conveyor based on vision detection. The method includes: acquiring image data and extracting position features, posture features, and sequence state features; comparing the position features, posture features, and sequence state features with preset benchmark values ​​in conjunction with position detection signals to determine whether the current conveying state is abnormal; executing corresponding adaptive adjustment actions according to the type of abnormal state; continuously acquiring image data and repeating the process of determining the current conveying state, and readjusting based on the difference between the re-detection result and the expected target until the conveying state returns to normal. This invention, through a closed-loop control method combining vision detection and adaptive adjustment, achieves intelligent identification and automatic elimination of various abnormal states during the automatic nut conveying process, thereby improving the stability and automation level of the conveying system.

Description

Technical Field

[0001] This invention relates to the field of automated production equipment technology, and in particular to a method and system for controlling the operation of an automatic nut conveyor based on vision detection. Background Technology

[0002] In highly automated welding production lines such as automobile manufacturing and metal structural component processing, automatic nut conveyors play a crucial role. Their core task is to replace tedious and error-prone manual operations by automatically, orderly, and accurately conveying uniformly sized welding nuts to the lower electrode positioning pins at the welding station, preparing them for subsequent projection welding. A typical automatic nut conveyor usually consists of a vibratory feeder sorting device, flexible or rigid conveying pipes, a pneumatic supply device at the end, and a central programmable logic controller (PLC) control system. The stability of this equipment's operation and the accuracy of its feeding directly determine the production cycle time of the welding station, the consistency of the final product quality, and the smoothness of the entire automation process.

[0003] However, the widely used automatic nut conveyors currently in use have inherent limitations in their control strategies. Their operating logic primarily relies on a set of pre-set mechanical structural parameters and fixed timing pneumatic logic. This control method is essentially an open-loop system, assuming that all nuts to be conveyed are completely identical and that the physical characteristics of the conveying environment remain constant. However, the actual situation in industrial settings is far more complex than this. The system lacks the ability to perceive the real-time status of the nuts during the conveying process, and it lacks the intelligence to adaptively adjust according to actual conditions. When faced with unavoidable disturbances such as minor differences in nut size, uneven thickness of the surface anti-rust oil film, or wear and tear on the conveying pipeline due to long-term operation, or changes in the friction coefficient caused by oil accumulation, this rigid control method frequently causes problems. Specifically, this manifests as nuts getting stuck or blocked at narrow or turning points in the conveying pipeline, positioning deviations at the final supply device outlet, and drastic fluctuations in conveying efficiency due to a mismatch between the upstream vibratory feeder's feeding rhythm and downstream demand. These anomalies not only cause unexpected production line shutdowns, requiring valuable time for manual intervention and troubleshooting, but more seriously, a nut that fails to be accurately delivered to the welding position will directly lead to welding failure or quality defects, introducing great uncertainty into the production process and significantly increasing maintenance costs and quality risks. Summary of the Invention

[0004] To address the shortcomings of existing technologies, this invention discloses a visual detection-based automatic nut conveyor operation control method and system, aiming to solve the technical problem that existing automatic nut conveyors are prone to conveying abnormalities when faced with changes in materials and environment due to the lack of real-time status perception and adaptive adjustment capabilities.

[0005] The technical solution of the present invention is as follows: In a first aspect, the present invention discloses a method for controlling the operation of an automatic nut conveyor based on vision detection, the method comprising: Acquire image data of the nuts to be transported along the transport path, and extract the position features, attitude features, and sequence state features of the nuts to be transported; By combining the position detection signals on the conveying path, the position features, attitude features and sequence state features are compared with preset reference values ​​to determine whether the current conveying state is abnormal. If the current delivery status is abnormal, perform the corresponding adaptive adjustment action aimed at eliminating the abnormal status according to the type of abnormal status; After performing adaptive adjustment, image data is continuously collected and the current delivery status is repeatedly determined. Adjustments are made again based on the difference between the re-detection results and the expected target until the delivery status returns to normal.

[0006] This technical solution constructs a closed-loop feedback control system based on visual perception, enabling the conveyor to see the nut status in real time and actively correct errors. This fundamentally changes the passive and rigid nature of the traditional open-loop control mode, greatly improving the stability and reliability of the conveying process.

[0007] Furthermore, the steps for acquiring image data of the nuts to be conveyed along the conveying path include: Raw images are acquired using industrial cameras and polarized light sources deployed at key nodes along the transport path; The original image was denoised using a polarization filtering algorithm to suppress the interference of oil stains and reflections on the surface of the nuts to be transported, and the image data was obtained.

[0008] Furthermore, abnormal states include blockage, positioning deviation, and abnormal feeding rhythm. The steps for performing appropriate adaptive adjustment actions to eliminate the abnormal state based on the type of abnormal state include: When the abnormal state type is a blockage state, an instantaneous fluid impact force is applied at the blockage location to release the blockage; When the abnormal state type is positioning deviation, the action parameters of the material discharge actuator are adjusted according to the amount of position or attitude deviation. When the abnormal state type is abnormal feeding rhythm, adjust the driving frequency or driving amplitude of the upstream feeding source.

[0009] Furthermore, the abnormal state also includes overlapping and stacking states, and the method further includes: Extract the contour envelope of the set of nuts to be transported from the image data, and calculate the complexity factor of the contour envelope; When the complexity factor exceeds the preset threshold and the centroid height of the set of nuts to be transported changes abruptly, it is determined that the nuts to be transported are in an overlapping and stacked state. The steps of performing corresponding adaptive adjustment actions aimed at eliminating the abnormal state based on the type of abnormal state also include: In response to the overlapping and stacking state, the upstream material supply source is controlled to switch from continuous vibration mode to discrete pulse vibration mode. The asymmetric vibration energy distribution is used to make the overlapping nuts to be conveyed complete the flat disassembly on the conveying path.

[0010] Furthermore, when the abnormal state type is a blockage state, the steps of applying a transient fluid impact force at the blockage location to relieve the blockage include: When the abnormal state type is a blockage state, the pneumatic device deployed at the blockage point is activated. A pneumatic device injects a pulse of instantaneous high-pressure airflow at a preset pressure into the obstruction location to blow away the stuck nut to be delivered.

[0011] Furthermore, the steps for adjusting the action parameters of the discharge actuator based on the deviation in position or attitude include: Analyze the reflective features of the surface of the nut to be transported in the image data to infer the oiliness level of the surface of the nut to be transported; Calculate the deviation between the positional and / or attitude characteristics and the preset reference values; First control information is generated based on the deviation and greasiness level, and the advance speed curve of the discharge actuator is adjusted according to the first control information. The inertial slippage of the nut to be conveyed during the advance process is suppressed by reducing the starting acceleration.

[0012] Furthermore, the steps for adjusting the feed speed curve of the discharge actuator based on the first control information include: The system acquires real-time temperature data of the current conveying environment and retrieves a preset viscosity-temperature mapping table based on the real-time temperature data to determine the rheological properties of the oil film on the surface of the nut to be conveyed. The starting acceleration in the first control information is corrected by combining the rheological characteristic parameters to obtain the compensated dynamic control parameters; The propulsion speed curve is reconstructed according to the compensated dynamic control parameters to offset the fluctuations in the adsorption force between the nut to be transported and the transport path caused by changes in ambient temperature.

[0013] Furthermore, the method also includes: Real-time monitoring of the current signal of the drive motor of the material discharge actuator; When a preset pulse signal appears in the current signal, it is determined that the nut to be delivered is physically obstructed; When it is determined that the nut to be delivered is physically obstructed, the current rigid pushing action is interrupted, and the micro-frequency oscillation logic is activated to control the drive motor to perform reciprocating motion, using the self-centering effect generated by the vibration to guide the nut to be delivered around the obstruction.

[0014] Furthermore, if the upstream material supply source is a vibratory feeder, and the abnormal state type is an abnormal feeding rhythm state, the steps for adjusting the driving frequency or driving amplitude of the upstream material supply source include: Based on the deviation between the current feeding rhythm and the preset rhythm, adjust the electromagnetic drive frequency of the vibratory feeder, or adjust the vibration voltage amplitude of the vibratory feeder, to change the moving speed of the nut to be conveyed.

[0015] Secondly, the present invention also discloses a vision-based automatic nut conveyor operation control system for performing the steps in any of the foregoing methods, the system comprising: The feature extraction module is used to acquire image data of the nuts to be transported in the transport path, and extract the position features, attitude features and sequence state features of the nuts to be transported; The status determination module is used to combine the position detection signals on the conveying path, compare the position features, attitude features and sequence status features with preset reference values, and determine whether the current conveying status is abnormal. The adjustment execution module is used to perform corresponding adaptive adjustment actions aimed at eliminating the abnormal state if the current delivery state is an abnormal state, based on the type of abnormal state. The closed-loop optimization module is used to continuously collect image data and repeatedly determine the current delivery status after performing adaptive adjustment actions. It then readjusts the data based on the difference between the re-detection results and the expected target until the delivery status returns to normal.

[0016] This technical solution provides a physical system architecture capable of implementing the aforementioned methods. By clearly defining functional modules, it offers a clear blueprint for the engineering implementation of the methods, making it possible to deploy intelligent control algorithms into stable and reliable industrial products.

[0017] In summary, this invention provides a vision-based automatic nut conveyor operation control method and system. The method deploys vision sensors at key nodes along the conveying path, enabling the system to acquire precise information in real time regarding the position, orientation, and arrangement of each nut to be conveyed. This real-time monitoring allows the system to detect abnormal states such as blockages, positioning deviations, and disordered feeding rhythms immediately. More importantly, this invention not only detects problems but also proactively resolves them. Based on predefined logic, it automatically executes corresponding optimized adjustment actions for different types of anomalies, such as clearing blockages with airflow pulses or finely adjusting the motion parameters of the actuator to compensate for positioning deviations. The entire process forms a continuous optimization closed loop of detection-judgment-adjustment-re-detection, ensuring that the conveying process remains in a controlled and stable state. Therefore, this invention significantly improves the reliability and accuracy of nut conveying, significantly reduces production interruptions caused by conveying failures, enhances the overall production efficiency and product quality of welding stations, and strengthens the equipment's adaptability to complex industrial environments. Attached Figure Description

[0018] Figure 1 This is a flowchart illustrating an automatic nut conveyor operation control method based on vision detection, provided as an embodiment of the present invention.

[0019] Figure 2 This is a schematic diagram of the operation control system for an automatic nut conveyor based on vision detection, provided in an embodiment of the present invention.

[0020] Labeling Explanation: 210, Feature Extraction Module; 220, State Determination Module; 230, Adjustment and Execution Module; 240, Closed-Loop Optimization Module. Detailed Implementation

[0021] The technical solutions of this invention will now be clearly and completely described with reference to the accompanying drawings. Obviously, the described embodiments are merely some, not all, of the embodiments of this invention. The components of this invention described and shown in the accompanying drawings can generally be arranged and designed in various different configurations. Therefore, the following detailed description of the embodiments of the invention provided in the drawings is not intended to limit the scope of the claimed invention, but merely to illustrate selected embodiments of the invention. All other embodiments obtained by those skilled in the art based on the embodiments of this invention without inventive effort are within the scope of protection of this invention.

[0022] It should be noted that similar reference numerals and letters in the following figures indicate similar items; therefore, once an item is defined in one figure, it does not need to be further defined and explained in subsequent figures. Furthermore, in the description of this invention, terms such as "first," "second," etc., are used only to distinguish descriptions and should not be construed as indicating or implying relative importance.

[0023] In increasingly automated modern industrial production lines, such as automotive body-in-white welding workshops or the manufacturing processes of large steel structures, the automatic feeding and precise positioning of nuts are crucial for ensuring welding quality and production cycle time. Traditional automatic nut conveyors typically rely on a fixed set of mechanical parameters and timing logic for open-loop control. While such systems may function under ideal conditions, their vulnerability becomes apparent when faced with unavoidable disturbances in actual production. For example, a new batch of nuts may have a thicker anti-rust oil film on its surface than the previous batch, altering its friction coefficient within the conveying pipe; or, after prolonged operation, slight wear or oil accumulation may occur on the inner wall of the conveying pipe. These subtle changes are imperceptible to traditional control systems, often resulting in nuts getting stuck at a bend in the pipe or slipping off their designated positioning pins due to inertia upon reaching the endpoint. Once such an anomaly occurs, the entire welding station is forced to stop, requiring manual intervention from operators. This not only severely impacts production efficiency but also poses a risk to the quality of the final product.

[0024] Firstly, please see Figure 1 This invention provides a vision-based automatic nut conveyor operation control method, the method comprising: S1. Obtain image data of the nuts to be transported in the transport path, and extract the position features, attitude features and sequence state features of the nuts to be transported; S2. Combine the position detection signals on the conveying path, compare the position features, attitude features and sequence state features with the preset reference values ​​to determine whether the current conveying state is abnormal. S3. If the current conveying state is abnormal, perform the corresponding adaptive adjustment action aimed at eliminating the abnormal state according to the type of abnormal state. S4. After performing the adaptive adjustment action, continuously collect image data and repeatedly determine the current delivery status. Adjust again based on the difference between the re-detection result and the expected target until the delivery status returns to normal.

[0025] Specifically, positional features refer to the spatial position information of the nut to be transported in the image coordinate system. In a specific implementation, this can be quantified as the two-dimensional coordinates of the nut's geometric center, i.e., the (x, y) values. This feature directly reflects the offset of the nut on the cross-section of the transport path.

[0026] Attitude characteristics refer to the directional or angular information of the nut to be conveyed along the conveying path. For a hexagonal nut, its attitude characteristic can be defined as the angle between one of its sides or one of its diagonals and the centerline of the conveying path. This characteristic is crucial to ensuring that the nut can pass smoothly through subsequent limiting structures or be accurately fitted into the locating pin.

[0027] Sequence state characteristics refer to the overall distribution of a group or column of nuts arranged along the conveying path. This does not refer to a single nut, but rather describes the group of nuts. Specifically, it can include the center-to-center distance between adjacent nuts, the number density of nuts per unit length, or the neatness of the nut queue. This characteristic is a key basis for determining whether the feeding rhythm is stable and whether there is accumulation or interruption.

[0028] Position detection signals refer to digital or analog signals output by physical sensors, such as proximity switches, magnetic sensors, or photoelectric sensors, installed at specific locations on the conveying pipeline, such as inlets, bends, and outlets. These signals directly detect whether the nut has arrived at, passed through, or left that physical position, thus providing a time reference and binary position status information independent of the vision system. This complements and verifies the continuous, high-dimensional features extracted by the vision system, together constituting a comprehensive state perception of the conveying process.

[0029] An abnormal state refers to any situation that deviates from the pre-set ideal conveying state. The ideal state is defined by a set of reference values, while the abnormal state is a deviation from this ideal state. It can be further subdivided into various types, such as pipe blockage caused by nuts getting stuck in the pipe and unable to flow normally, outlet positioning deviation caused by deviations in position or posture characteristics from the reference, abnormal feeding rhythm caused by disordered sequence state characteristics, or overlapping and stacking caused by excessively small nut spacing and overlapping compression in the conveying link.

[0030] Adaptive adjustment refers to control behaviors that are dynamically generated and executed to restore a normal state after an abnormal condition is detected, based on the type and severity of the abnormality. For example, a slight positional deviation may only require fine-tuning the motion parameters of the end effector, while severe pipeline blockage may require initiating strong physical intervention.

[0031] In a specific implementation scenario, the method proposed in this invention is implemented on a nut conveying system equipped with an industrial personal computer as the main controller. The conveying path of the system is a transparent or semi-transparent flexible tube with an inner diameter slightly larger than the outer diameter of the nut. Key nodes, such as the vibratory feeder outlet, the midpoint of the long straight section of the tube, and before the end discharge port, are equipped with industrial cameras and matching light sources.

[0032] The execution flow of the method begins with acquiring image data. When one or more nuts enter the camera's field of view and trigger a position detection signal, such as that from a fiber optic sensor, the main controller sends a trigger command to the industrial camera at that location. The camera's image sensor exposes and captures a frame of digital image containing the nuts, and then sends the raw data of this frame of image back to the main controller via a gigabit Ethernet interface.

[0033] Upon receiving image data, the main controller immediately invokes its integrated image processing function library, such as the open-source OpenCV library, to perform feature extraction. First, image preprocessing, such as grayscale conversion and binarization, segments the nuts from the background. Next, a contour detection algorithm is used to find the closed contour of each nut in the image. For each contour, its geometric moments are calculated to obtain the coordinates of its centroid, which is the nut's positional feature. Simultaneously, by calculating the minimum bounding rectangle of the contour, the angle between the long side of the rectangle and the horizontal direction of the image can be obtained; this angle value serves as the nut's pose feature. To obtain sequence state features, the controller analyzes the centroid positions of all identified nuts in the image and calculates the projected distances of adjacent nut centroids in the conveying direction, forming a spacing sequence. This spacing sequence is the sequence state feature, which objectively reflects the nut arrangement density, distribution uniformity, and queue status in the current conveying link.

[0034] Next is the status determination. The main controller integrates the feature data extracted from vision with the position detection signals collected from the conveying path to perform joint determination. Internally, it pre-stores the baseline values ​​and tolerance ranges corresponding to each feature. The position detection signals provide a timing basis for status determination, used to verify the actual passage time, dwell time, and conveying interval of the nuts, compensating for the limitations of single vision detection and improving the reliability of the results. For example, for the end discharge port camera, the position feature baseline is set as the image center point (320, 240), with a pixel tolerance of ±10; the attitude feature baseline angle is 0°, with an angle tolerance of ±5°; the standard nut spacing baseline is 50 mm, with a spacing tolerance of ±5 mm. The controller compares the real-time feature parameters with the baseline thresholds and combines them with the on / off timing signals fed back by the sensors for comprehensive judgment, classifying the abnormal type according to the feature over-limit category and passage status. Specifically: Pipeline blockage determination: Based on the position detection signal, it is confirmed that the nut has reached the corresponding detection section. If the visual acquisition of multiple consecutive frames shows that the centroid coordinate of the nut has basically no displacement, and the sensor does not detect the nut leaving the pipeline for a long time, and the dwell time exceeds the preset limit, it is determined that the nut has blocked the pipeline.

[0035] Positioning deviation status judgment: After the position detection signal feedback nut is positioned in the material discharge positioning area, the real-time extracted nut centroid coordinates and deflection angle are compared with the standard position and standard posture. If the coordinate deviation or angle deviation exceeds the corresponding tolerance range, it can be determined that there is a positioning deviation such as position offset or angle deflection.

[0036] Determination of abnormal feeding rhythm: Based on the time interval between adjacent nuts passing through the same detection point recorded by the position detection signal, and combined with the nut spacing sequence calculated by vision, if the passage interval and arrangement spacing deviate from the standard range, and there is a situation of nuts piling up or material interruption during feeding, it is determined to be an abnormal feeding rhythm.

[0037] Once an anomaly is detected, the system enters an adaptive adjustment phase. The main controller queries a preset rule engine or control logic table based on the anomaly type to initiate corresponding adjustment actions. For example, if a positioning deviation is detected, a correction command is sent to the servo motor driver of the end-effector's material feeding mechanism. If a pipe blockage is detected, the corresponding pneumatic nozzle is immediately triggered, outputting a momentary high-pressure airflow pulse to impact the stuck nut, breaking the blockage and restoring smooth pipe transport. If nut stacking is detected, the vibration frequency and amplitude of the upstream vibratory feeder are simultaneously reduced to slow the feeding rate, preventing continuous nut stacking and compression, and the airflow is used to disperse the dense nut queue. If a feeding rhythm disorder is detected, the vibratory feeder's feeding intensity is reduced for excessively fast feeding and overly dense queues, while the output frequency is increased for excessively slow feeding and station vacancies, dynamically balancing the feeding rhythm of the entire conveyor chain to ensure continuous, orderly, and uninterrupted nut transport.

[0038] One of the core advantages of this invention lies in its closed-loop optimization process. After performing an adjustment action, the camera is triggered again to capture a new image, repeating the feature extraction and comparison process described above. At this time, the controller calculates the new difference between the actual features and the reference value after the initial adjustment. If the difference still exceeds the tolerance, a second adjustment is performed based on this new difference. This process can be designed as a proportional-integral-derivative control loop, where the adjustment amount is proportional to the remaining error. For example, if a stuck nut only moves halfway after the first airflow impact, its position is detected as still incorrect, and a second impact may be performed, or the air pressure or duration of the next impact may be increased. This adjustment-re-detection-re-adjustment loop continues at a very high frequency until all feature values ​​return to the preset tolerance range, at which point the system determines that the conveying state has returned to normal and continues the regular conveying process.

[0039] By introducing this vision-based closed-loop feedback control, the nut conveyor has been transformed from a passive, blind execution device into a system with real-time perception, proactive diagnosis, and intelligent error correction capabilities. This enables it to cope with various working conditions and significantly improves the stability and reliability of automated production lines.

[0040] In real-world industrial environments, especially in the transport of metal parts like nuts coated with anti-rust oil, acquiring clear, interference-free images is fundamental to all subsequent analyses. Metal surfaces are highly susceptible to specular reflection under ordinary lighting, creating high-brightness glare areas. These areas severely disrupt the nut's contour information, preventing image processing algorithms from accurately extracting its position and orientation features. Therefore, this invention further proposes a method for acquiring image data of the nuts to be transported along the transport path, comprising the following steps: Raw images are acquired using industrial cameras and polarized light sources deployed at key nodes along the transport path; The original image was denoised using a polarization filtering algorithm to suppress the interference of oil stains and reflections on the surface of the nuts to be transported, and the image data was obtained.

[0041] The polarization filtering algorithm here is implemented at the physical level through a combination of optical components. In practice, a polarizing filter, or analyzer, is installed in front of the camera lens. Simultaneously, a polarizing filter, or polarizer, is also installed in front of the light source providing illumination to the camera, such as a ring LED light. During installation, the polarization directions of the analyzer and polarizer need to be adjusted so that they are perpendicular to each other, a configuration known as orthogonal polarization.

[0042] Its working principle lies in the fact that the light emitted from the polarizer is linearly polarized. When this light shines on the surface of the nut, two types of reflection occur. Part of the light is scattered in the rough, diffuse reflection area of ​​the nut surface, a process that disrupts the polarization state of the light, making it unpolarized or partially polarized. The other part of the light undergoes specular reflection on the smooth oil film on the nut surface, a process that largely maintains the polarization state of the light. When both types of reflected light simultaneously strike the camera lens, the unpolarized light from the diffuse reflection area can partially pass through the analyzer perpendicular to it, thus forming an image of the nut itself. However, the specular reflection light from the oil film glare area, because its polarization direction is perpendicular to the analyzer, is mostly blocked and cannot enter the camera sensor.

[0043] In this way, the glaring oil reflections in the original acquired image are greatly suppressed, and key details such as the edge contour of the nut and the threaded holes become clearly visible. Subsequent software image processing algorithms, such as edge detection and contour extraction, can be performed on an image with a very high signal-to-noise ratio, thereby ensuring the accuracy and stability of the extracted position, orientation, and other features, laying a solid data foundation for the reliable operation of the entire closed-loop control system.

[0044] After identifying an abnormal conveying status, the system needs to take targeted measures. This invention further classifies abnormal states and provides corresponding adjustment strategies. Abnormal states include blockage, positioning deviation, and abnormal feeding rhythm. The steps for performing appropriate adaptive adjustment actions to eliminate the abnormal state based on the type of abnormal state include: When the abnormal state type is a blockage state, an instantaneous fluid impact force is applied at the blockage location to release the blockage; When the abnormal state type is positioning deviation, the action parameters of the material discharge actuator are adjusted according to the amount of position or attitude deviation. When the abnormal state type is abnormal feeding rhythm, adjust the driving frequency or driving amplitude of the upstream feeding source.

[0045] Specifically, when the vision system detects a blockage, such as at a bend in a pipeline where nuts within the camera's field of view remain stationary for an extended period and their accumulation continues to increase, the system triggers a blockage-clearing logic. At this point, the system activates a fluid impact device. This device can be implemented in several ways. One approach uses high-pressure gas, with one or more nozzles pre-installed near the blockage point. Controlled by a solenoid valve, a powerful jet of gas is instantaneously sprayed onto the stuck nut, using the impact force to loosen it and allow it to continue moving. Another approach utilizes liquid, especially in situations requiring cleaning or cooling of parts. A small high-pressure pump can be used to instantly spray a stream of water or oil to clear the blockage.

[0046] When the vision system detects a positioning deviation, this typically occurs at the discharge port at the end of the conveyor. The camera finds that the center of the nut to be pushed out is off-center from the target positioning pin, or that its hexagonal angle is incorrect and cannot be aligned. In this case, the system dynamically adjusts the motion parameters of the discharge actuator based on the positional deviation calculated by the vision system (e.g., 3 mm in the x-direction, -2 mm in the y-direction) or the attitude deviation (e.g., 15 degrees in the angle). If the discharge mechanism is a servo motor-driven push rod, the controller can modify the target coordinates of this push, incorporating the deviation as a compensation value into the motion command. If it is a simple cylinder push rod, the system might indirectly affect the final position by adjusting an adjustable mechanical stop at its front end or changing its push-out speed curve.

[0047] When the vision system detects an abnormal feeding rhythm, this typically occurs at monitoring points near the upstream feeding source. The camera analyzes the sequence characteristics of the nuts in the image, identifying whether the spacing between them is consistently greater than or less than the set normal range. For example, excessive spacing indicates slow feeding, potentially causing waiting at downstream stations; excessive spacing indicates excessive feeding, easily leading to accumulation in subsequent paths. In this case, the system sends an adjustment command to the upstream feeding source, such as a vibratory feeder. This command can change the AC frequency driving the vibratory feeder's electromagnet or the amplitude of the driving voltage. Generally, increasing the voltage amplitude increases the vibration intensity of the vibratory feeder, speeding up the nut movement; conversely, decreasing it slows it down. The system continuously fine-tunes these driving parameters based on the difference between the current feeding rhythm and the target rhythm until the nut conveying speed returns to a stable and ideal state.

[0048] Regarding the aforementioned scheme of using instantaneous fluid impact force to remove blockages, a specific and efficient implementation method is to utilize a pneumatic device. When the abnormal state type is a blockage, the steps to apply instantaneous fluid impact force to the blockage location to remove the blockage include: When the abnormal state type is a blockage state, the pneumatic device deployed at the blockage point is activated. A pneumatic device injects a pulse of instantaneous high-pressure airflow at a preset pressure into the obstruction location to blow away the stuck nut to be delivered.

[0049] In a specific hardware configuration, the pneumatic unit consists of a central compressed air source at the factory, an air tank, a precision pressure reducing valve, a high-speed solenoid valve, and one or more specially designed nozzles. During the design phase of the nut conveyor, several key points prone to blockage can be identified based on the pipeline geometry and historical experience, such as bends with small radii and pipe diameter changes. Nozzles are installed on the outside of these points, with their nozzles aligned with the inside of the pipeline through small holes drilled in the pipe wall.

[0050] Once the main controller confirms a blockage at a specific point through visual analysis, it immediately sends an opening signal to the high-speed solenoid valve controlling the nozzle at that point. The solenoid valve opens instantaneously, and a powerful pulse of high-pressure air, stabilized at, for example, 0.7 MPa by a precision pressure-reducing valve from the air tank, is precisely delivered through the nozzle to the stuck nut or stack of nuts. The pulse is crucial; the solenoid valve's opening time is precisely controlled within a very short range, such as 50 to 200 milliseconds. This brief but intense impact is sufficient to break the static friction or wedging force between the stuck nut and the pipe wall, loosening it without disrupting the arrangement of other nuts in the pipeline due to continuous airflow. After each pulse injection, the vision system immediately assesses the effect. If the blockage is not completely cleared, the controller can decide to perform a second pulse injection, or slightly increase the pressure set on the pressure-reducing valve during the next injection, gradually increasing the impact force until the blockage is successfully cleared.

[0051] Similarly, for schemes to adjust the upstream feeding rhythm, when the upstream feeding source is specifically a vibratory feeder widely used in industrial production, the adjustment method can be further specified. When the upstream feeding source is a vibratory feeder, and the abnormal state type is an abnormal feeding rhythm state, the steps to adjust the driving frequency or driving amplitude of the upstream feeding source include: Based on the deviation between the current feeding rhythm and the preset rhythm, adjust the electromagnetic drive frequency of the vibratory feeder, or adjust the vibration voltage amplitude of the vibratory feeder, to change the moving speed of the nut to be conveyed.

[0052] The operation of a vibratory feeder relies on an electromagnet and spring plate system beneath its base. A specialized vibratory feeder controller drives the electromagnet by outputting alternating current of a specific frequency and amplitude, causing the vibratory feeder to produce minute vibrations in a specific direction, thereby driving the material inside the feeder to climb and move upwards along a spiral track.

[0053] In the control method of this invention, a communication connection is established between the main controller and the vibratory feeder controller. When a camera deployed near the vibratory feeder outlet detects an abnormal feeding rhythm, the main controller calculates the percentage deviation between the current rhythm and the target rhythm. For example, if the target is to feed 2 nuts per second, but the actual detected rate is 1 nut per second, the deviation is -50%. A control algorithm within the main controller, such as a proportional-integral (PI) controller, calculates an adjustment amount based on this deviation value.

[0054] This adjustment is then translated into specific instructions for the vibratory feeder controller. One method is to adjust the vibration voltage amplitude. The main controller can output an analog voltage signal of 0 to 10 volts to the vibratory feeder controller, which is proportional to the vibration intensity of the vibratory feeder. When slow feeding is detected, the main controller increases the output analog voltage, for example, from 5 volts to 6 volts, thereby enhancing the vibration and speeding up the nut's movement. Another method is to adjust the electromagnetic drive frequency. Some more advanced vibratory feeder controllers allow the drive frequency to be set directly via a communication interface. Changing the frequency can sometimes find a more efficient resonance point, allowing the nut to move more smoothly and quickly at the same vibration intensity. The main controller can decide whether to adjust the amplitude, frequency, or a combination of both based on preset empirical data or through online optimization algorithms, to restore the feeding rhythm to the set target value as quickly as possible, ensuring that the supply at the source of the entire conveyor chain is stable and matches downstream demand.

[0055] In addition to addressing common anomalies such as blockage, positioning deviation, and feeding rhythm, this invention further considers a more complex and challenging anomaly: overlapping or piling of nuts during transport. This not only directly leads to subsequent positioning failures but also easily triggers more serious jamming. To address this, this invention proposes a specialized detection and handling solution.

[0056] Abnormal states also include overlapping and stacking states, and the method further includes: Extract the contour envelope of the set of nuts to be transported from the image data, and calculate the complexity factor of the contour envelope; When the complexity factor exceeds the preset threshold and the centroid height of the set of nuts to be transported changes abruptly, it is determined that the nuts to be transported are in an overlapping and stacked state. The steps of performing corresponding adaptive adjustment actions aimed at eliminating the abnormal state based on the type of abnormal state also include: In response to the overlapping and stacking state, the upstream material supply source is controlled to switch from continuous vibration mode to discrete pulse vibration mode. The asymmetric vibration energy distribution is used to make the overlapping nuts to be conveyed complete the flat disassembly on the conveying path.

[0057] The implementation of this solution comprises two core components: accurate detection and ingenious deconstruction.

[0058] This solution uses a conventional 2D industrial camera to acquire images, which cannot directly obtain the 3D spatial position and layer height data of the nuts. Therefore, it relies on 2D image features to achieve stacking recognition. Furthermore, this camera typically captures images from the side or at an angle downwards. The detection process no longer analyzes the features of individual nuts, but instead treats all nuts within the camera's field of view as a whole set. First, an image processing algorithm is used to fit the outermost convex hull contour of the nut set, i.e., the contour envelope.

[0059] Next, the complexity factor of the contour envelope is calculated. There are several ways to calculate the complexity factor; a simple and effective indicator is the perimeter ratio, calculated as (4*π*A) / P², where A is the area enclosed by the contour envelope and P is the perimeter of the contour envelope. For an ideal, single-column array of nuts, its contour envelope is close to a smooth, slender rectangle, resulting in a low complexity factor. However, once overlap and stacking occur, the contour envelope becomes uneven, with multiple bulges, causing its perimeter P to increase sharply relative to its area A, significantly reducing the complexity factor. Alternatively, its reciprocal can be used as the complexity factor to increase it. Since relying solely on the complexity factor may lead to misjudgments, the solution introduces a second criterion: abrupt changes in centroid height. This involves calculating the coordinates of the centroid of the entire nut set in the vertical direction of the image. The specific calculation method is as follows: first, all pixel regions belonging to nut entities in the image are segmented; then, the vertical coordinates of all valid pixels are counted; finally, the ordinate of the centroid of the set is calculated using a mean-averaging algorithm, thus effectively representing the height position. The calculation formula is: Yc = (y1 + y2 + ... + yn) / n, where yi is the vertical coordinate of a single effective pixel, n is the total number of pixels in the nut area, and Yc is the centroid coordinate of the set in the vertical direction. For example, under normal tiling conditions, the vertical coordinates of the effective pixels of the nut are 36, 38, 37, 39, and 40, and the calculated centroid height Yc = 38. After stacking, the vertical coordinates of the pixels become 49, 51, 50, 52, and 48, and the calculated centroid height Yc = 50. A significant increase in the value indicates a sudden change in the centroid height. In normal horizontal transport, this centroid height should remain stable. When one nut jumps onto another nut, the overall centroid of the set will undergo a significant upward change. Only when both the contour envelope complexity exceeds a preset threshold and a sudden change in centroid height is met will it be reliably determined that an overlapping stacking state has occurred, thus avoiding false alarms caused by the tilting posture of a single nut.

[0060] In terms of handling, once overlapping and stacking are confirmed, a special instruction is immediately sent to the upstream vibratory feeder controller to switch its operating mode from the conventional continuous vibration mode to discrete pulse vibration mode. In continuous vibration mode, the vibratory feeder provides a smooth and continuous vibration force, suitable for the smooth conveying of materials. In discrete pulse vibration mode, the controller outputs a series of short and intense vibration pulses, with brief static intervals between each pulse. More importantly, the energy distribution of these pulses is asymmetrical. This can be achieved by controlling the current waveform of the driving electromagnet, for example, by making the current rise rapidly and then fall slowly. This asymmetrical vibration produces a directional shaking or throwing effect, which makes it easier to cause a lateral displacement of the nuts stacked on top compared to symmetrical vertical vibration. Under the action of a series of such pulses, the nuts on top are gradually shaken to the side, slide off the back of the nuts below, and finally lay flat again on the conveyor track, completing the deconstruction process. After the vision system confirms that the overlapping state has been resolved, the vibratory feeder is controlled to switch back to continuous vibration mode to resume normal conveying.

[0061] While simple position compensation is effective for the positioning deviation mentioned above, it is a reactive measure. To improve positioning accuracy from the root cause, especially when dealing with nuts with oily surfaces, this invention proposes a more refined and predictive control strategy.

[0062] The steps for adjusting the motion parameters of the discharge actuator based on the deviation in position or posture include: Analyze the reflective features of the surface of the nut to be transported in the image data to infer the oiliness level of the surface of the nut to be transported; Calculate the deviation between the positional and / or attitude characteristics and the preset reference values; First control information is generated based on the deviation and greasiness level, and the advance speed curve of the discharge actuator is adjusted according to the first control information. The inertial slippage of the nut to be conveyed during the advance process is suppressed by reducing the starting acceleration.

[0063] Specifically, the greasiness grade refers to the overall physical state of the oil layer formed by the anti-rust oil and processing residue oil adhering to the outer surface of the nut, including its thickness, uniformity of coverage, and fullness of adhesion. This state directly determines the friction coefficient between the nut and the contact surface of the conveying pipe and the discharge actuator, thereby affecting the degree of slippage and offset during the pushing process.

[0064] The core idea of ​​this strategy is that the nut slips during the process of being pushed to the locating pin, mainly due to the excessive acceleration at the moment of initiation, which overcomes the static friction between the nut and the pushing mechanism. The magnitude of this static friction is closely related to the oiliness level of the nut surface. Therefore, this solution first assesses the oiliness level visually. After suppressing most of the glare using polarized illumination, some reflective features related to the oil film are still retained in the image. For example, when the oil film is thicker, the highlight spots on the nut surface may appear larger and have more blurred edges; when the oil film is thinner, it may exhibit sharper and smaller metallic luster points. By extracting the size, average brightness, and gradient of these highlight areas as reflective features and comparing them with a pre-calibrated database, the current oiliness level of the nut can be inferred.

[0065] Specifically, a multi-feature weighted threshold grading method can be used for determination, adapted to real-time industrial control calculations. It assigns a weighted score based on three indicators: highlight area, average surface brightness, and edge gradient, classifying the oiliness level into 1-5 levels. For example, with a total score of 100, the weights are allocated as follows: highlight area accounts for 40%, average surface brightness accounts for 30%, and edge gradient accounts for 30%. The total score range corresponds to the oiliness level.

[0066] The scores and greasiness levels correspond to the following standards: Level 1: Dry and oil-free, [0,20]; Level 2: Slightly oily, [21,40]; Level 3: Standard oil film, [41,60]; Level 4: Slightly thick oil film, [61,80]; Level 5: Heavy oil stains, [81,100].

[0067] Suppose we define the following baseline values: the highlight area is 50 pixels when dry and 500 pixels when heavily coated; the average surface brightness is 240 when dry and 160 when heavily coated; the edge gradient is 200 when dry and 30 when heavily coated. During the judgment process, we first calculate the individual score corresponding to each measured feature parameter, for example, individual score = (measured value - dry baseline) / (heavy coating baseline - dry baseline) * 100, then sum them to obtain the total score and match the level interval, for example, total score = ∑(individual score * weight).

[0068] Taking the measured parameters of 225 pixels of highlight area, 201 pixels of average brightness, and 93 pixels of edge gradient as an example, the highlight area score = (225-50) / (500-50)*100 = 175 / 450*100≈38.9 points; the average brightness score = (240-201) / (240-160)*100 = (39) / (80)*100≈48.8 points; the edge gradient score = (200-93) / (200-30)*100 = (107) / (170)*100≈62.9 points. Note that at this time, the decrease in brightness and the decrease in gradient both indicate that there is too much oil, so the basis order in the calculation is the opposite of that of the area. The total score is calculated as follows: 38.9*40%+48.8*30%+62.9*30%=15.56+14.64+18.87=49.07 points. The total score of 49 points is within the interval [41,60]. Therefore, the oiliness level of the nut to be delivered is determined to be a level 3 standard oil film.

[0069] Subsequently, based on the level of grease, control commands are generated in conjunction with the position and posture deviation. The starting acceleration of the discharge actuator is reasonably reduced according to the degree of oil film lubrication, effectively avoiding the slippage problem of the contact surface caused by grease and reducing positioning deviation from the source.

[0070] When a push action is required, the system not only calculates the required compensation for position and attitude deviations as before, but also considers the previously assessed oiliness level. These two factors together constitute the first set of control information. Based on this information, the main controller dynamically generates an optimal push speed curve. Specifically, this involves adjusting the starting acceleration. If the oiliness level is high, meaning the nut is very slippery and prone to inertial slippage, the controller will significantly reduce the starting acceleration for this push, allowing the push rod to accelerate very gently, ensuring the nut moves closely with the push rod. Conversely, if the nut surface is relatively dry, a larger starting acceleration can be used to shorten the push time. In this way, the system shifts from passively compensating for results to actively managing the process, fundamentally improving positioning accuracy and consistency by predicting and suppressing slippage.

[0071] Furthermore, the ambient temperature in industrial environments is not constant, and the viscosity of the oil film is extremely sensitive to temperature. The physical properties of the oil film differ significantly between cold operation in winter and continuous operation in summer, which directly affects the accuracy of the aforementioned control strategy based on oiliness levels. To address this challenge, this invention introduces a temperature compensation mechanism.

[0072] The steps for adjusting the feed speed curve of the discharge actuator based on the first control information include: The system acquires real-time temperature data of the current conveying environment and retrieves a preset viscosity-temperature mapping table based on the real-time temperature data to determine the rheological properties of the oil film on the surface of the nut to be conveyed. The starting acceleration in the first control information is corrected by combining the rheological characteristic parameters to obtain the compensated dynamic control parameters; The propulsion speed curve is reconstructed according to the compensated dynamic control parameters to offset the fluctuations in the adsorption force between the nut to be transported and the transport path caused by changes in ambient temperature.

[0073] The implementation of this solution first requires installing a temperature sensor, such as a thermocouple or PT100 resistance temperature detector, at a critical point on the nut conveyor, such as near the end discharge mechanism, to acquire real-time ambient temperature data. Simultaneously, during the system commissioning phase, a viscosity-temperature mapping table needs to be established beforehand. This table records the viscosity and other relevant rheological properties of the rust-preventive oil used at different temperatures, such as the adsorption coefficient. This table can be obtained through experimental measurements or by consulting the oil's technical manual.

[0074] For example, a simplified viscosity-temperature mapping table might look like this: Temperature 10°C -> Rheological property parameter (adsorption force coefficient) = 1.5; Temperature 25°C -> Rheological property parameter (adsorption force coefficient) = 1.0 (reference); Temperature 40°C -> Rheological property parameter (adsorption force coefficient) = 0.7; During operation, the main controller reads real-time temperature data before generating the propulsion speed curve. Assuming the current temperature is 10°C, the controller queries the viscosity-temperature mapping table and obtains an adsorption coefficient of 1.5. This means that the oil film viscosity is high, the adsorption force is strong, and the nut is less likely to slip. The controller then uses this coefficient to correct the starting acceleration threshold suggested in the first control information based on visual assessment of oiliness. For example, the correction formula could be: Final acceleration = Base acceleration * Adsorption coefficient, where the base acceleration is the starting acceleration before correction, and the final acceleration is the starting acceleration after correction. Thus, at low temperatures, the system automatically allows a higher starting acceleration to improve efficiency. Conversely, when the equipment runs for a long time and the temperature rises to 40°C, the adsorption coefficient becomes 0.7, and the system further reduces the starting acceleration to cope with the increased risk of slippage due to the thinner oil film. Through this dual compensation (visual assessment of oil film thickness + temperature-compensated oil film viscosity), extremely high-precision prediction and control of the nut's dynamic behavior are achieved, ensuring stable and reliable positioning performance under various ambient temperatures.

[0075] Finally, despite the powerful capabilities of vision systems, minute physical obstacles may arise that are undetectable by vision in certain situations, such as a tiny burr on the nut itself or a small piece of welding slag adhering to the conveyor track. If the material handling mechanism continues to rigidly push according to the predetermined program in such cases, it may cause damage to the mechanism, the nut, or the track. To address this, the present invention also incorporates a protection and obstacle avoidance mechanism based on force feedback.

[0076] The method also includes: Real-time monitoring of the current signal of the drive motor of the material discharge actuator; When a preset pulse signal appears in the current signal, it is determined that the nut to be delivered is physically obstructed; When it is determined that the nut to be delivered is physically obstructed, the current rigid pushing action is interrupted, and the micro-frequency oscillation logic is activated to control the drive motor to perform reciprocating motion, using the self-centering effect generated by the vibration to guide the nut to be delivered around the obstruction.

[0077] This mechanism relies on real-time monitoring of the current of the drive motor (usually a servo motor or stepper motor). The motor driver itself typically has current monitoring capabilities and can feed this data back to the main controller. During normal pushing, the motor load is relatively constant and predictable, and its operating current fluctuates within a normal range. When the nut at the top of the push rod encounters an unexpected physical obstacle, the motor will instantaneously increase its output torque to continue executing the command. This will be directly reflected as a sharp, brief spike in its drive current. The main controller continuously monitors this current signal, and once it detects that the value exceeds a preset safety threshold, for example, 150% of the normal operating current, and this exceeds the threshold for several milliseconds, the system determines that the nut has encountered a physical obstacle.

[0078] Once an obstruction is detected, the main controller immediately takes two actions. First, it interrupts the currently executing rigid pushing command to prevent damage from a head-on collision. Next, it activates a special micro-frequency oscillation logic. This logic sends a series of high-frequency, small-amplitude forward and reverse motion commands to the motor driver, controlling the push rod to perform rapid reciprocating motion near the obstruction point, for example, oscillating at a frequency of 30 Hz with a amplitude of ±0.2 mm. This minute vibration is transmitted to the stuck nut, effectively breaking the static friction between it and the obstacle and giving the nut an opportunity for slight attitude adjustments. This phenomenon, caused by vibration, where an object seeks a stable posture or avoids an obstacle, is known as the self-centering effect. Often, just a brief jolt like this is enough to allow the nut to... By utilizing its chamfered edges or smooth corners, the system cleverly bypasses tiny burrs or weld spatter. After a short period of oscillation, such as 200 milliseconds, the system attempts to push forward slowly again at a preset speed lower than normal, while continuing to monitor the current. If the current returns to normal, it means the obstacle has been bypassed, and the system can resume the normal pushing process. If the current spikes again, the system can repeat the oscillation process, or after several attempts, it may eventually determine a serious fault and trigger an alarm, awaiting manual intervention. This mechanism adds a final safety line to the entire system, making it more intelligent and robust in the face of the unknown and unexpected events.

[0079] Secondly, see Figure 2 The present invention also provides a vision-based automatic nut conveyor operation control system for performing the steps in any of the foregoing methods, the system comprising: The feature extraction module 210 is used to acquire image data of the nuts to be transported in the transport path, and extract the position features, attitude features and sequence state features of the nuts to be transported; The status determination module 220 is used to combine the position detection signal on the conveying path, compare the position features, attitude features and sequence status features with the preset reference value, so as to determine whether the current conveying status is an abnormal status. The adjustment execution module 230 is used to perform corresponding adaptive adjustment actions aimed at eliminating the abnormal state according to the type of abnormal state if the current conveying state is an abnormal state. The closed-loop optimization module 240 is used to continuously collect image data and repeatedly determine the current transport status after performing adaptive adjustment actions, and make further adjustments based on the difference between the re-detection results and the expected target until the transport status returns to normal.

[0080] In terms of hardware, this system can be manifested as an industrial personal computer or embedded controller integrating a high-performance processor, memory, storage, and a rich set of input / output interfaces. In terms of software, these modules can be implemented as different software processes, threads, or function libraries.

[0081] The feature extraction module 210 is responsible for communicating with multiple industrial cameras deployed on the conveyor path and receiving raw image data acquired by the cameras. This module integrates a complete image processing algorithm library, which can perform the aforementioned operations such as polarization filtering, image preprocessing, contour extraction, and geometric feature calculation, and finally output structured feature data, such as the precise coordinates and angles of each nut and the spacing of the nut queue.

[0082] The status determination module 220 receives real-time feature data from the feature extraction module 210 and reads preset benchmark values ​​and tolerance ranges from the memory. The core of this module is a comparison engine that compares the real-time data with the benchmark and, based on the comparison result and a set of preset logical rules, determines whether the current conveying status is normal or abnormal. If it is determined to be abnormal, it further categorizes the abnormality based on the type of feature exceeding the limits, classifying it as a blockage state, positioning deviation state, abnormal feeding rhythm state, or overlapping accumulation state, and then passes the determination result to the next module.

[0083] The adjustment execution module 230 executes specific physical adjustment actions based on the anomaly type output by the state determination module 220. This module connects to various actuators on the conveyor via the controller's output interface, such as a digital output port, analog output port, or serial communication port. These actuators include solenoid valves controlling pneumatic devices, servo motor drivers driving the discharge mechanism, and controllers controlling the vibratory feeder. The closed-loop optimization module 240 ensures that the adjustment actions are not one-off, blind operations. After the adjustment execution module 230 completes an action, the closed-loop optimization module 240 immediately instructs the feature extraction module 210 to perform a new round of image acquisition and analysis, comparing the new state determination result with the desired target. If a deviation still exists, it drives the adjustment execution module 230 to perform another adjustment, forming a continuous feedback loop aimed at eliminating errors until the system fully returns to normal.

[0084] Through the collaborative work of these four modules, the system transforms the aforementioned methodology into an autonomous and self-regulating entity, providing a complete hardware and software solution for achieving highly reliable automatic nut delivery.

[0085] The above description is merely an embodiment of the present invention and is not intended to limit the scope of protection of the present invention. For those skilled in the art, the present invention can have various modifications and variations. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the scope of protection of the present invention.

Claims

1. A nut automatic conveyor operation control method based on visual detection, characterized by, The method includes: Image data of the nuts to be transported in the transport path is acquired, and the position features, attitude features and sequence state features of the nuts to be transported are extracted; By combining the position detection signals on the conveying path, the position features, the attitude features, and the sequence state features are compared with preset reference values ​​to determine whether the current conveying state is an abnormal state. If the current delivery status is abnormal, perform the corresponding adaptive adjustment action aimed at eliminating the abnormal status according to the type of abnormal status; After performing the adaptive adjustment action, the image data is continuously collected and the process of determining the current delivery status is repeated. Adjustments are made again based on the difference between the re-detection result and the expected target until the delivery status returns to normal.

2. The method for controlling the operation of an automatic nut conveyor based on vision detection according to claim 1, characterized in that, The step of acquiring image data of the nut to be conveyed in the conveying path includes: Raw images are acquired using industrial cameras and polarized light sources deployed at key nodes along the transport path; The original image is denoised using a polarization filtering algorithm to suppress the interference of oil stains and reflections on the surface of the nut to be transported, thereby obtaining the image data.

3. The method for controlling the operation of an automatic nut conveyor based on vision detection according to claim 1, characterized in that, The abnormal states include blockage, positioning deviation, and abnormal feeding rhythm. The step of performing corresponding adaptive adjustment actions aimed at eliminating the abnormal state based on the type of abnormal state includes: When the abnormal state type is the blockage state, an instantaneous fluid impact force is applied at the blockage location to release the blockage; When the abnormal state type is the positioning deviation state, the action parameters of the material discharge actuator are adjusted according to the deviation of position or attitude. When the abnormal state type is the abnormal state of the feeding rhythm, adjust the driving frequency or driving amplitude of the upstream feeding source.

4. The method for controlling the operation of an automatic nut conveyor based on vision detection according to claim 3, characterized in that, The abnormal state also includes an overlapping stacking state, and the method further includes: Extract the contour envelope of the set of nuts to be transported from the image data, and calculate the complexity factor of the contour envelope; When the complexity factor exceeds a preset threshold and the centroid height of the set of nuts to be transported changes abruptly, it is determined that the nuts to be transported are in an overlapping and stacked state. The steps of performing corresponding adaptive adjustment actions aimed at eliminating the abnormal state based on the type of abnormal state also include: In response to the overlapping stacking state, the upstream material supply source is controlled to switch from continuous vibration mode to discrete pulse vibration mode, and the overlapping nuts to be transported are laid flat and deconstructed on the transport path by using asymmetric vibration energy distribution.

5. The method for controlling the operation of an automatic nut conveyor based on vision detection according to claim 3, characterized in that, When the abnormal state type is the blockage state, the step of applying a momentary fluid impact force at the blockage location to relieve the blockage includes: When the abnormal state type is the blockage state, the pneumatic device deployed at the blockage point is activated. The pneumatic device injects a pulse of instantaneous high-pressure airflow at a preset pressure toward the obstruction location to blow away the stuck nut to be delivered.

6. The method for controlling the operation of an automatic nut conveyor based on vision detection according to claim 3, characterized in that, The steps of adjusting the action parameters of the discharge actuator according to the deviation of position or posture include: Analyze the reflective features of the surface of the nut to be transported in the image data to infer the oiliness level of the surface of the nut to be transported; Calculate the deviation between the position features and / or the attitude features and the preset reference value; First control information is generated based on the deviation and the oiliness level, and the pushing speed curve of the discharge actuator is adjusted according to the first control information to suppress the inertial slippage of the nut to be conveyed during the pushing process by reducing the starting acceleration.

7. The method for controlling the operation of an automatic nut conveyor based on vision detection according to claim 6, characterized in that, The step of adjusting the feed speed curve of the discharge actuator according to the first control information includes: The real-time temperature data of the current conveying environment is obtained, and a preset viscosity-temperature mapping table is retrieved based on the real-time temperature data to determine the rheological characteristics of the oil film on the surface of the nut to be conveyed. The starting acceleration in the first control information is corrected by combining the rheological characteristic parameters to obtain the compensated dynamic control parameters; The propulsion speed curve is reconstructed according to the compensated dynamic control parameters to offset the fluctuations in the adsorption force between the nut to be transported and the transport path caused by changes in ambient temperature.

8. The method for controlling the operation of an automatic nut conveyor based on vision detection according to claim 6, characterized in that, The method also includes: Real-time monitoring of the current signal of the drive motor of the material discharge actuator; When a preset pulse signal appears in the current signal, it is determined that the nut to be delivered is physically obstructed; When it is determined that the nut to be delivered is physically obstructed, the current rigid pushing action is interrupted, and the micro-frequency oscillation logic is activated to control the drive motor to perform reciprocating motion, using the self-centering effect generated by the vibration to guide the nut to be delivered around the obstruction.

9. The method for controlling the operation of an automatic nut conveyor based on vision detection according to claim 3, characterized in that, The upstream material supply source is a vibratory feeder. When the abnormal state type is the abnormal feeding rhythm state, the step of adjusting the driving frequency or driving amplitude of the upstream material supply source includes: Based on the deviation between the current feeding rhythm and the preset rhythm, adjust the electromagnetic drive frequency of the vibratory feeder, or adjust the vibration voltage amplitude of the vibratory feeder, to change the moving speed of the nut to be conveyed.

10. A vision-based automatic nut conveyor operation control system, used to execute the steps of the method according to any one of claims 1 to 9, characterized in that, include: The feature extraction module is used to acquire image data of the nuts to be transported in the transport path, and extract the position features, attitude features and sequence state features of the nuts to be transported; The status determination module is used to combine the position detection signal on the conveying path, compare the position feature, the attitude feature and the sequence status feature with a preset reference value, so as to determine whether the current conveying status is an abnormal status. The adjustment execution module is used to perform corresponding adaptive adjustment actions aimed at eliminating the abnormal state if the current delivery state is an abnormal state, based on the type of abnormal state. The closed-loop optimization module is used to continuously collect the image data and repeat the process of determining the current delivery state after the adaptive adjustment action is performed, and to make further adjustments based on the difference between the re-detection result and the expected target, until the delivery state returns to normal.

Images