High-speed intelligent screw machine control system and method based on artificial intelligence and manipulator

Abstract

The invention relates to the technical field of industrial automation equipment, discloses a high-speed intelligent screw machine control system and method based on artificial intelligence and a manipulator, and solves the problems that an existing screw machine is low in locking efficiency, poor in positioning precision, insufficient in product changing flexibility and free of real-time anomaly detection processing capacity. The control system comprises a hardware layer, a software layer and a cooperative control layer, the hardware layer is provided with a double-vision identification structure, an industrial manipulator, a locking execution mechanism module, a feeding module, a sensor module, an industrial personal computer and a control module; the software layer is internally provided with a convolutional neural network model and a reinforcement learning model, is provided with a process template library, and further comprises a visual identification module, an AI algorithm module, a motion control module, a locking execution control module, a data storage and analysis module, a task scheduling module and a man-machine interaction module; the cooperative control layer is provided with a control interface and an MES data reporting interface, and bidirectional data interaction with an external production line control system and a factory MES system is achieved.

Description

Technical Field

[0001] This invention relates to the field of industrial automation equipment technology, specifically to a high-speed intelligent screw machine control system and method based on artificial intelligence and a robotic arm. Background Technology

[0002] In the field of industrial automated assembly, screw fastening is a fundamental and frequently performed operation in industries such as electronics manufacturing, automotive parts assembly, and home appliance production. Its fastening efficiency and precision directly determine product quality and production capacity. Currently, screw fastening machines on the market are mainly divided into three categories: manual, semi-automatic, and traditional fully automatic. Manual screw fasteners rely entirely on manual operation. Operators are prone to fatigue, which can lead to problems such as uneven fastening force and screw hole positioning deviation. The screw fastening efficiency per shift is generally less than 5,000 screws, making them unsuitable for large-scale mass production scenarios.

[0003] Although semi-automatic screw machines achieve automatic screw feeding, manual operation of the locking mechanism is still required for positioning. The positioning accuracy error is usually greater than ±0.5mm, and the high degree of manual involvement leads to high production labor costs.

[0004] Traditional fully automatic screw machines use a fixed program and hard control mode, which can only adapt to the fastening requirements of a single type of workpiece. When the workpiece model is changed, the mechanical structure and program parameters need to be readjusted, which takes 2-4 hours. This results in poor flexibility. At the same time, this type of equipment lacks real-time anomaly detection and processing capabilities. When faced with production anomalies such as missing screws, stripped threads, or workpiece misalignment, it cannot stop the machine in time to make adjustments, which directly leads to an excessively high product defect rate.

[0005] With the increasing demand for intelligent, high-speed, and flexible assembly lines from industries such as electronics and automobiles, the technical pain points of existing screw machines have become key obstacles to improving production efficiency. There is an urgent need for a screw machine control system and method that integrates intelligent algorithms and high-precision mechanical operation to solve the problems of low efficiency, poor precision, low changeover efficiency, and weak abnormal handling capabilities of existing equipment, and to achieve automation, intelligence, and flexibility of the entire screw fastening process. Summary of the Invention

[0006] The technical problem to be solved by the present invention is to provide a high-speed intelligent screw machine control system and method based on artificial intelligence and robotic arms, so as to solve the problems of low fastening efficiency, poor positioning accuracy, insufficient production changeover flexibility, and lack of real-time anomaly detection and processing capabilities of existing screw machines.

[0007] The technical solution adopted by the present invention to solve the above-mentioned technical problems is as follows: On the one hand, this invention provides a high-speed intelligent screw machine control system based on artificial intelligence and a robotic arm, comprising: The system comprises a hardware layer, a software layer, and a collaborative control layer. These three layers interact and cooperate through a data interface to achieve fully automated control of the screw fastening process. The hardware layer includes a dual-vision recognition structure and an industrial robot. The dual-vision recognition structure includes a global camera mounted on the workbench and a local camera mounted on the end of the robot. The hardware layer also includes a locking actuator module mounted on the end of the robot, a feeding module with a screw detection sensor, a sensor module, an industrial computer, and an industrial-grade control module. The industrial computer is the core interaction hub between the hardware layer and the software layer. The software layer incorporates a convolutional neural network model and a reinforcement learning model, includes a process template library, and further comprises a visual recognition module, an artificial intelligence algorithm module, a motion control module, a locking execution control module, a data storage and analysis module, a task scheduling module, and a human-computer interaction module. Each module is decoupled and collaborates through the task scheduling module to achieve full-process software control of workpiece recognition, screw hole positioning, locking parameter optimization, and data storage and analysis. The collaborative control layer is equipped with a control interface and an MES data reporting interface, which respectively enable bidirectional data interaction with the external production line control system and the factory MES system.

[0008] This solution utilizes a modular design based on a three-layer architecture to achieve coordinated linkage between system hardware execution, software intelligent computation, and external system integration, enabling fully automated control of the screw fastening process. Through a combination of dual-vision recognition structures and a robotic arm, dual positioning—both global workpiece positioning and precise local screw hole positioning—is achieved at the hardware level, solving the problem of low positioning accuracy in traditional equipment. Software design based on dual AI models and a process template library enables the system to intelligently identify workpieces, optimize fastening parameters in real time, and automatically match production parameters for multiple models. A standardized design with dual external interfaces allows the system to integrate into the factory production line control system and the MES system, enabling bidirectional interaction and remote management of production data.

[0009] Furthermore, the industrial robot in the hardware layer is a six-axis robot with a repeatability accuracy of ≤±0.02mm, used to perform screw gripping and positioning operations; The sensor module in the hardware layer includes a compressed air pressure sensor, an electrostatic voltage sensor, and a metal proximity sensor, which are used to detect working air pressure, electrostatic voltage at the end of the equipment tightening tool, and missing nail detection, respectively.

[0010] This solution employs a high-precision six-axis robot to ensure high accuracy in screw hole positioning and screw fastening from a hardware parameter perspective, avoiding workpiece damage or poor fastening due to positioning deviations. The configuration of multiple types of sensors enables multi-dimensional detection of working air pressure, electrostatic voltage, and the presence or absence of screws, providing hardware data support for real-time anomaly detection in the system.

[0011] Furthermore, the convolutional neural network model is integrated into the visual recognition module for workpiece contour extraction, workpiece model recognition, and calculation of screw hole position coordinates and attitude angles; the reinforcement learning model is integrated into the artificial intelligence algorithm module for real-time optimization of locking parameters based on locking process data; and the process template library is built into the data storage and analysis module for storing production task parameters for multiple workpiece models and supporting retrieval and matching.

[0012] In this solution, a convolutional neural network model is integrated into the visual recognition module to achieve image acquisition, feature extraction, and coordinate calculation functions, ensuring the accuracy and efficiency of workpiece and screw hole recognition. By integrating a reinforcement learning model into the artificial intelligence algorithm module, the model is linked with the fastening process data to achieve data acquisition, parameter optimization, and command issuance, making the fastening parameters adapt to the actual needs of different workpieces / working conditions and continuously reducing the screw fastening defect rate. By setting up a process template library, the storage and retrieval of production parameters for multiple models form a standardized process, and the production parameters of the configured models can be retrieved with one click, realizing rapid switching of production.

[0013] Furthermore, the control interface in the collaborative control layer is used to receive workpiece arrival signals, workpiece barcode numbers, workpiece models, and workpiece good product information transmitted from the external production line control system; the MES data reporting interface in the collaborative control layer is used to upload real-time production data, statistical data, trend analysis data, and abnormal alarm data to the factory MES system.

[0014] In this solution, by clearly defining the input data type of the control interface, the system can be more accurately connected with the production line control system, enabling automatic acquisition of workpiece information and synchronization of production instructions, thereby improving the level of production automation. By clearly defining the output data type of the MES data reporting interface, the system's production data and abnormal data can be uploaded in all dimensions, providing data support for the factory's MES system's production management, data traceability, and trend analysis, and realizing the synergy between the screw fastening process and the overall production management of the factory.

[0015] On the other hand, the present invention also provides a high-speed intelligent screw machine control method based on artificial intelligence and a robotic arm, which is applied to the above-mentioned control system. The method includes the following steps: S1. The system completes hardware self-test and status initialization, loads intelligent model and historical production task parameters, and enters automatic production preparation state to wait for workpieces to arrive. S2. After the workpiece is detected to be in place, the image is acquired by the global camera and the workpiece model and contour coordinates are identified by the intelligent model. The corresponding production task parameters are matched and the screw hole robot coordinates are calculated to generate the screw fastening task queue. S3. Sequentially position the screw holes through the local camera according to the screw fastening task queue, control the screw fastening actuator to complete the screw fastening, synchronously collect the screw fastening process data and optimize the screw fastening parameters through the intelligent model until all screws on the workpiece are fastened. S4. After completing the workpiece fastening, determine the workpiece's qualification, summarize the production data and upload it to the factory's MES system, control the robot arm to complete the automatic screw feeding and gripping, and reset to the working posture to wait for the next workpiece. S5. Real-time detection of various abnormal signals during production. If an abnormality is detected, the system will stop immediately and trigger an alarm, record the abnormal information, and reset the system to resume production after the abnormality is eliminated. S6. Stop automatic production according to the production operation instructions, complete the reset of each module and the production task closure, count the production data of the day and generate a production report, upload it to the factory MES system and then complete the system operation.

[0016] This solution replaces the traditional screw fastening machine's operation mode, which lacks standardized processes and involves a lot of manual intervention, with a standardized control method covering the entire process from production preparation, workpiece identification, fastening execution, material preparation, anomaly handling, and production completion. This achieves full automation and intelligence of the screw fastening process.

[0017] Furthermore, step S1 specifically includes: The industrial computer first completes its own hardware self-test, and then checks the remaining hardware modules one by one through the control module. When a fault occurs, an alarm is triggered and the fault location is displayed. After the self-test passes, the listeners of each module are initialized, the convolutional neural network model and reinforcement learning model are loaded, the task scheduler is initialized and the parameters of the previous production task are loaded, and after the ready signal is fed back, the control module program is reset, the robot returns to the initial position, and the material shortage detection and automatic material picking are completed, and the automatic production waiting state is entered.

[0018] In this solution, hardware faults are detected in advance through self-testing design, avoiding production anomalies caused by faulty modules participating in production. Furthermore, by completing the reset of PLC and robot arm and material shortage detection, locking errors caused by initial equipment state deviations are avoided, laying the foundation for subsequent high-precision locking.

[0019] Furthermore, step S2 also includes: if no corresponding production task parameters are matched, an alarm is triggered and a prompt to create a new process template is given. The newly created process template is stored in the process template library and the production parameter matching step is executed again. The production task parameters include workpiece model, screw specifications, screw hole position, fastening sequence, working height, fastening torque, fastening speed, screw hole reliability threshold, and screw hole obstruction threshold.

[0020] In this solution, when production task parameter matching fails, the problem of traditional equipment being unable to adapt to new workpiece models can be solved by creating a new template and re-matching it, thereby improving the system's production flexibility.

[0021] Furthermore, methods for creating new process templates include: Enter the template name, workpiece model, screw model, and various production and identification parameters; The workpiece image is acquired by a global camera, the screw holes are delineated and the fastening sequence is set, and the rough coordinates of the screw hole robot are calculated. The actual working height of the screw hole is calculated by identifying the surface features of the screw hole using a local camera. Store the template parameters in the process template library to complete the creation of the process template.

[0022] In this solution, a standardized process template is used to create a new workflow, making the template configuration for new workpiece models more efficient and accurate. A dual-vision approach combining global camera sizing and local camera height measurement is adopted to achieve full-dimensional calibration of the screw hole plane coordinates and height, ensuring the accuracy of the screw hole coordinates and providing basic data for subsequent high-precision fastening. The template parameters can be repeatedly retrieved after being stored in the process template library, enabling one-time configuration and multiple uses, significantly reducing the production preparation time for new workpiece models and improving the system's production flexibility.

[0023] Furthermore, in step S3, the precise positioning of the screw hole using a local camera includes: controlling the robot arm to align the local camera with the screw hole, identifying the screw hole features through an intelligent model, eliminating invalid screw holes, calculating the precise robot arm coordinates for aligning the electric screwdriver with the screw hole, controlling the robot arm to move so that the electric screwdriver is precisely aligned with the screw hole; and sending the fastening parameters to the control module for execution after the industrial control computer verifies their rationality.

[0024] In this solution, during the implementation of local camera precision positioning, the use of screw hole identification, invalid screw hole elimination, accurate coordinate calculation, and screwdriver alignment avoids blind drilling of invalid screw holes such as those that are obstructed or already locked. In addition, the rationality verification step of the locking parameters ensures that the optimized locking parameters are within the range of the equipment.

[0025] Furthermore, in step S5, the various abnormal signals include screw missing signal, torque exceeding standard signal, air pressure too low signal, hardware mechanism movement failure signal, and image recognition abnormal signal; the abnormal information includes the time of abnormality, abnormality type, abnormal data, and abnormality location.

[0026] This solution refines the types of abnormal signals to achieve full-dimensional anomaly detection during screw fastening, solving the problem of traditional equipment having a single dimension of anomaly detection and failing to detect problems in a timely manner. By clarifying the recorded content of abnormal information, it enables full-dimensional traceability of abnormal data in terms of time, type, data, and location, providing accurate data support for subsequent anomaly cause analysis, equipment maintenance, and process optimization, and solving the problem of traditional equipment having no or incomplete anomaly information records.

[0027] The beneficial effects of this invention are: (1) Improve locking efficiency and accuracy: By employing an industrial robotic arm in conjunction with a dual vision recognition system, dual precision positioning is achieved, enabling both global workpiece positioning and precise local positioning of screw holes, thus avoiding positioning deviations caused by manual intervention. Through the coordination of data transmission between modules by an industrial control computer, the instruction latency is reduced from 50ms in traditional systems to 8ms, ensuring the real-time performance of the fastening process. Each screw hole is first identified and then fastened, which not only allows for the calculation of more accurate screw hole positions but also enables the elimination of defective screw holes based on their actual working conditions, preventing incorrect fastening.

[0028] (2) Achieve highly flexible production: A process template library is built, which can automatically switch to products that already exist in the library during production with a switching time of less than 1 second. For products that are not in the process template library, the switching time for creating a new template is less than 5 minutes, solving the problem of long changeover and debugging time of traditional equipment and adapting to the "multi-variety, small-batch" production mode. The workpiece model is automatically identified and the production parameters are automatically matched without the need for manual intervention in parameter debugging, further improving production flexibility.

[0029] (3) Possesses real-time anomaly detection and processing capabilities: Through multi-sensor modules and real-time system monitoring, the system can detect production anomalies such as missing screws, excessive torque, low air pressure, and hardware movement failures in real time. Upon detection of an anomaly, the machine will stop immediately and trigger an audible and visual alarm, thereby reducing the defect rate and minimizing production losses. Anomaly information is automatically recorded to provide data support for subsequent production optimization.

[0030] (4) Achieve intelligent and traceable production processes: By integrating convolutional neural networks and reinforcement learning models, intelligent workpiece recognition, precise screw hole positioning, and dynamic optimization of fastening parameters are achieved. The data storage and analysis module enables persistent storage of fastening data, production data, and abnormal data. At the same time, it connects with the factory's MES system through the MES data reporting interface to achieve real-time uploading of production data, remote monitoring, and full-process traceability, providing data support for factory production management and optimization. Attached Figure Description

[0031] Figure 1 This is a diagram of the control system architecture for the high-speed intelligent screw machine based on artificial intelligence and robotic arms in this invention.

[0032] Figure 2 This is a flowchart of the high-speed intelligent screw machine control method based on artificial intelligence and robotic arms in this invention.

[0033] Figure 3 This is a flowchart illustrating the configuration of the production process template in this invention. Detailed Implementation

[0034] This invention aims to provide a high-speed intelligent screw fastening machine control system and method based on artificial intelligence and a robotic arm, solving the problems of low fastening efficiency, poor positioning accuracy, insufficient changeover flexibility, and lack of real-time anomaly detection and handling capabilities in existing screw fastening machines. Its core idea is to build a high-speed intelligent screw fastening machine control system with a layered architecture design, comprising a hardware layer, a software layer, and a collaborative control layer. This integrates the visual recognition capabilities of convolutional neural networks and the parameter optimization capabilities of reinforcement learning models, coupled with the high-precision operation of a 6-axis industrial robotic arm, addressing the pain points of low efficiency, poor accuracy, insufficient flexibility, and weak anomaly handling capabilities in existing screw fastening machines. Simultaneously, a standardized and intelligent screw fastening control method is designed to achieve fully automated control of the entire process from system initialization, workpiece identification, high-precision fastening, screw feeding to anomaly handling and production completion. This balances the high speed and high precision of screw fastening with production flexibility, achieving seamless integration with existing factory production line control systems and MES systems, ultimately achieving intelligent, automated, and standardized screw fastening processes.

[0035] See Figure 1 The high-speed intelligent screw fastening machine control system based on artificial intelligence and robotic arms provided by this invention includes a hardware layer, a software layer, and a collaborative control layer. These layers work together to achieve fully automated control of the screw fastening process. In one exemplary embodiment, the specific structure of each layer is as follows: 1. Hardware layer Robotic arm: A 6-axis industrial robotic arm with a repeatability accuracy of ±0.02mm is used to perform screw gripping and positioning operations.

[0036] Locking actuator: mounted on the end of the robot arm, with adjustable tightening parameters, used for automatic feeding, completing the locking operation, and providing feedback on the locking result.

[0037] Vision module: includes two industrial cameras, a light source and an image acquisition card. The global camera is mounted on the worktable for workpiece positioning; the local camera is mounted on the end effector of the robot arm for screw hole recognition and precise positioning.

[0038] Feeder: A push-type automatic feeder is used, equipped with a screw detection sensor, to realize automatic screw feeding and missing screw detection.

[0039] Sensors include a compressed air pressure sensor, an electrostatic voltage sensor, and a metal proximity sensor, which are used to detect working air pressure, electrostatic voltage at the end of the equipment tightening tool, and missing nail detection, respectively.

[0040] Industrial PC: As the core interaction hub between system hardware and software, it is responsible for running the human-machine interface (HMI), controlling the robot's motion, coordinating PLC data transmission, running image recognition and artificial intelligence algorithms, storing real-time production logs (such as equipment operating status and locking process data), and supporting remote monitoring functions (interfacing with the factory's MES system via Ethernet to achieve real-time uploading of production data).

[0041] Control module: An industrial-grade PLC is used to control the logic of the hardware equipment (such as starting and stopping the feeding module and executing the tightening tool locking).

[0042] 2. Software layer Visual recognition module: Includes a convolutional neural network (CNN) model, used to extract workpiece contours, identify workpiece models, calculate screw position coordinates, and identify attitude angles.

[0043] AI algorithm module: Includes reinforcement learning models to optimize latch-up parameters, reducing the latch-up defect rate to below 0.5%.

[0044] Motion control module: responsible for the robot's motion control, motion trajectory planning, and motion speed adjustment.

[0045] Screw fastening execution control module: responsible for communicating with the control module to realize screw fastening execution control, fastening data acquisition, sensor data acquisition and status monitoring.

[0046] Data storage and analysis module: Uses a database to persistently store locking data (including workpiece model, locking time, torque value, and number of defective products), realizes data statistics and trend analysis, and the data can be synchronously backed up to the factory cloud server through the industrial control computer, providing data support for production optimization.

[0047] Task scheduling module: As the event hub of the system, it decouples other modules from each other, coordinates the modules, and arranges and executes each step of the production task.

[0048] Human-Machine Interaction Module: Running on an industrial computer, it provides operators with a visual operating interface, supporting production parameter settings, equipment status display, equipment start / stop operations, real-time display of work details, abnormal alarm pop-ups, and production report query and export.

[0049] 3. Collaborative Control Layer Control interface: The production line control system or the line PLC can use this interface to transmit workpiece arrival signals, workpiece barcode numbers, workpiece models, and workpiece quality information to achieve more flexible and reliable automated production.

[0050] MES Data Reporting Interface: The data storage and analysis module can use this interface to report real-time data, statistical data, and trend analysis data to the MES system.

[0051] Based on the above system, the implementation process of the high-speed intelligent screw machine control method based on artificial intelligence and robotic arms provided by this invention is as follows: Figure 2 As shown, it includes the following implementation process: S1. System Initialization S11. Start the control system. The industrial computer first performs a self-test on its own hardware (such as memory, hard disk, and communication interface). After the self-test is passed, the PLC performs tests on the locking actuator, feeding module, and sensors one by one, and then tests the robot and vision module. The test results are fed back to the industrial computer HMI interface in real time. If there is a fault (such as no signal from the sensor, interruption of robot communication, etc.), the industrial computer immediately triggers an alarm and displays the fault location.

[0052] S12. After the self-test passes, initialize the system status: initialize the PLC status listener to monitor the status of the locking actuator, feeder, and sensors in real time; initialize the robot status listener to monitor the robot status in real time; load the learning model and image recognition model, and initialize the vision module; initialize the task scheduler, loading the parameters of the previous production task (including workpiece model, screw specifications, screw hole position, working height, and process parameters (torque, speed, screw hole locking sequence)) as initial parameters. After initialization, send a "ready" signal to the HMI interface and enter the system main interface.

[0053] S13. The operator enters the production preparation interface by clicking the "Automatic Production" button on the industrial control computer HMI interface. Simultaneously, the PLC program is reset, the robotic arm returns to its initial position, and material shortage detection and automatic material handling are executed. Once everything is ready, the operator clicks "Start Production" to initiate the "Automatic Production" state and waits for the workpiece to arrive.

[0054] S2. Workpiece positioning and recognition S21. The workpiece flows into the current station through the automatic production line. The line limit switch or photoelectric switch detects that the workpiece has arrived, stops the workpiece at the current station, and feeds back the workpiece arrival signal through the PLC interface.

[0055] S22. The task scheduling module detects the workpiece arrival signal and sends a "workpiece recognition" command to the vision module. The vision module acquires a global image, preprocesses the workpiece image (denoising, grayscale conversion, edge enhancement), then identifies the workpiece contour features, determines the workpiece model, identifies the planar coordinates of the workpiece contour in the camera's field of view, and feeds back the recognition results.

[0056] S23. The task scheduling module compares the identified model with the currently initialized production task parameters. If the models do not match, it searches for the production task parameters for the current model in the process template library of the data storage module. If no production task parameters for the current model are found, an alarm pop-up window "No process template for the current model" appears on the HMI interface. After the operator creates a process template for the current model, "Automatic Production" is restarted. If the production task parameters for the current model are found, the parameters are set as the current production task parameters. If the models match, the currently initialized production task parameters are used directly.

[0057] When the system's process template library lacks production task parameters for the current workpiece model, these parameters must be added to the library. An example production process template configuration process can be found here. Figure 3 The details are as follows: a. The operator clicks the "New Template" button on the industrial control computer HMI to enter the process of creating a new process template.

[0058] b. The operator enters the template name, workpiece model (or can read the current workpiece model through the PLC interface), screw model, tightening parameters (torque, angle), and visual recognition parameter thresholds (such as screw hole occlusion threshold, credibility threshold), and clicks "Next".

[0059] c. The operator clicks the "Global Photo" button to capture a global image and displays the captured image on the HMI interface. The operator then circles the screw holes to be fastened on the image and sets the order. The visual recognition module is called to identify the workpiece contour features in the image, obtain the camera coordinates of the workpiece contour, and calculate the robot's (x,y) world coordinates (rough coordinates) for each screw hole by combining the pixel coordinates of the screw holes.

[0060] d. The operator places the marker used for distance measurement on the surface of the screw hole and clicks the "Measure Height" button next to the screw hole marker on the HMI interface to start measuring the working height of the screw hole. The robot arm is moved by the motion control module so that the local camera is aligned with the "rough coordinates" obtained in step c. The marker feature is identified by the vision recognition module. The size of the identified marker in the image is compared with the standard size and calculated to obtain the actual working height of the screw hole.

[0061] e. Through the data storage and statistical analysis module, the newly created process template configuration is stored in the process template library for automatic production tasks to call.

[0062] S24. Extract the screw hole fastening sequence, screw hole position (relative position of the screw hole relative to the workpiece contour), and working height from the production task parameters, and combine them with the workpiece contour plane coordinates obtained from global image recognition to calculate the robot coordinates corresponding to each screw hole, and generate a screw fastening task queue.

[0063] S3. High-precision locking S31. The task scheduling module extracts the first screw fastening task.

[0064] S32. The task scheduling module sends a motion command to the motion control module to move the robotic arm so that the local camera at the end of the robotic arm is aligned with the screw hole. After the robotic arm stops in place, the motion control module reports the "motion in place" result.

[0065] S33. After receiving the message that the local camera at the end of the robot arm is aligned with the screw hole, the task scheduling module calls the local camera image recognition to identify the screw hole features. If no screw hole is identified, the screw hole is obstructed, or the screw hole is already secured with a screw, the HMI interface will pop up an alarm window saying "No valid screw hole," prohibiting subsequent operations to avoid "blind drilling" that could damage the workpiece. If a screw hole is detected and its condition is good, the module extracts the center camera coordinates of the screw hole and, combined with the robot arm's current coordinates and the positional offset between the local camera and the electric screwdriver, calculates the precise robot arm coordinates for aligning the electric screwdriver with the screw hole. Then, it sends a motion command to the motion control module to move the robot arm so that the electric screwdriver at the end of the robot arm is aligned with the screw hole.

[0066] S34. After receiving the electric screwdriver aligned with the screw hole, the task scheduling module sends a tightening command to the PLC control module. The PLC controls the tightening actuator to complete the screw tightening according to the current tightening parameters (T0, n0) and feeds back the tightening process curve data, final torque, final angle, tightening result (OK / NG), and NG reason.

[0067] S35. Data is transmitted from the PLC to the industrial control computer, where it is collected and persistently stored by the data storage and analysis module. The torque and angle change trends are displayed in real-time curve form on the HMI interface. At the same time, the actual fastening data is compared with the preset requirements. If the requirements are met, the fastening is deemed qualified; otherwise, it is deemed defective. The qualified / defective results are automatically stored in the industrial control computer's production log.

[0068] S36. The learning model optimizes the locking parameters based on the real-time collected torque and angle data, generating the optimal torque T1 and speed n1. The optimized parameters are fed back to the industrial control computer, which verifies the rationality of the parameters (such as whether the torque is within the range of the equipment). After the verification is passed, the parameters are sent to the PLC.

[0069] S37. After completing a screw locking task, the task scheduling module repeats steps S32 - S36 to continue with the next screw locking task until all screw locking tasks for the workpiece are completed.

[0070] S38. If all screw locking tasks are qualified, the workpiece is determined to be qualified; otherwise, the workpiece is determined to be defective. The data storage and analysis module then summarizes the production data of the workpiece and stores it persistently. At the same time, it reports to the MES production system through the MES interface and synchronously updates the real - time production data and the qualified rate statistical data on the HMI interface. Then it notifies the production line to release the workpiece through the PLC interface.

[0071] S39. After completing all screw locking tasks for the entire workpiece, the task scheduling module will next execute the "screw feeding and grasping" task.​​​​​​​​​​​​​​​​​​S52. After the operator troubleshoots the abnormality, they click the "Reset" button on the industrial control computer HMI. The industrial control computer sends a reset command to the PLC, and the PLC controls each module to restore its initial state. After the reset is completed, the HMI interface returns to normal production status and production continues.

[0077] S6. Stop and End of Work S61. The operator clicks the "Stop" button on the industrial control computer HMI to exit the automatic production state; the task scheduling module executes the exit automatic production task: sends a reset command to the PLC, and the PLC controls each mechanism module to restore the initial state; sends a reset command to the robot arm, and the robot arm returns to the standby position and adjusts to the standby posture; sends a reset command to the vision module to stop real-time image acquisition; the data storage and statistical analysis module updates the production task end time to complete the production task closed loop.

[0078] S62. After exiting automated production, the operator clicks the "End Work" button through the industrial control computer HMI. The data storage and statistical analysis module automatically compiles the daily production data (output, pass rate, defective product cause classification, production cycle time) and generates production reports (including data charts). At the same time, the production data is uploaded to the factory MES system via Ethernet. The production reports can be automatically sent to the relevant responsible persons via email through the system email configuration. After the data processing is completed, the system automatically shuts down. Example

[0079] This embodiment takes the screw fastening task in the assembly of TV motherboard and power board as an example. The number of screws fastened is 4 to 6 per workpiece, and the production batch is 1200 to 1500 workpieces per day. The industrial control computer is used to connect with the factory's MES system to complete the production task distribution and data upload. The PLC is used to connect with the factory's SCADA system based on the Modbus TCP protocol to complete the workpiece arrival detection and basic workpiece information exchange.

[0080] The equipment parameters are as follows: Robotic arm: A 6-axis industrial robotic arm is selected, with a repeatability accuracy of ±0.02mm.

[0081] Vision module: The global camera uses a 12-megapixel industrial CCD camera, and the local camera uses a 6-megapixel industrial CCD camera, paired with a strip light source.

[0082] Locking actuator: It adopts a nail storage structure design and is equipped with an electric screwdriver. The locking spindle torque range is 0.1-5 N·m.

[0083] Feeder: A push-type automatic feeder is selected, with a feeding speed of 400 pieces / minute.

[0084] Industrial PC: Equipped with an Intel Core i7-10700 processor, 16GB DDR4 memory, 512GB SSD solid-state drive, running Ubuntu 22.04 operating system, and supporting Ethernet (RJ45 interface) and RS485 interface communication.

[0085] Control module: Industrial-grade PLC, supporting Modbus TCP protocol communication.

[0086] The implementation process is as follows: Step 1. System Initialization: After the industrial control computer passes its self-test, the HMI interface displays "System testing in progress, please wait." The control PLC performs tests on the actuator module, feeding module, sensors, and robot arm. Upon successful testing, system initialization begins, and the HMI displays "System initialization, please wait." This initialization includes the PLC status listener, robot arm status listener, loading models for the vision module and algorithm module, and initializing the task scheduler, loading parameters from the previous production task as initial parameters. After initialization, a "Ready" signal is sent to the HMI interface, and the system main interface is entered. Since the last produced TV model was 55DH66, the current production model displayed on the interface is 55DH66.

[0087] Step 2. The operator clicks the "Automatic Production" button on the HMI main interface to enter the automatic production preparation interface. The system checks whether the PLC and robot are communicating well, whether the PLC has been reset, whether the robot is in the ready position, whether there is a shortage of materials, and whether there are no alarms. If there is a state that is not ready, the operator will be prompted on the HMI interface and instructed on how to handle it. When everything is ready, the operator clicks the "Start Production" button, and the system enters the "Automatic Production" state.

[0088] Step 3. The production workpiece automatically enters the production station of the screw machine along the production line. The factory SCADA system writes the workpiece arrival signal, workpiece model, and workpiece barcode number to the PLC. The PLC status listener detects the workpiece arrival, reads the workpiece model (55D66H) and barcode number (CS200025188982), calls the vision module to identify the workpiece contour features, identifies the model as 55D66H, and successfully matches the current production task parameters. The current number of screws to be fastened is 4. Create a queue of 4 screw fastening tasks and create new production data for this workpiece.

[0089] Step 4. Move the robotic arm to the locking position, align the local camera with the screw hole, identify the screw hole, and calculate the precise coordinates of the electric screwdriver for the first screw hole, correcting for an error of 0.012mm; move the robotic arm to align the electric screwdriver with the first screw hole; send a tightening command to the PLC, and the PLC controls the screw locking mechanism to perform the locking. After execution, the locking result, final torque, final rotation angle, and tightening process curve data are fed back and transmitted to the industrial control computer; the industrial control computer updates the current workpiece production data, the HMI displays the tightening process curve, and the learning model optimizes the task parameters to 0.42 N·m and 560 rpm. After the industrial control computer verifies and passes the optimization, it saves the optimized task parameters and sends them to the PLC.

[0090] Step 5. Complete the fastening of all 4 screws. All 4 screws are fastened successfully with a cycle time of 10.2 seconds. Set the production data for this workpiece as qualified. Link and save the fastening results and process data of all 4 screws. At the same time, report this production data to the MES system in real time. The MES system can remotely view the real-time production information. The HMI interface updates the production data list and can view the fastening details of each screw on the workpiece. Release the workpiece, and the workpiece automatically flows to the next station. The robot moves to the screw picking position, picks up the screws to fill the gap, and then returns to the ready position, adjusts its working posture, and waits for the next workpiece to arrive.

[0091] Step 6. When 610 units of 55D66H have been produced, the production line will switch the production model to 65D7H. After the workpiece enters the workstation, the factory's SCADA system writes the workpiece arrival signal, workpiece model, and workpiece barcode number to the PLC. The PLC status listener detects the workpiece arrival, reads the workpiece model (65D7H) and barcode number (CS200025199281), calls the vision module to identify the workpiece contour features, and identifies the model as 65D7H. This does not match the current production task parameters, and no process template for model 65D7H is found in the system's process template library. The automatic production task stops, an alarm signal is sent to the PLC, triggering an audible and visual alarm, and the HMI interface displays an alarm pop-up box saying "No process template for the current model".

[0092] Step 7. Upon receiving the system's audible and visual alarm, the operator checks and sees the message "No process template for the current model." They click "Alarm Reset" to disable the alarm, then click the "Create Template" button to enter the new process template interface. The operator enters the template name "65D7H Power Board Locking," the product model "65D7H," selects the screw type "M3×5-7 / 2-Br," selects the tightening parameters "torque 0.42 N·m, speed 500 rpm, pre-rotation time 120 ms," enters the screw hole reliability threshold "0.9," and enters the screw hole obstruction threshold "0.8." Click "Global Photo," drag the red circle on the photo image to align it with the screw holes to identify the screw holes to be fitted, and adjust the order. This model requires fitting 6 screws. Then click "Next." Place the "black on the outside, white on the inside" circular marker on the screw hole surface. Click the "Measure Height" button on the HMI. The robot arm moves sequentially above the screw holes. The vision module identifies the marker features and calculates the working height of each screw hole. After completion, it automatically returns to the standby position, and the HMI updates the height of each screw hole in real time. Click the "Finish" button to save the process template. The entire template configuration takes 2 minutes and 49 seconds.

[0093] Step 8. The operator clicks the "Start Production" button on the HMI to continue automatic production. Steps 3, 4, and 5 are repeated to complete the fastening of all 6 screws. All 6 screws are qualified, with a cycle time of 12.8 seconds.

[0094] Step 9. 1340 workpieces were fastened on the same day without any fault alarms. The industrial control computer automatically collected data (output 1340, 4 defective units, pass rate 99.7%, total number of fastened screws 6820, number of defective fastened screws 6, pass rate 99.9%) and reported it to the MES system. The operator clicked the "Finish" button on the HMI, and the system automatically generated a Word document of the daily production report and sent it to the production process manager via email. After the report was sent, the HMI displayed a 30-second countdown to shutdown. When the countdown ended, the system shut down.

[0095] Although embodiments of the present invention have been described, it will be understood by those skilled in the art that various changes, modifications, substitutions and alterations can be made to these embodiments without departing from the principles and spirit of the present invention, and all such changes and alterations shall not depart from the protection scope of the present invention.

Claims

1. A high-speed intelligent screw machine control system based on artificial intelligence and a robot, characterized in that, It comprises a hardware layer, a software layer and a collaborative control layer, which interact through a data interface to realize automatic control of the whole process of screw locking. The hardware layer is provided with a double-vision recognition structure and an industrial robot, the double-vision recognition structure comprises a global camera installed above a workbench and a local camera installed at the end of the robot, and the hardware layer further comprises a locking execution mechanism module carried at the end of the robot, a feeding module provided with a screw detection sensor, a sensor module, an industrial computer and an industrial control module, and the industrial computer is the core interaction hub of the hardware layer and the software layer. The software layer is provided with a convolutional neural network model and a reinforcement learning model, a process template library, a vision recognition module, an artificial intelligence algorithm module, a motion control module, a locking execution control module, a data storage and analysis module, a task scheduling module and a human-computer interaction module, and each module is decoupled and cooperated through the task scheduling module to realize the whole process software control of workpiece recognition, screw hole positioning, locking parameter optimization and data storage and analysis. The collaborative control layer is provided with a control interface and an MES data reporting interface to realize bidirectional data interaction with the external production line control system and the factory MES system.

2. The high-speed intelligent screw machine control system based on artificial intelligence and robots according to claim 1, wherein the industrial robot in the hardware layer adopts a six-axis robot with a repeat positioning accuracy of ≤±0.02mm for realizing screw grabbing and positioning operation.

3. The high-speed intelligent screw machine control system based on artificial intelligence and robots according to claim 1, wherein the convolutional neural network model is integrated in the vision recognition module for workpiece contour extraction, workpiece model recognition, screw hole position coordinate calculation and attitude angle calculation, the reinforcement learning model is integrated in the artificial intelligence algorithm module for real-time optimization of locking parameters according to locking process data, and the process template library is built in the data storage and analysis module for storing production task parameters of multiple models of workpieces and supporting retrieval and matching.

4. The high-speed intelligent screw machine control system based on artificial intelligence and robots according to claim 1, wherein the control interface in the collaborative control layer is used for receiving workpiece positioning signals, workpiece bar codes, workpiece models and workpiece good product information transmitted by the external production line control system, and the MES data reporting interface in the collaborative control layer is used for uploading real-time production data, statistical data, trend analysis data and abnormal alarm data to the factory MES system. The method comprises the following steps: S1. The system completes hardware self-checking and state initialization, loads intelligent models and historical production task parameters, and enters an automatic production preparation state to wait for workpiece positioning. S2. After detecting the positioning of the workpiece, the global camera is used to collect images, the intelligent model is used to identify the workpiece model and contour coordinates, corresponding production task parameters are matched, screw hole robot coordinates are calculated, and a screw locking task queue is generated. ​ ​ 5. The control method of high-speed intelligent screw machine based on artificial intelligence and robot, applied to the control system of any one of claims 1-4, characterized in that, ​ ​ ​ S3. The local camera is used to sequentially fine-position the screw holes according to the locking task queue, the screw locking is completed by controlling the locking execution mechanism, the locking process data is synchronously collected, the locking parameters are optimized by the intelligent model, and the locking of all screws of the workpiece is completed; S4. After the workpiece locking is completed, the workpiece qualification is determined, the production data is summarized and uploaded to the factory MES system, the robot is controlled to complete the automatic feeding and grabbing of the screws, and is reset to the working posture to wait for the next workpiece; S5. During the production process, various abnormal signals are detected in real time, and after the abnormality is detected, the system is reset to resume production after the abnormality is eliminated; S6. According to the production operation instruction, the automatic production is stopped, each module is reset, the production task is closed loop, the daily production data is counted, and the production report is generated, which is uploaded to the factory MES system to complete the system work.

6. The high-speed intelligent screw machine control method based on artificial intelligence and robot according to claim 5, wherein step S1 specifically comprises: The industrial computer first completes self-checking of its own hardware, then detects the remaining hardware modules one by one through the control module, triggers an alarm and displays the fault position when a fault occurs; after the self-checking is passed, the listeners of each module are initialized, the convolutional neural network model and the reinforcement learning model are loaded, the task scheduler is initialized and the last production task parameters are loaded, after the ready signal is fed back, the control module program is reset, the robot is returned to the initial position, and the lack of material is detected and automatically taken, and the automatic production waiting state is entered.

7. The high-speed intelligent screw machine control method based on artificial intelligence and robot according to claim 5, wherein step S2 further comprises: if the corresponding production task parameters are not matched, an alarm is triggered and a new process template is prompted, and after the new process template is stored in the process template library, the production parameter matching step is re-executed; the production task parameters include workpiece model, screw specification, screw hole position, locking sequence, working height, locking torque, locking speed, screw hole reliability threshold and screw hole shielding threshold.

8. The high-speed intelligent screw machine control method based on artificial intelligence and robot according to claim 7, wherein the way of creating a new process template comprises: inputting the template name, workpiece model, screw model and various production and identification parameters; acquiring the workpiece image through the global camera, circumscribing the screw hole and setting the locking sequence, and calculating the coarse coordinates of the screw hole for the robot; identifying the surface marker features of the screw hole through the local camera, and calculating the actual working height of the screw hole; storing the template parameters in the process template library to complete the creation of the process template.

9. The high-speed intelligent screw machine control method based on artificial intelligence and robot according to claim 5, wherein in step S3, the fine positioning of the screw hole by the local camera specifically comprises: controlling the robot to align the local camera with the screw hole, identifying the screw hole features by the intelligent model, excluding invalid screw holes, calculating the accurate robot coordinates of the electric screwdriver for aligning with the screw hole, controlling the robot to move to make the electric screwdriver accurately align with the screw hole; and the locking parameters are sent to the control module for execution after being reasonably verified by the industrial computer. ​ ​ ​ ​ 10. The control method of the high-speed intelligent screw machine based on artificial intelligence and a robot according to any one of claims 5 to 9, characterized in that, In step S5, the various types of abnormal signals include a screw missing signal, a torque exceeding signal, an air pressure being too low signal, a hardware mechanism movement failure signal, and an image recognition abnormality signal; and the abnormal information includes an abnormality occurrence time, an abnormality type, abnormality data, and an abnormality occurrence position.

Images