Multi-level automated assembly system and control method

Abstract

A kind of multi-level automated assembly system and control method, system includes management layer, control layer and equipment layer, management layer sets task management system and single-point control system, control layer sets electric control system and measurement system, equipment layer sets equipment set;Task management system is connected with electric control system, task management system is used to receive, create and manage assembly task, and assembly task is issued to electric control system to call process flow;Single-point control system is used to control actuator in single point;In precision requirement link, trigger measurement function to check;Electric control system is connected with equipment set simultaneously, and each equipment is driven to complete corresponding action in process flow;Measurement system is connected with image acquisition equipment in equipment set, drives image acquisition equipment to carry out image acquisition, and uses the image collected to calculate measurement result, and measurement result is fed back to electric control system.The automated assembly system of the application has clear architecture, good expansibility and simple implementation.

Description

Technical Field

[0001] This invention belongs to the field of automated assembly, specifically relating to a multi-level automated assembly system and control method. Background Technology

[0002] With the deep integration of intelligent manufacturing and embodied intelligence technology, welding robots have evolved from the traditional teach-and-reproduce model to an intelligent collaborative approach of "multi-sensor perception - real-time decision-making - autonomous execution." In typical industrial scenarios such as welding of engineering machinery structural components, circumferential welding, and steel structure building node welding, achieving high-precision welding of complex workpieces requires integrating multiple types of sensing devices, including vision sensors, force sensors, temperature sensors, and lidar, as well as execution devices such as robotic arms, positioners, wire feeding mechanisms, and protective gas control units, to construct a multi-device collaborative welding system. However, the embodied intelligent welding robot systems currently used in industry are essentially still "simple assembly of equipment," lacking a standardized and modular collaborative architecture. This leads to core problems such as a single control method and poor scalability, specifically manifested in the following two types of solutions: One type is the traditional modular fixed architecture. This solution, designed to meet specific welding task requirements, directly connects sensors and actuators from different manufacturers to the robot controller via customized hardwiring, representing a typical "hardware stacking + function patchwork" model. The devices lack unified interface specifications and data interaction standards, achieving only "physical connection" through independent adapter circuits or dedicated drivers, rather than "intelligent collaboration." The corresponding control method is extremely simplistic; all welding paths and process parameters are preset via the teach pendant. Sensor data is only used for local independent feedback (such as visual correction of weld deviations and force adjustment of welding pressure), not participating in global decision-making, and cannot dynamically adjust the control strategy based on workpiece status and environmental changes. Furthermore, scalability is extremely poor. When adding a laser contour sensor to optimize thick plate welding quality or replacing the dual-wire welding mechanism to improve efficiency, not only must the hardware wiring be redesigned and a dedicated protocol conversion module developed, but the robot controller's underlying program must also be modified. When switching welding workpieces, the teaching path must be completely re-taught, making it unsuitable for multi-variety, small-batch production needs.

[0003] Another type is the preliminary integrated distributed architecture. Although this solution achieves communication interconnection among multiple devices through industrial Ethernet, it still cannot escape the essence of "simple patchwork". The perception, decision-making and execution modules lack a unified architectural specification, and data transmission is only achieved through communication links. It has not formed a deeply collaborative modular design, and its control method is still relatively simple. Although a simple dynamic path planning algorithm is introduced, the core logic still relies on preset rules and can only handle simple disturbances under preset working conditions. When encountering complex situations in the welding process, it cannot quickly adjust the control strategy, resulting in increased weld formation errors. The defects in scalability are also prominent: the functional modules are strongly coupled with the hardware devices. The perception data fusion algorithm and the path planning algorithm are all integrated into the same software platform. When it is necessary to upgrade the embodied intelligence algorithm, the entire software architecture must be reconstructed, resulting in poor compatibility and high upgrade costs. When adding devices such as positioners and automatic control units for protective gas flow, it is necessary to reconfigure communication parameters, modify task scheduling logic, and even adjust the robot kinematic model, which prolongs the system debugging cycle and seriously affects production continuity.

[0004] In summary, existing embodied intelligent welding robot systems have the following prominent problems: (1) The control method is too simple and the adaptive capability is insufficient: The control logic relies on preset teaching or fixed rules, lacks a dynamic decision-making mechanism driven by multi-source data collaboration, and cannot adapt to complex working conditions such as changes in workpiece state and environmental interference, making it difficult to guarantee welding accuracy and stability. (2) Poor scalability, making equipment upgrades and scenario adaptation difficult: The lack of a unified interface standard and modular architecture for multiple devices means that the integration of new devices / sensors requires large-scale hardware and software modifications, resulting in long and costly cycles for process changes and equipment upgrades, which cannot meet the needs of flexible production. (3) The system integration is complex and the maintenance cost is high: The heterogeneous interfaces and coupled modules caused by the patchwork of equipment make system debugging and troubleshooting extremely difficult. When a sensor or actuator fails, it can easily trigger a chain reaction, resulting in long maintenance downtime.

[0005] The aforementioned problems directly limit the large-scale application of embodied intelligent welding robots in the industry, making it difficult to adapt to the actual needs of modern industry for high-precision, high-flexibility, and low-cost production. Summary of the Invention

[0006] The purpose of this invention is to address the problems in the prior art by providing a multi-level automated assembly system and control method. The system has a clear architecture, good scalability, and is simple to implement, thus meeting the needs of flexible production.

[0007] To achieve the above objectives, the present invention provides the following technical solution: In the first aspect, a multi-level automated assembly system is provided, including a management layer, a control layer and an equipment layer. The management layer is equipped with a task management system and a single-point control system, the control layer is equipped with an electrical control system and a measurement system, and the equipment layer is equipped with a set of equipment. The task management system is connected to the electrical control system. The task management system is used to receive, create and manage assembly tasks, and to send assembly tasks to the electrical control system for process flow invocation. The single-point control system is connected to the electrical control system. The single-point control system is used to perform single-point control on the actuators and to send control commands to the electrical control system for single-point control. The electronic control system is connected to the measurement system. When the accuracy requirement is reached, the electronic control system triggers the measurement function for verification. The electrical control system is also connected to the equipment assembly to drive each piece of equipment to complete the corresponding actions in the process flow; The measurement system is connected to the image acquisition device in the equipment set, which drives the image acquisition device to acquire images, calculates the measurement results using the acquired images, and feeds the measurement results back to the electronic control system.

[0008] As a preferred embodiment, the task management system and measurement system are implemented on a personal computer PC, the single-point control system is implemented through configuration software, the electrical control system is implemented through a programmable logic controller (PLC), the image acquisition device is a 2D or 3D camera, and the equipment in the equipment set includes a robotic arm, a gantry crane, and a turntable.

[0009] As a preferred embodiment, the device communication methods of the device set include Modbus, OPCUA, Ethernet, and EnterCAT, and the PLC of the electrical control system supports communication development and expansion.

[0010] As a preferred solution, the task management system and measurement system located on the PC communicate with the electronic control system through OPCUA; the single-point control system implemented by configuration software also communicates with the electronic control system through OPCUA; the measurement system located on the PC is directly connected to the 2D or 3D camera through the transmission control protocol TCP for command interaction and data transmission.

[0011] As a preferred embodiment, the task management system includes the following functional modules: The Manufacturing Execution System (MES) communication module establishes a send-receive mechanism with the upstream management unit's MES, receives overall task control, and reports task execution status, material storage status, and site equipment status in real time. The Automated Guided Vehicle (AGV) docking module establishes communication with external AGVs transporting materials, automatically receives materials and frames transported for assembly, and automatically coordinates with the AGVs to transfer the assembled frames. The task management module manages and controls the scheduling of tasks entering the system, as well as the materials entering the system, and supports manually importing tasks from the local machine. The task execution module controls the start, pause, or stop of tasks, and can monitor and display the progress of the task process; The equipment management module records and displays the equipment status, basic equipment information, alarm logs, and equipment maintenance records within the system. The production report module stores and records completed and incomplete tasks within a certain period, and displays them in the form of reports. The user management module is designed with three user permissions: "Administrator", "Operator" and "Maintenancer", with the level decreasing in that order, so that users can manage personnel permissions as needed. The system settings module enables two functions: communication configuration and database configuration.

[0012] As a preferred embodiment, the single-point control system includes the following functional modules: The step management module allows for the completion of incomplete tasks by controlling individual steps at a single point. The tool control module enables input and output I / O operations on the actuator, facilitating user maintenance. The equipment control module implements control functions, including start-up, reset, power-off, emergency stop, and ground rail movement. The fault alarm module provides alarms for faults and anomalies that occur during task execution or single-point control.

[0013] As a preferred embodiment, the measurement system includes the following functional modules: The coarse positioning module captures and locates the assembly material by matching templates to the image features of the material itself, which is used to address errors that occur during the material loading process. The precise positioning module performs sub-pixel-level positioning based on the edge contour features of the assembled workpiece, and closely adjusts the relative position between the tool and the assembled workpiece to the standard position to ensure consistent gripping. The assembly precision positioning module performs sub-pixel-level position positioning based on the edge contour features of the workpiece to be assembled and the workpiece to be assembled, and adjusts the relative assembly position to ensure the assembly accuracy requirements. The image acquisition module controls a 2D or 3D camera to acquire images via TCP communication; The screw-driving measurement module performs circle fitting and center calculation for the special circular structure of screw holes, guiding the robotic arm's screw-driving operation. The QR code recognition module scans the QR code on the materials entering the system to verify their identification number.

[0014] Secondly, a control method for a multi-level automated assembly system is provided, including the following steps: The task management system distributes assembly tasks to the electrical control system for process flow invocation; A single-point control system sends control commands to the electrical control system to perform single-point control of the actuator; The electrical control system drives each piece of equipment in the assembly to complete the corresponding actions in the process flow; In the accuracy requirement stage, the electronic control system triggers the measurement function of the measurement system for verification. The measurement system drives the image acquisition device to acquire images and uses the acquired images to calculate the measurement results, which are then fed back to the electronic control system.

[0015] As a preferred approach, the following steps are performed in the task management system: Establish a communication mechanism with the upstream management unit's Manufacturing Execution System (MES) to receive overall task control and report task execution status, material storage status, and site equipment status in real time; It establishes communication with the Automated Guided Vehicle (AGV) that transports external materials, receives materials and frames transported for assembly, and works with the AGV to transfer the assembled frames. The system manages and controls tasks entering the system in a unified manner, and also manages materials entering the system, supporting manual import of tasks from local storage. It can control the start, pause, or stop of tasks, and monitor and display the progress of the task process; The system records and displays the device status, basic device information, alarm logs, and device maintenance records within the system. The system stores and records completed and incomplete tasks within a certain period, and displays the statistics in the form of reports. The system is designed with three user permissions: "Administrator", "Operator" and "Maintenancer", with the level decreasing progressively. Personnel permissions are managed according to specific needs. Configure communication and database settings.

[0016] As a preferred approach, the following steps are performed in the measurement system: To address the errors generated during the material loading process, template matching and positioning are performed based on the image features of the assembly materials themselves to achieve coarse positioning for grasping; Based on the edge contour features of the assembled workpiece, sub-pixel level positioning is performed, and the relative position between the tool and the assembled workpiece is adjusted to the standard position to ensure consistent gripping and achieve precise gripping positioning. Based on the edge contour features of the workpiece to be assembled and the workpiece to be assembled, sub-pixel-level positioning is performed, and the relative assembly position is adjusted to ensure the assembly accuracy requirements and achieve precise assembly positioning. Control 2D or 3D cameras to acquire images via TCP communication; For the special circular structure of screw holes, we perform circular fitting and center calculation to guide the robotic arm in screw driving operations; Materials entering the system are numbered and verified by scanning the QR code on the material.

[0017] Compared with the prior art, the present invention has at least the following beneficial effects: This invention constructs a clear hierarchical architecture comprising a management layer, a control layer, and an equipment layer. This layered design avoids the simple stacking of equipment and dispersed control functions common in existing technologies. The task management system and single-point control system in the management layer, along with the electrical control system and measurement system in the control layer, have clearly defined roles and work in a coordinated manner, making the overall system control more centralized and efficient, significantly improving overall system management effectiveness. The hierarchical architecture of this invention has excellent scalability. In the management layer, the task management system can flexibly receive, create, and manage various assembly tasks; the single-point control system can easily add single-point control functions for new actuators; in the control layer, the electrical control system can be adjusted and expanded according to new process requirements, and the measurement system can adapt to more types of measurement tasks; in the equipment layer, the equipment set can be easily added or removed according to actual production needs. This scalability allows the system to quickly adapt to assembly tasks of different scales and complexities, meeting diverse production demands. The electrical control system and measurement system of this invention are closely connected; in areas requiring high precision, the electrical control system can automatically trigger the measurement function for verification. The measurement system connects to the image acquisition devices in the equipment set, drives them to acquire images, accurately calculates the measurement results using the acquired images, and then promptly feeds the results back to the electrical control system. This automated accuracy verification mechanism effectively avoids errors that may arise from manual verification, significantly improves assembly accuracy, and ensures product quality. The management system at the management level can uniformly receive, create, and manage assembly tasks, and accurately distribute these tasks to the electrical control system, which then invokes the corresponding process flow. The single-point control system can perform single-point control on the actuators and distribute control commands to the electrical control system, achieving precise execution of single-point control. The electrical control system is responsible for driving each piece of equipment in the equipment set to complete the corresponding actions in the process flow, ensuring collaborative work between the devices. This unified task management and collaborative equipment control method makes the entire assembly process smoother and more efficient, improving system flexibility. This invention, through its unique system architecture and collaborative work between its components, has significant advantages in improving system flexibility, reliability, assembly accuracy, and scalability, and has broad application prospects and promotional value. Attached Figure Description

[0018] To more clearly illustrate the technical solutions of the embodiments of the present invention, the drawings used in the description of the embodiments will be briefly introduced below. Obviously, the following drawings are only some of the embodiments of the present invention. For those skilled in the art, other embodiments can be obtained based on these drawings without creative effort.

[0019] Figure 1 A schematic diagram of the main design architecture of the multi-level automated assembly system according to an embodiment of the present invention; Figure 2 A schematic diagram of the module composition of a multi-level automated assembly system according to an embodiment of the present invention. Detailed Implementation

[0020] The technical solutions of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments.

[0021] In existing automated assembly systems, due to the simple assembly of equipment, the lack of hierarchical design in the system architecture, and the high coupling of control logic, it is difficult to achieve independent expansion and maintenance of functional modules. This problem leads to the need for global adjustments when facing the integration of new equipment or changes in processes, which significantly reduces the flexibility and maintainability of the system and affects production efficiency and system stability.

[0022] For example, in the automated assembly process of electronic products, when it is necessary to integrate a new 3D camera into an existing assembly line to improve positioning accuracy, the task management, equipment control and measurement functions must be reconfigured because the control system has a single architecture. This leads to complicated system debugging, interruption of the assembly process, and inability to guarantee the collaborative work between the new equipment and the original equipment.

[0023] If the above problems are not resolved, the system will be unable to adapt to the needs of technological development, and will face high-risk system reconstruction when equipment is updated or functions are expanded, increasing maintenance difficulty and downtime, thereby affecting the overall production continuity and product quality consistency.

[0024] For this purpose, please refer to Figure 1 This invention proposes a multi-level automated assembly system, including a management layer, a control layer, and an equipment layer. The management layer is equipped with a task management system and a single-point control system, the control layer is equipped with an electrical control system and a measurement system, and the equipment layer is equipped with a set of equipment. The task management system is connected to the electrical control system. The task management system is used to receive, create and manage assembly tasks, and to send assembly tasks to the electrical control system for process flow invocation. The single-point control system is connected to the electrical control system. The single-point control system is used to perform single-point control on the actuators and to send control commands to the electrical control system for single-point control. The electronic control system is connected to the measurement system. When the accuracy requirement is reached, the electronic control system triggers the measurement function for verification. The electrical control system is also connected to the equipment assembly to drive each piece of equipment to complete the corresponding actions in the process flow; The measurement system is connected to the image acquisition device in the equipment set, which drives the image acquisition device to acquire images, calculates the measurement results using the acquired images, and feeds the measurement results back to the electronic control system.

[0025] In the multi-level automated assembly system proposed in the above embodiments of the present invention, the management layer is configured as the highest level of the system, mainly responsible for macro-level task scheduling and system management functions. It integrates a task management system and a single-point control system to realize the planning and intervention of the entire assembly process.

[0026] The control layer is configured as an intermediate level between the management layer and the equipment layer. Its primary responsibility is to receive instructions from the management layer and translate them into actions executable by the equipment layer, while also processing feedback information from the equipment layer. This layer integrates the electrical control system and the measurement system to achieve precise equipment control and process monitoring.

[0027] The device layer is configured as the lowest level of the system and interacts directly with the physical devices. This layer contains a collection of devices used to perform specific assembly operations, such as material handling and component installation.

[0028] The task management system is configured as a core component of the management layer, with main functions including receiving, creating, and managing assembly tasks. This system is responsible for issuing planned assembly tasks to the electrical control system to invoke the corresponding process flows.

[0029] The single-point control system is configured as an auxiliary control component in the management layer, primarily used for independent and precise control of actuators. This system can issue control commands to the electronic control system to achieve single-point operation of specific actuators.

[0030] The electronic control system is configured as the core control unit in the control layer, responsible for receiving instructions from the task management system and the single-point control system, and converting them into drive signals for each device in the equipment set. This system is also responsible for triggering measurement functions for verification at points requiring high precision.

[0031] The measurement system is configured as a quality control unit in the control layer. Its main function is to drive the image acquisition device to acquire images, process the acquired images, and calculate the measurement results. These results are then fed back to the electronic control system for process adjustment or quality judgment.

[0032] The equipment assembly is configured as physical execution units in the equipment layer, containing various devices used to complete the assembly process. These devices execute corresponding actions through the drive of an electrical control system.

[0033] Image acquisition devices are configured as visual sensing units within the measurement system to capture image information during the assembly process. These images form the basis for the measurement system's accuracy verification and result calculations.

[0034] Actuators are configured as specific operational units within a set of equipment, such as robotic arms or grippers, which directly perform physical actions to complete specific steps in the assembly task.

[0035] The process flow is configured as a series of operational steps and sequences required to complete a specific assembly task. The electrical control system, based on the tasks assigned by the task management system, invokes the corresponding process flow to guide the operation of the equipment assembly.

[0036] This embodiment of a multi-level automated assembly system is designed with a structure including a management layer, a control layer, and an equipment layer. This layered architecture facilitates modular management and control of the system. The management layer is configured as the top layer of the system, primarily responsible for macro-level task scheduling and system management. For example, the management layer can be implemented as a software platform based on a general-purpose computer, receiving task instructions input by the operator through a user interface or obtaining task information through file import. The control layer is configured as a bridge between the management layer and the equipment layer, responsible for translating the instructions from the management layer into specific actions at the equipment layer. For example, the control layer can be implemented as an industrial controller, such as a programmable logic controller (PLC), which runs preset control logic internally. The equipment layer is configured as the bottom layer of the system, directly interacting with the physical equipment. For example, the equipment layer can contain various actuators, such as robotic arms, conveyors, and fixtures, which are connected to the control layer via cables or wirelessly.

[0037] Within the management layer, a task management system and a single-point control system are configured. The task management system is configured to receive, create, and manage assembly tasks. For example, it can provide a task list interface, allowing operators to manually enter new assembly tasks or load predefined task templates from local storage. This system also distributes assembly tasks to the electrical control system to invoke the corresponding process flow. For instance, the task management system can send task parameters and process flow numbers to the electrical control system via a standard data interface. The single-point control system is configured for single-point control of actuators. For example, it can provide a simple button or slider interface, allowing operators to directly control the movement of a single joint of a robotic arm or the opening and closing of a gripper. This system sends control commands to the electrical control system to achieve single-point control of the actuators. For example, the single-point control system can send specific control words or signals to the electrical control system to instruct it to execute a preset single-point action.

[0038] The control layer includes an electrical control system and a measurement system. The electrical control system is configured to connect to the task management system, receiving assembly tasks and invoking corresponding process flows based on the task content. For example, the electrical control system can select the corresponding process from its internally stored process flow library based on the received task ID and begin execution. The electrical control system is also configured to connect to a single-point control system, receiving single-point control commands and driving corresponding actuators to complete single-point control actions. For example, when the single-point control system sends a "lift robotic arm" command, the electrical control system will drive the corresponding motor of the robotic arm to perform the lifting action. The electrical control system is also connected to the equipment assembly to drive each piece of equipment to complete corresponding actions in the process flow. For example, the electrical control system can control the start and stop of the conveyor belt through output signals or control the movement trajectory of the robotic arm through pulse signals.

[0039] The measurement system is configured to connect to the electronic control system. The electronic control system triggers the measurement function for verification at accuracy-critical stages. For example, during assembly, once a critical component is placed in position, the electronic control system sends a trigger signal to the measurement system, instructing it to perform positional accuracy measurement. The measurement system is also configured to connect to an image acquisition device within the equipment set to drive the image acquisition device to acquire images. For example, the measurement system can send a shutter command to the image acquisition device to capture an image of the current scene. The measurement system uses the acquired images to calculate the measurement results. For example, the measurement system can perform image processing algorithms such as edge detection and feature matching to calculate the actual position or size of the component. The calculated measurement results are then fed back to the electronic control system. For example, the measurement system can send the measurement data to the electronic control system via a data bus for judgment and decision-making.

[0040] Compared to existing technologies that rely on simple, piecemeal equipment and limited control methods in industrial settings, this invention demonstrates significant technological contributions. Firstly, by introducing a layered architecture comprising a management layer, a control layer, and an equipment layer, this system achieves clear division of responsibilities and modular design. The management layer is responsible for macro-level task management and scheduling, the control layer for instruction conversion and execution, and the equipment layer focuses on physical operations. This layered structure effectively addresses the problems of unclear architecture and poor scalability in existing systems, enabling more flexible configuration and expansion when faced with changes in product models or process flows.

[0041] Secondly, the close integration of the task management system and the electrical control system automates and intelligently manages the receipt, creation, and management of assembly tasks, as well as the invocation of process flows. This overcomes the limitations of the single control method in existing systems, improving production efficiency and the accuracy of task execution. For example, the task management system can automatically create and issue tasks based on orders, eliminating the need for frequent manual intervention and significantly improving the level of production automation. Furthermore, the introduction of the single-point control system provides a convenient and independent control method for the debugging, maintenance, and emergency intervention of the actuators. This ensures the continuity of automated production while also improving the maintainability and safety of the system. For example, maintenance personnel can precisely move the robotic arm through the single-point control system without interrupting the entire automated process.

[0042] Furthermore, the integration of the electronic control system and the measurement system, especially the triggering of measurement functions for verification in areas requiring high precision, and the use of image acquisition equipment for measurement result calculation and feedback, greatly enhances the quality control capabilities during the assembly process. This solves the problem of existing systems lacking effective real-time quality monitoring methods, ensuring the accuracy and consistency of product assembly.

[0043] In summary, this embodiment, through its unique multi-level architecture design and collaborative working mechanism between systems, provides an automated assembly solution with a clear architecture, flexible control, good scalability, and real-time quality control capabilities, significantly improving the efficiency, accuracy, and reliability of industrial automated assembly.

[0044] In actual deployment and operation, if the specific composition of the implementation platform and equipment set of each system module is not clearly defined, problems such as complex system integration, difficulty in performance optimization, and improper equipment selection may be encountered, thereby affecting the overall stability, response speed and assembly accuracy of the system.

[0045] In this regard, this application further proposes a specific implementation method of the above-mentioned multi-level automated assembly system, wherein the task management system and measurement system are implemented on a personal computer PC, the single-point control system is implemented through configuration software, the electrical control system is implemented through a programmable logic controller (PLC), the image acquisition device is a 2D or 3D camera, and the equipment in the equipment set includes a robotic arm, a gantry, and a turntable.

[0046] Specifically, the task management and measurement systems are implemented on personal computers (PCs), meaning these systems can fully utilize the powerful computing capabilities, flexible software development environment, and rich graphical user interface of PCs. For example, the task management system can run on a PC as a desktop application or a web-based application, handling complex task scheduling, data management, and user interaction. Similarly, the measurement system can utilize PCs for high-precision image processing, complex algorithm calculations, and measurement result analysis. The single-point control system is implemented using configuration software, a commonly used human-machine interface (HMI) and supervisory control and data acquisition (SCADA) system development platform in industry. It provides a graphical configuration environment, enabling users to intuitively design control interfaces and logic, achieving direct and flexible control of actuators, and facilitating debugging and maintenance. The electrical control system is implemented using programmable logic controllers (PLCs). PLCs are core controllers in industrial automation, ensuring the precise execution of assembly processes and the stable operation of equipment assemblies through their high reliability, real-time performance, and anti-interference capabilities. Image acquisition equipment can be either 2D or 3D cameras. 2D cameras are typically used for planar positioning, dimensional measurement, and defect detection, while 3D cameras can acquire three-dimensional spatial information of objects, enabling more precise spatial positioning and complex shape recognition. The equipment suite includes robotic arms, gantry cranes, and turntables. The robotic arms (industrial robots) have multiple degrees of freedom and can perform precise grasping, placement, and tightening operations. Gantry cranes are typically used for large-scale, high-precision material handling or workpiece positioning. Turntables are used to change the posture of workpieces or switch between different workstations to adapt to multi-angle assembly requirements.

[0047] This application deploys the task management and measurement systems on a personal computer (PC), enabling the system to handle complex task logic and high-precision visual data. Simultaneously, the single-point control system implemented through configuration software provides operators with an intuitive and convenient interface for equipment operation and debugging. The electrical control system employs a programmable logic controller (PLC), ensuring the real-time performance and stability of industrial control, thereby reliably driving the robotic arms, gantry cranes, and turntables in the equipment assembly to complete assembly actions. When the task management system receives an assembly task on the PC, it sends it to the PLC. The PLC drives the robotic arms, gantry cranes, and turntables in the equipment assembly to work collaboratively according to the process flow. For example, the gantry crane moves the workpiece to a designated area, the robotic arm performs gripping and initial positioning, and the turntable adjusts the workpiece's posture. In stages requiring high-precision verification, the electrical control system triggers the measurement system on the PC, which drives a 2D or 3D camera to acquire images. The acquired images are processed on the PC to calculate accurate measurement results, which are then fed back to the electrical control system. The PLC fine-tunes the actions of the equipment assembly based on the measurement results to meet accuracy requirements. Furthermore, operators can directly send commands to the electrical control system through a single-point control system implemented with configuration software to individually control the robotic arm, gantry, or turntable, such as performing manual resets or testing specific steps. This layered and specific implementation optimizes the functions of each level: the PC handles high-level management and complex calculations, the PLC handles low-level real-time control, the configuration software provides convenient human-machine interaction, and the specific automated equipment performs physical operations, together forming a highly efficient, precise, and easy-to-operate automated assembly system.

[0048] The following is a concrete example: the task management system and measurement system can be deployed on an industrial PC running a Windows operating system. The task management system uses a desktop application developed based on the .NET framework, while the measurement system uses vision software such as LabVIEW or Halcon for image processing and measurement algorithm implementation. The single-point control system can be developed using Siemens WinCC configuration software, which, through its graphical interface and scripting functions, enables direct operation of actuators such as robotic arms, gantry cranes, and turntables. The electrical control system can use a Siemens S7-1500 series PLC, programmed using TIA Portal software, responsible for the logic control, equipment driving, and safety interlocks of the entire assembly process. The image acquisition equipment can be specifically configured as a Basler ace series industrial 2D camera for planar positioning and dimensional inspection, and a Keyence LJ-V series laser profilometer (as a type of 3D camera) for acquiring the three-dimensional height information of the workpiece. The robotic arm in the equipment set can be an ABB IRB 1200 six-axis robot, the gantry can be a three-axis gantry structure built with linear modules driven by servo motors, and the turntable can be a high-precision servo turntable, such as a rotary platform driven by a Harmonic Drive reducer.

[0049] By deploying the task management and measurement systems on personal computers (PCs) using the aforementioned technical solutions, the powerful computing capabilities and flexible software development environment of PCs can be fully utilized to achieve complex task scheduling, data analysis, and high-precision visual measurement algorithms, thereby improving the system's intelligence level and data processing efficiency. Single-point control systems are implemented through configuration software, providing operators with an intuitive and convenient human-machine interface, greatly simplifying equipment debugging, maintenance, and troubleshooting, and improving system operability. The electrical control system uses a programmable logic controller (PLC), ensuring the real-time performance, stability, and reliability of industrial control, effectively handling complex industrial conditions and guaranteeing precise execution of the assembly process. Image acquisition equipment uses 2D or 3D cameras, combined with robotic arms, gantry cranes, and turntables, enabling the system to flexibly perform high-precision positioning, grasping, and assembly operations according to different assembly requirements, significantly improving assembly accuracy and efficiency. These specific implementation methods work together to effectively solve problems such as complex system integration, difficult performance optimization, and inappropriate equipment selection, resulting in a multi-level automated assembly system with higher stability, faster response speed, and superior assembly quality.

[0050] This application proposes a multi-level automated assembly system, comprising a management layer, a control layer, and an equipment layer, and specifies the main components and some communication methods of each layer. For example, the task management system and measurement system are implemented on a personal computer (PC), the single-point control system is implemented through configuration software, the electrical control system is implemented through a programmable logic controller (PLC), and the equipment in the equipment set includes robotic arms, gantry cranes, and turntables. However, in actual automated assembly environments, the equipment layer often contains various types of equipment from different manufacturers, which may use different communication protocols. If the electrical control system cannot effectively support multiple communication protocols, or lacks communication development and expansion capabilities, it will be difficult to achieve smooth integration and efficient collaboration among the devices within the equipment layer, thus affecting the flexibility, compatibility, and ease of future upgrades of the entire system.

[0051] In this regard, this application further proposes that the device communication methods for the aforementioned device set include Modbus, OPCUA, Ethernet, and EnterCAT, and that the PLC of the electrical control system supports communication development and expansion. Modbus is a serial communication protocol widely used in industrial automation. It can be implemented through RS-232 or RS-485 physical interfaces, or by transmitting data over Ethernet via the Modbus TCP / IP protocol. OPCUA (OPC Unified Architecture) is a platform-independent, service-oriented architecture used for data exchange in industrial automation. It provides secure, reliable, and cross-platform communication capabilities, supports complex data types and information models, and can be implemented by integrating it into the device firmware through a software library, or by data interaction through OPC UA server / client software. Ethernet is a widely used local area network technology, often used in industry for high-speed, high-capacity data transmission, such as connecting devices through standard Ethernet interfaces (e.g., RJ45) and communicating using the TCP / IP protocol stack. EnterCAT is a high-performance, real-time Ethernet fieldbus protocol widely used in motion control and automation due to its high speed, high precision, and low latency. Its implementation typically requires devices to integrate EtherCAT slave controllers, with network management and data exchange handled by an EtherCAT master controller. The PLC of the electrical control system supports communication development and expansion, meaning that the PLC has the ability to interact with devices using different communication protocols through programming or configuration. This is usually achieved through the PLC's built-in communication modules, function blocks, or library files, allowing engineers to write communication programs according to specific device protocols. Furthermore, expansion capability refers to the PLC's ability to increase or upgrade the types and number of supported communication protocols by adding hardware modules (such as Ethernet modules, serial communication modules, and fieldbus modules) or software function packages.

[0052] This application's solution employs multiple industrial communication protocols, such as Modbus, OPCUA, Ethernet, and EnterCAT, within the equipment suite, and ensures that the PLC of the electrical control system possesses communication development and expansion capabilities. This enables the electrical control system to act as the core controller, effectively managing and coordinating diverse devices at the equipment layer. The PLC can establish stable and reliable communication links based on the communication protocols of different devices through its built-in communication interfaces or expansion modules. For example, for robotic arms requiring high real-time performance, the PLC can perform precise motion control via the EtherCAT protocol; for general data exchange, Ethernet or OPCUA can be used; and for some traditional or simple sensors or actuators, Modbus can be used for control. This multi-protocol support capability, combined with the PLC's communication development and expansion features, ensures that the electrical control system can seamlessly integrate devices from different manufacturers using different communication standards, thereby achieving unified scheduling and precise control of the entire equipment suite. The electrical control system drives each device to complete the corresponding actions in the process flow and triggers measurement functions for verification at precision-required stages. The entire process of instruction issuance, data acquisition, and status feedback relies on these diverse communication protocols, collectively constructing a flexible, efficient, and scalable automated assembly system.

[0053] The following is a concrete example. In an automated assembly system, the equipment assembly might include a precision robotic arm using the EtherCAT protocol, a gantry controlled via the EtherNet / IP protocol, a turntable using the Modbus TCP protocol, and a vision inspection system exchanging data with a PLC via OPCUA. As the core of the electrical control system, a Programmable Logic Controller (PLC) is selected. The PLC itself may integrate an Ethernet interface and support adding other communication functions through expansion modules. For high-speed real-time communication with the EtherCAT robotic arm, the PLC can be configured with an EtherCAT master communication module. For the EtherNet / IP gantry, the PLC exchanges data and performs control via its standard Ethernet interface using the corresponding communication function blocks. The Modbus TCP turntable can communicate with the PLC through the same Ethernet network, with the PLC configured as a Modbus TCP client. Simultaneously, the PLC's built-in OPCUA server function is activated for data interaction with the measurement system on the PC. Through PLC programming software, engineers can write or configure corresponding communication programs for each protocol, achieving unified management and coordinated control of all equipment.

[0054] Through the above technical solution, this application significantly improves the compatibility, flexibility, and scalability of multi-level automated assembly systems at the equipment level. Because the PLC of the electrical control system supports multiple mainstream industrial communication protocols such as Modbus, OPCUA, Ethernet, and EnterCAT, and possesses communication development and expansion capabilities, the system can easily integrate devices from different manufacturers using different communication standards, effectively solving the integration challenges caused by the complexity of device types and inconsistent communication protocols at the equipment level. This not only simplifies the system integration process and reduces integration costs, but also enables the system to adapt to future technological developments and equipment upgrades without requiring large-scale modifications to the entire control architecture. Therefore, the solution of this application ensures efficient collaborative work between components at the equipment level, improving the operating efficiency, stability, and lifecycle value of the entire automated assembly system.

[0055] In the aforementioned multi-level automated assembly system, the various systems and devices in the management, control, and equipment layers are distributed across different hardware platforms and software environments. For example, the task management and measurement systems are implemented on a personal computer (PC), single-point control systems are implemented through configuration software, electrical control systems are implemented through programmable logic controllers (PLCs), and the equipment layer includes 2D or 3D cameras. Communication between these heterogeneous systems and devices faces challenges in terms of compatibility, real-time performance, and data transmission efficiency. If the communication mechanism is unclear or inconsistent, it may lead to data interaction delays and instruction execution errors, thereby affecting the stability and accuracy of the entire assembly system.

[0056] In this regard, this application further proposes that the PC-based task management system and measurement system communicate with the electronic control system through OPCUA; the single-point control system implemented through configuration software also communicates with the electronic control system through OPCUA; and the measurement system located on the PC is directly connected to the 2D or 3D camera through the transmission control protocol TCP for command interaction and data transmission.

[0057] OPCUA (Open Platform Communications Unified Architecture) is an open, cross-platform, service-oriented architecture for data exchange in industrial automation. It provides a secure, reliable, and vendor-independent communication method, enabling data interoperability between different systems and devices. As a unified communication protocol, OPCUA solves communication compatibility issues between heterogeneous systems, supports multiple data types, and provides security mechanisms such as data encryption and user authentication. It can be implemented by integrating OPCUA client modules into single-point control systems implemented in PC-based task management systems, measurement systems, and configuration software, and integrating OPCUA server modules into the PLC of the electrical control system, enabling data subscription and read / write operations between the client and server. Alternatively, it can be implemented by deploying a separate OPCUA gateway or proxy server to convert data from different systems into OPCUA format for transmission, thereby simplifying the integration complexity of each system.

[0058] TCP (Transmission Control Protocol) is a connection-oriented, reliable, byte-stream-based transport layer communication protocol. It ensures the sequential, complete, and error-free transmission of data packets and is widely used in scenarios requiring highly reliable data transmission. TCP communication provides a stable and reliable data channel between a PC-based measurement system and a 2D or 3D camera, ensuring the accurate transmission of image data and control commands, which is crucial for high-precision measurement. This can be achieved by having the measurement system act as a TCP client, establishing a connection with the 2D or 3D camera as a TCP server, and using socket programming to send commands and receive image data. Alternatively, it can be implemented using the camera manufacturer's SDK (Software Development Kit), which typically encapsulates a TCP-based communication interface, simplifying the development process and allowing the measurement system to directly call APIs for command interaction and data transmission.

[0059] "Direct connection" refers to data transmission between two devices without intermediate protocol conversion layers or additional network equipment, but rather establishing communication directly through a physical link and corresponding protocols. A direct connection between a measurement system and a 2D or 3D camera can reduce potential latency and points of failure in the communication link, improving the real-time performance and reliability of data transmission, and is particularly suitable for time-sensitive image acquisition and control command interaction. This can be achieved by directly connecting the PC-side measurement system to the 2D or 3D camera via an Ethernet cable, configuring both with IP addresses within the same subnet, thus achieving point-to-point network communication. Alternatively, it can be achieved by using a network card supporting Power over Ethernet (PoE) and the camera, providing power to the camera while simultaneously transmitting data, further simplifying wiring and connections.

[0060] This application's solution introduces a standardized communication protocol to construct an efficient and reliable communication architecture, addressing the communication challenges between heterogeneous systems in a multi-level automated assembly system. Specifically, the PC-based task management system and measurement system, as well as the single-point control system implemented through configuration software, all communicate with the electrical control system using the OPCUA protocol. The adoption of the OPCUA protocol enables these systems, located on different platforms and with different functional roles, to interact with the electrical control system, which serves as the core control unit, in a unified, secure, and reliable manner. The task management system accurately sends assembly tasks and process flow instructions to the electrical control system via the OPCUA protocol and receives task execution status; the single-point control system sends precise actuator control instructions to the electrical control system via the OPCUA protocol. This unified communication method greatly simplifies system integration, improves data transmission interoperability and real-time performance, and ensures accurate transmission and execution of instructions from the management and control layers. Furthermore, to meet the real-time and reliability requirements of image data transmission in high-precision measurement, the PC-based measurement system directly connects with the 2D or 3D cameras in the equipment set using the TCP transmission control protocol. The connection-oriented and reliable transmission characteristics of the TCP protocol ensure that image acquisition commands are accurately sent to the camera, while the high-resolution image data acquired by the camera is transmitted back to the measurement system for processing in a complete and orderly manner. This direct connection method reduces intermediate links and communication latency, ensuring that the measurement system can acquire accurate image information in a timely manner, thus providing reliable data support for the electrical control system to trigger measurement functions for verification in areas requiring high precision. By using the OPCUA protocol to achieve unified communication between the management and control layers, and the TCP protocol to achieve high-speed and reliable communication between the measurement system and the image acquisition device, this solution effectively solves the challenges of communication compatibility, real-time performance, and data transmission efficiency between heterogeneous systems, ensuring the efficient and stable operation of the entire multi-level automated assembly system.

[0061] In one specific implementation, the PC-based task management system and measurement system can run on the same industrial PC, which is connected to the PLC containing the electrical control system via an Ethernet cable. This industrial PC can be equipped with software modules supporting OPCUA client functionality for data interaction with the OPCUA server integrated into the PLC. For example, the task management system can send a "start assembly task A" command to the PLC via the OPCUA protocol and subscribe to variables in the PLC representing the "completion status of task A". Simultaneously, the single-point control system can be implemented using configuration software (such as Siemens WinCC, Rockwell FactoryTalk View, etc.), which also has an OPCUA client interface and can send control commands such as "move the robotic arm to the specified position" to the PLC via the OPCUA protocol. For communication between the measurement system and the image acquisition device, the PC-based measurement system can be directly connected to an industrial-grade 2D or 3D camera via another Ethernet card. This camera can be configured as a TCP / IP-enabled device and assigned a fixed IP address. The image acquisition module in the measurement system can establish a TCP connection with the camera using programming languages ​​such as C# or Python through a Socket programming interface. For example, when the electronic control system triggers the measurement function, the measurement system sends a "acquire an image" command to the 2D camera via a TCP connection. Upon receiving the command, the camera immediately acquires the image and streams the raw image data back to the measurement system via TCP. After receiving the image data, the measurement system can then perform subsequent image processing and measurement result calculations.

[0062] Through the above technical solutions, this application effectively solves the problems of compatibility, real-time performance, and reliability of communication between different levels and platform systems in a multi-level automated assembly system. The introduction of the OPCUA protocol provides a unified, secure, and efficient data exchange standard between the PC-based task management system, measurement system, and single-point control system implemented with configuration software, and the electrical control system. This avoids integration difficulties and data conversion overhead caused by protocol incompatibility, significantly improving the efficiency and accuracy of management command issuance and status feedback. Simultaneously, the measurement system uses a direct TCP connection with the 2D or 3D camera, ensuring the real-time performance and integrity of high-precision image data transmission, reducing the risk of communication delays and data loss, thereby guaranteeing the accuracy and timeliness of verification at precision-critical stages. This optimized communication architecture enables the entire automated assembly system to work more stably and efficiently, improving the automation level of the assembly process and product quality.

[0063] In some implementations, the task management system resides on a personal computer (PC) and is used to receive, create, and manage assembly tasks, as well as to distribute these tasks to the electrical control system for process flow invocation. However, in real-world, complex automated assembly scenarios, simply possessing basic task reception, creation, and management functions is insufficient to meet the demands for deep integration with the upstream Manufacturing Execution System (MES), automated material flow, refined task control and monitoring, comprehensive equipment status monitoring, production data statistical analysis, and multi-user access control. This can lead to problems such as information silos, inefficient material flow, opaque task execution processes, untimely equipment maintenance, and chaotic operating permissions, thereby affecting the overall efficiency, flexibility, and reliability of the assembly system.

[0064] For this, please refer to Figure 2 This application further proposes that the task management system includes the following functional modules: MES communication module, AGV docking module, task management module, task execution module, equipment management module, production report module, user management module, and system settings module.

[0065] The MES communication module is responsible for data interaction between the task management system and the Manufacturing Execution System (MES) of the upstream management unit. Its function is to establish a stable communication link, receive overall task control instructions and production plans from the MES, and promptly feed back key data such as task execution status, material storage status, and equipment status at each station to the MES. Implementation methods may include, but are not limited to: data subscription and publishing via standardized interface protocols (such as OPC UA, MQTT); or bidirectional data transmission via customized APIs (Application Programming Interfaces). The Automated Guided Vehicle (AGV) docking module manages communication and coordination between the task management system and external AGVs transporting materials. Its main function is to automatically receive materials and frames transported by AGVs to the assembly station, and after the assembly task is completed, automatically coordinate the AGVs to transfer the assembled frames to the next process or storage area. Implementation methods may include, but are not limited to: information exchange with the AGV scheduling system via wireless communication technologies (such as Wi-Fi, Bluetooth); or simple status handshakes and instruction interactions with AGVs via preset I / O signals. The task management module is the core of the task management system, responsible for the unified scheduling, management, and control of all assembly tasks entering the system. It not only manages the task lifecycle but also tracks and manages materials entering the system. Furthermore, this module allows users to manually import new assembly tasks from their local machines. Implementation methods may include, but are not limited to: database-based task queue and state machine management; or using priority scheduling algorithms for task scheduling. The task execution module provides fine-grained control and real-time monitoring of the assembly task execution process. It allows users to start, pause, or stop ongoing tasks and displays the current progress of the task in real time. Implementation methods may include, but are not limited to: obtaining task status through real-time data interaction with the electrical control system; or visually displaying the task flowchart and current execution steps through a graphical user interface (GUI). The equipment management module centrally manages the operating status, basic information, alarm logs, and maintenance records of all equipment within the system. It can obtain equipment operating parameters in real time, record faults and anomalies, and store equipment maintenance history for easy user query and management. Implementation methods may include, but are not limited to: establishing an equipment information database for unified storage; or collecting equipment data in real time and displaying it visually via protocols such as OPC UA. The production report module is responsible for the unified storage, recording, and statistical analysis of completed and incomplete assembly tasks within a specific period. It can generate and display this data in report form, providing data support for production management. Implementation methods may include, but are not limited to: periodically extracting and summarizing data from the task database; or using business intelligence (BI) tools to generate customized production reports.The user management module aims to implement hierarchical management of system user permissions. It designs three different levels of user permissions: "Administrator," "Operator," and "Maintenancer," with permission levels decreasing progressively. This ensures that users with different roles can only access and operate functions within their scope of responsibility, thereby improving system security and controllability. Implementation methods may include, but are not limited to, role-based access control (RBAC) mechanisms; or management through user authentication and authorization services. The system settings module provides functions for adjusting core system configuration parameters, mainly including communication configuration and database configuration. Communication configuration allows users to set connection parameters with external systems (such as MES, AGV, and electrical control systems), while database configuration is used to manage the database connection information used by the system. Implementation methods may include, but are not limited to, storing configuration parameters through configuration files (such as XML and JSON); or providing a graphical interface for users to modify and save parameters.

[0066] In the aforementioned multi-level automated assembly system, the task management system integrates the aforementioned functional modules to achieve comprehensive and refined management of the entire assembly process. Specifically, the MES communication module first establishes a connection with the upstream MES system, receives production instructions and task plans, and provides real-time feedback on the system's production status. These received tasks are then uniformly scheduled and managed by the task management module to ensure orderly task execution and support manual import to address special needs. When a task requires materials, the AGV docking module communicates with the AGV system to coordinate the automatic transportation and receipt of materials, and arranges the transfer of finished products after assembly. The task execution module is responsible for starting, pausing, or stopping specific assembly tasks according to the schedule, and monitors the task progress in real time through interaction with the PLC system to ensure that the assembly process proceeds as planned. Simultaneously, the equipment management module continuously monitors the operating status of equipment such as robotic arms, gantry cranes, and turntables in the equipment set, records alarm information and maintenance history, and provides equipment support for task execution. The production report module collects, stores, and statistically analyzes the execution status of all tasks, generating visual reports to provide a basis for production decisions. The user management module uses hierarchical access control to ensure that different roles (administrators, operators, and maintainers) can only perform operations within their designated areas of responsibility, enhancing system security and stability. The system settings module provides flexible configuration capabilities, allowing users to adjust communication parameters and database connections according to actual needs, ensuring system adaptability and maintainability. Through the collaborative operation of these modules, the task management system can not only efficiently process assembly tasks but also achieve seamless integration with external systems and equipment, forming a highly automated, intelligent, and easily manageable assembly production environment. This effectively solves the problems of low integration, imprecise management, and opaque data faced by traditional task management systems in complex automation scenarios.

[0067] The following is a concrete example. As a specific implementation, the task management system can be deployed on an industrial PC running a Windows operating system. The MES communication module can adopt an OPC UA client / server architecture, exchanging data with the upstream MES system via the OPC UA protocol. For example, it receives information such as production order numbers, product models, and quantities from the MES, and reports data such as the current workstation's production cycle time, yield rate, and equipment utilization rate. The Automated Guided Vehicle (AGV) docking module can integrate the SDK (Software Development Kit) provided by the AGV manufacturer, communicating with the AGV scheduling system via Ethernet, sending material request instructions, and receiving AGV arrival and departure status information. The task management module can store task information, bills of materials, and scheduling rules based on a relational database (such as SQL Server or MySQL), and provide a web-based user interface for operators to view, edit, and manually import tasks. The task execution module can be designed using a state machine pattern, interacting with the PLC via the Modbus TCP / IP protocol. For example, it sends a "start process X" command and receives feedback signals such as "process X completed" or "process X failed," while simultaneously displaying task progress in real-time on the user interface in Gantt chart format. The equipment management module can periodically collect operational data (such as motor current, temperature, and fault codes) from the equipment set, including robotic arms, gantry cranes, and turntables, via the Ethernet / IP protocol, and store it in a database. Alarms are triggered when an anomaly is detected. The production reporting module can use open-source reporting tools (such as Apache FOP) to generate daily, weekly, and monthly reports in PDF format based on preset templates and automatically send them to relevant personnel via email. The user management module can use the OAuth 2.0 protocol for user authentication and authorization, assigning different roles to administrators, operators, and maintenance personnel. For example, administrators can modify system configurations, operators can only start / pause tasks, and maintenance personnel can view equipment logs. The system settings module provides a graphical configuration interface that allows administrators to modify parameters such as the OPC UA server address and database connection string, and supports importing and exporting configuration files.

[0068] Through the aforementioned technical solution, the task management system, through the synergistic effect of its internal MES communication module, AGV docking module, task management module, task execution module, equipment management module, production report module, user management module, and system settings module, significantly enhances the overall management capabilities of the multi-level automated assembly system. This solution achieves seamless integration with the upstream MES system, ensuring accurate production planning and real-time data feedback, eliminating information silos. Simultaneously, the AGV docking module enables automated and precise material flow, significantly improving material handling efficiency and reducing manual intervention. The task management and execution modules provide refined scheduling, control, and monitoring of assembly tasks, making the production process more transparent and controllable. The equipment management module ensures the stability and maintainability of equipment operation, effectively preventing and quickly responding to equipment failures. The production report module provides comprehensive data support for production decisions, promoting continuous optimization of the production process. Furthermore, the user management and system settings modules enhance the system's security, flexibility, and ease of use. These improvements collectively address the problems faced by traditional task management systems in complex automated assembly environments, such as insufficient integration, low automation, extensive management, low data utilization, and poor security, thereby significantly improving the overall operational efficiency, production quality, and management level of the assembly system.

[0069] In some of the embodiments described above in this application, the management layer is equipped with a single-point control system for single-point control of the actuators. However, in practical applications, relying solely on general single-point control functions is insufficient to efficiently and flexibly handle the execution of incomplete tasks, equipment maintenance and debugging, and the handling of sudden failures. This may lead to complex operation, difficult maintenance, and impact on production efficiency and safety.

[0070] In other embodiments, the single-point control system includes a step management module, a tool control module, an equipment control module, and a fault alarm module. The step management module is used to complete incomplete task requirements by controlling individual steps at a single point; the tool control module implements input and output I / O operations on the actuators, facilitating user maintenance; the equipment control module implements control functions, including start, reset, power-off, emergency stop, and track movement; and the fault alarm module provides alarms for faults and anomalies that occur during task execution or single-point control.

[0071] Specifically, the step management module is designed to handle incomplete task requirements, completing individual steps through single-point control. This means that the module allows operators or maintenance personnel to precisely control the system to perform a specific, independent action or step without initiating the entire assembly process. The module provides a list of steps through a graphical user interface (GUI). After the user selects a specific step, the module sends the corresponding control command to the electrical control system. Alternatively, a series of single-step operations can be packaged using scripts or preset macro commands for users to invoke on demand. The tool control module implements input and output (I / O) operations on actuators, facilitating user maintenance. Actuators typically refer to end effectors on equipment such as robotic arms and gantry cranes, including grippers, suction cups, and screwdrivers. I / O operations allow direct control of the on / off states and statuses of these tools. The module provides buttons or switches through a software interface, directly mapping them to the digital or analog input / output ports of the actuators, such as controlling the opening and closing of grippers or the suction / release of suction cups. Simultaneously, advanced users or external programs can directly send I / O control commands through a command-line interface or API. The equipment control module implements control functions for the main equipment in the equipment set, including start, reset, power-off, emergency stop, and track movement. This provides macro-level control capabilities for the entire system or key subsystems, which is crucial for system initialization, fault recovery, and safe operation. This module can communicate with the power supply system (PLC) to send predefined control commands, such as start / stop signals in the PLC program. Alternatively, it can be directly connected to the safety circuit via a hardware interface, such as an emergency stop button, while the software module provides corresponding status monitoring and reset functions. The fault alarm module alarms for faults and anomalies that occur during task operation or single-point control. It monitors the system status and promptly notifies the user when an anomaly is detected to prevent equipment damage or personal injury. This module can listen for alarm signals or error codes from the PLC and display alarm information on the user interface, while also providing audible and visual alerts. Furthermore, it can analyze sensor data, such as temperature, pressure, and current, and trigger an alarm when the data exceeds a preset threshold.

[0072] The single-point control system of this application integrates a step management module, a tool control module, an equipment control module, and a fault alarm module, forming a comprehensive and flexible equipment operation and maintenance platform. When the system needs to perform incomplete tasks or for maintenance and debugging, the step management module allows operators to precisely select and execute individual assembly steps, avoiding the redundancy of starting the entire complex task process. For example, if only a certain gripping action needs to be verified, the step management module can directly drive the robotic arm to execute that gripping step without involving subsequent assembly, measurement, or other steps. Meanwhile, the tool control module provides direct I / O operation capabilities for the actuators, enabling maintenance personnel to easily test or adjust the on / off states of the tools, such as manually controlling the opening and closing of the grippers to check their mechanical performance or replace parts. The equipment control module provides key control functions for the entire equipment assembly at a more macroscopic level, such as start, reset, power-off, emergency stop, and ground rail movement, which are crucial for system initialization, safe shutdown, and rapid recovery after a fault. For example, in the event of an equipment malfunction, an emergency stop operation can be quickly executed through the equipment control module to ensure safety. Throughout all these operations, the fault alarm module continuously monitors the system's operating status. Upon detecting any fault or anomaly, it immediately triggers an alarm, promptly notifying operators and effectively preventing potential equipment damage or safety accidents. Through the synergistic effect of these modules, the single-point control system not only meets diverse local operational needs but also significantly improves system maintainability, safety, and operational flexibility, compensating for the shortcomings of basic single-point control functions in handling complex field requirements.

[0073] The following example illustrates this: the aforementioned single-point control system can be deployed on an industrial panel PC running an application based on configuration software. The process management module presents a user-friendly graphical interface listing all individually executable assembly steps, such as "Grab Material A," "Place Material B," and "Tighten Screw C." When the user clicks "Grab Material A," this module generates the corresponding PLC instruction, which is sent to the electrical control system via the OPCUA protocol. The electrical control system then drives the robotic arm to execute the preset gripping action. The tool control module provides a series of buttons on the same interface, each corresponding to an I / O operation of an actuator, such as "Gripper Open," "Gripper Close," "Suction Cup Adhesion," and "Suction Cup Release." Users can directly control the movement of the robotic arm's end effector gripper or suction cup by clicking these buttons, facilitating function checks during maintenance. The equipment control module can include function buttons such as "System Start," "System Reset," "System Power Down," "Emergency Stop Reset," and "Ground Rail Forward" and "Ground Rail Backward." For example, if the system stops for any reason, maintenance personnel can send a reset command to the electrical control system by clicking the "System Reset" button, restoring the entire assembly line to its initial state. The rail movement function allows operators to manually adjust the position of the rails without initiating a full task, facilitating material loading or unloading. The fault alarm module displays the alarm information in real-time at the top of the interface. When the electrical control system detects anomalies such as "robotic arm overtravel," "sensor failure," or "motor overload," it sends alarm information to the single-point control system via the OPCUA protocol. The fault alarm module immediately displays the alarm content in red font on the interface, possibly accompanied by a buzzer sound, and simultaneously records the alarm event to a local log file for later review.

[0074] Through the aforementioned technical solutions, the single-point control system is no longer merely a simple actuator control system, but possesses more refined and comprehensive operation and management capabilities. The step management module enables operators to flexibly execute incomplete tasks, greatly improving the efficiency of debugging, testing, and partial operations, and avoiding unnecessary overall process startups. The tool control module and equipment control module provide maintenance personnel with direct and convenient equipment I / O operations and macro-control methods, significantly simplifying daily equipment maintenance, troubleshooting, and system recovery processes, and reducing reliance on specialized programming knowledge. The fault alarm module can promptly detect and report abnormal situations during task execution or single-point control, effectively improving system safety and reliability, and reducing downtime and potential equipment damage risks caused by faults. The integration of these modules enables the entire multi-level automated assembly system to possess higher operational flexibility, maintenance convenience, and operational safety when dealing with complex and ever-changing production site demands, thereby improving overall production efficiency and automation levels.

[0075] In some embodiments described above, the multi-level automated assembly system, through the collaborative work of the management layer, control layer, and equipment layer, realizes the reception, management, and distribution of assembly tasks, as well as equipment driving and measurement verification. The measurement system is connected to an image acquisition device to drive the device to acquire images, calculates measurement results using the acquired images, and feeds these results back to the electrical control system. However, in actual automated assembly processes, material loading may contain errors, the gripping and assembly stages require extremely high precision, and material identification and traceability, as well as precise guidance for specific processes (such as screw driving), are also necessary. Basic measurement functions are insufficient to meet these complex and high-precision requirements, potentially leading to insufficient assembly accuracy, low efficiency, or chaotic material management.

[0076] In this application, the measurement system includes the following functional modules: a coarse positioning module, a fine positioning module, an assembly fine positioning module, an image acquisition module, a screw driving measurement module, and a QR code recognition module.

[0077] The coarse positioning module addresses errors generated during material loading by performing template matching and positioning based on image features of the assembled materials. This module aims to quickly and initially determine the approximate position and orientation of the materials, providing a foundation for subsequent precise positioning. This can be achieved by comparing a pre-set material template with real-time acquired images to find the approximate location of the materials; or by using feature point detection algorithms, such as SIFT, SURF, or ORB, to extract significant feature points from the material images and match them with a pre-stored feature point set to determine the initial position of the materials.

[0078] The precise positioning module performs sub-pixel-level accurate positioning based on the edge contour features of the assembled workpiece, approximating the relative position between the tool and the workpiece to a standard position to ensure consistent gripping. This module further improves positioning accuracy based on coarse positioning, ensuring the actuator can accurately grip the material. This can be achieved by using edge detection algorithms, such as Canny or Sobel, to extract the workpiece's edge contour, then fitting precise geometric features using methods such as least squares or Hough transform to calculate sub-pixel-level center point or key point coordinates; or by utilizing sub-pixel matching technology based on grayscale correlation or phase correlation to perform local high-precision searches in the coarse positioning area to achieve even higher positioning accuracy.

[0079] The assembly precision positioning module performs sub-pixel-level positioning based on the edge contour features of the workpiece to be assembled and the workpiece to be assembled, approximating and adjusting the relative assembly position to ensure assembly accuracy requirements. This module aims to ensure that the two workpieces are aligned with extremely high precision during assembly. This can be achieved by simultaneously identifying the edge contours of both the workpiece to be assembled and the workpiece to be assembled, calculating the relative positional deviation between them, and correcting it according to a preset assembly datum. This typically involves multi-target recognition and coordinate system transformation. Alternatively, it can utilize vision-guided technology to monitor the relative position of the workpieces in real time during assembly and dynamically adjust the motion trajectory of the actuator based on the deviation until the preset assembly accuracy requirements are met.

[0080] The image acquisition module communicates via the Transmission Control Protocol (TCP) to control a 2D or 3D camera for image acquisition. This module is fundamental to all vision processing functions and is responsible for acquiring high-quality image data. It can be implemented by communicating with the camera through a standard industrial camera interface, such as GigE Vision or USB3 Vision, sending trigger commands, setting parameters such as exposure time and gain, and receiving image data; or by using the software development kit (SDK) provided by the camera manufacturer for secondary development, enabling deeper control over the camera, including image preprocessing and region of interest (ROI) setting.

[0081] The screw-driving measurement module is designed for the unique circular structure of screw holes. It performs circular fitting and center calculation to guide the robotic arm in screw-driving operations. This module is specifically used for precise guidance during screw assembly. It can be implemented by extracting the screw hole region from the acquired image using image segmentation and thresholding, and then using Hough circle transform or least squares circle fitting algorithms to accurately calculate the center coordinates and radius of the screw hole; or by combining a deep learning model to train a neural network model capable of recognizing screw holes and outputting their center positions, thereby improving robustness in complex backgrounds.

[0082] The QR code recognition module scans the QR codes on materials entering the system to verify their identification numbers. This module is used for automated material identification and traceability. It can be implemented by using image processing libraries, such as OpenCV, with QR code decoding algorithms to locate, correct, and decode the QR code area in the image to obtain the material's unique identification information; or by integrating a professional QR code recognition SDK. This SDK typically has optimized algorithms for different types of QR codes, such as QR Code and DataMatrix, to improve recognition speed and accuracy.

[0083] The solution presented in this application achieves comprehensive and high-precision visual guidance and verification of the automated assembly process through the collaborative operation of various functional modules within the aforementioned measurement system. Specifically, when materials enter the system, the QR code recognition module first verifies their identity and number to ensure their correctness. Subsequently, before the actuator grasps the materials, the image acquisition module acquires an image of the materials, the coarse positioning module performs preliminary positioning to address feeding errors, and then the fine positioning module performs sub-pixel-level precise positioning to ensure the actuator can grasp materials with extremely high consistency. During the assembly stage, the image acquisition module again acquires images of the workpiece to be assembled and the workpiece to be assembled. The fine positioning module analyzes the edge contours of both to achieve sub-pixel-level relative position adjustment, thereby ensuring assembly accuracy. For processes requiring screw driving, the screw driving measurement module can accurately identify the circular structure of the screw hole and calculate its center, providing precise coordinate guidance for the robotic arm's screw driving operation. Throughout the entire process, the image acquisition module continuously provides high-quality visual data support. The integration of these modules enables the measurement system to handle a variety of high-precision vision tasks, from material identification, grasping, and positioning to specific process guidance, and to feed back accurate measurement results to the electrical control system. This allows the electrical control system to more accurately drive the robotic arms, gantry cranes, and turntables in the equipment assembly to complete the corresponding actions in the process flow, effectively solving the accuracy and efficiency problems caused by insufficient vision measurement capabilities in traditional automated assembly.

[0084] The following is a concrete example. On an automated assembly line for mobile phone screens and mid-frames, firstly, when the pallet containing the mid-frame is delivered to the assembly station by an AGV (Automated Guided Vehicle), the QR code recognition module in the measurement system immediately scans the QR code on the mid-frame to verify its model and batch information. Next, the robotic arm prepares to grasp the screen. At this time, the image acquisition module controls a 2D camera to capture an image of the screen in the feeder. The coarse positioning module quickly identifies the approximate position of the screen through template matching, and then the fine positioning module performs sub-pixel analysis of the screen's edge contours, accurately calculating the screen's center and orientation to guide the robotic arm to accurately grasp the screen. When the robotic arm moves the screen above the mid-frame, the image acquisition module captures another image of the screen and mid-frame. The fine positioning module simultaneously identifies the edge features of the screen and mid-frame, calculates the relative positional deviation between them, and provides real-time feedback to the electronic control system. The electronic control system fine-tunes the robotic arm's movement based on the feedback information, ensuring that the screen is precisely aligned and placed on the mid-frame with sub-pixel accuracy. If screws need to be fixed later, the screw-driving measurement module will identify the screw holes at the junction of the screen and the mid-frame, perform circular fitting, and calculate the precise center coordinates of each screw hole to guide the screw-driving robotic arm to perform precise screw fastening operations.

[0085] By integrating a coarse gripping positioning module, a fine gripping positioning module, an assembly fine positioning module, an image acquisition module, a screw driving measurement module, and a QR code recognition module into the measurement system, the automated assembly system of this application can significantly improve the accuracy, efficiency, and reliability of the assembly process. Specifically, the combination of the coarse gripping positioning module and the fine gripping positioning module effectively solves the problems of material feeding error and gripping consistency, ensuring that the actuator can accurately grip materials. The assembly fine positioning module greatly improves the accuracy of assembly through sub-pixel-level positioning, meeting the requirements of high-precision assembly. The screw driving measurement module provides precise screw hole positioning for specific process steps, avoiding assembly failures caused by inaccurate positioning. The QR code recognition module realizes automated material identification and traceability, enhancing the controllability of the production process. The image acquisition module, as the basic source of visual data, provides real-time and accurate image information for all measurement and positioning functions. The synergistic effect of these modules enables the entire automated assembly system to achieve high-precision control and verification throughout the entire process from material identification, grasping, positioning to final assembly when handling complex and high-precision assembly tasks. This effectively reduces manual intervention, improves production efficiency and product quality, and reduces scrap rates.

[0086] Another embodiment of the present invention provides a control method for a multi-level automated assembly system, comprising the following steps: The task management system distributes assembly tasks to the electrical control system for process flow invocation; A single-point control system sends control commands to the electrical control system to perform single-point control of the actuator; The electrical control system drives each piece of equipment in the assembly to complete the corresponding actions in the process flow; In the accuracy requirement stage, the electronic control system triggers the measurement function of the measurement system for verification. The measurement system drives the image acquisition device to acquire images and uses the acquired images to calculate the measurement results, which are then fed back to the electronic control system.

[0087] In one possible implementation, the following steps are performed in the task management system: Establish a communication mechanism with the upstream management unit's Manufacturing Execution System (MES) to receive overall task control and report task execution status, material storage status, and site equipment status in real time; It establishes communication with the Automated Guided Vehicle (AGV) that transports external materials, receives materials and frames transported for assembly, and works with the AGV to transfer the assembled frames. The system manages and controls tasks entering the system in a unified manner, and also manages materials entering the system, supporting manual import of tasks from local storage. It can control the start, pause, or stop of tasks, and monitor and display the progress of the task process; The system records and displays the device status, basic device information, alarm logs, and device maintenance records within the system. The system stores and records completed and incomplete tasks within a certain period, and displays the statistics in the form of reports. The system is designed with three user permissions: "Administrator", "Operator" and "Maintenancer", with the level decreasing progressively. Personnel permissions are managed according to specific needs. Configure communication and database settings.

[0088] In one possible implementation, the following steps are performed in the measurement system: To address the errors generated during the material loading process, template matching and positioning are performed based on the image features of the assembly materials themselves to achieve coarse positioning for grasping; Based on the edge contour features of the assembled workpiece, sub-pixel level positioning is performed, and the relative position between the tool and the assembled workpiece is adjusted to the standard position to ensure consistent gripping and achieve precise gripping positioning. Based on the edge contour features of the workpiece to be assembled and the workpiece to be assembled, sub-pixel-level positioning is performed, and the relative assembly position is adjusted to ensure the assembly accuracy requirements and achieve precise assembly positioning. Control 2D or 3D cameras to acquire images via TCP communication; For the special circular structure of screw holes, we perform circular fitting and center calculation to guide the robotic arm in screw driving operations; Materials entering the system are numbered and verified by scanning the QR code on the material.

[0089] In the above embodiments, the descriptions of each embodiment have different focuses. For parts that are not described in detail or recorded in a certain embodiment, please refer to the relevant descriptions of other embodiments.

[0090] The above-described embodiments are only used to illustrate the technical solutions of this application, and are not intended to limit them. Although this application has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features. Such modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of this application, and should all be included within the protection scope of this application.

Claims

1. A multi-level automated assembly system, characterized in that, It includes a management layer, a control layer, and an equipment layer. The management layer sets up a task management system and a single-point control system, the control layer sets up an electrical control system and a measurement system, and the equipment layer sets up a collection of equipment. The task management system is connected to the electrical control system. The task management system is used to receive, create and manage assembly tasks, and to send assembly tasks to the electrical control system for process flow invocation. The single-point control system is connected to the electrical control system. The single-point control system is used to perform single-point control on the actuators and to send control commands to the electrical control system for single-point control. The electronic control system is connected to the measurement system. When the accuracy requirement is reached, the electronic control system triggers the measurement function for verification. The electrical control system is also connected to the equipment assembly to drive each piece of equipment to complete the corresponding actions in the process flow; The measurement system is connected to the image acquisition device in the equipment set, which drives the image acquisition device to acquire images, calculates the measurement results using the acquired images, and feeds the measurement results back to the electronic control system.

2. The multi-level automated assembly system according to claim 1, characterized in that, The task management system and measurement system are implemented on a personal computer PC. The single-point control system is implemented through configuration software. The electrical control system is implemented through a programmable logic controller (PLC). The image acquisition device is a 2D or 3D camera. The equipment in the set includes a robotic arm, a gantry crane, and a turntable.

3. The multi-level automated assembly system according to claim 2, characterized in that, The device communication methods of the device set include Modbus, OPCUA, Ethernet, and EnterCAT. The PLC of the electrical control system supports communication development and expansion.

4. The multi-level automated assembly system according to claim 2, characterized in that, The task management system and measurement system located on the PC communicate with the electronic control system via OPCUA; the single-point control system implemented by the configuration software also communicates with the electronic control system via OPCUA; the measurement system located on the PC is directly connected to the 2D or 3D camera via the TCP transmission control protocol for command interaction and data transmission.

5. The multi-level automated assembly system according to claim 1 or 2, characterized in that, The task management system includes the following functional modules: The Manufacturing Execution System (MES) communication module establishes a send-receive mechanism with the upstream management unit's MES, receives overall task control, and reports task execution status, material storage status, and site equipment status in real time. The Automated Guided Vehicle (AGV) docking module establishes communication with external AGVs transporting materials, automatically receives materials and frames transported for assembly, and automatically coordinates with the AGVs to transfer the assembled frames. The task management module manages and controls the scheduling of tasks entering the system, as well as the materials entering the system, and supports manually importing tasks from the local machine. The task execution module controls the start, pause, or stop of tasks, and can monitor and display the progress of the task process; The equipment management module records and displays the equipment status, basic equipment information, alarm logs, and equipment maintenance records within the system. The production report module stores and records completed and incomplete tasks within a certain period, and displays them in the form of reports. The user management module is designed with three user permissions: "Administrator", "Operator" and "Maintenancer", with the level decreasing in that order, which makes it easy for users to manage personnel permissions as needed. The system settings module enables two functions: communication configuration and database configuration.

6. The multi-level automated assembly system according to claim 1 or 2, characterized in that, The single-point control system includes the following functional modules: The step management module allows for the completion of incomplete tasks by controlling individual steps at a single point. The tool control module enables input and output I / O operations on the actuator, facilitating user maintenance. The equipment control module implements control functions, including start-up, reset, power-off, emergency stop, and ground rail movement. The fault alarm module provides alarms for faults and anomalies that occur during task execution or single-point control.

7. The multi-level automated assembly system according to claim 1 or 2, characterized in that, The measurement system includes the following functional modules: The coarse positioning module captures and locates the assembly material by matching templates to the image features of the material itself, which is used to address errors that occur during the material loading process. The precise positioning module performs sub-pixel-level positioning based on the edge contour features of the assembled workpiece, and closely adjusts the relative position between the tool and the assembled workpiece to the standard position to ensure consistent gripping. The assembly precision positioning module performs sub-pixel-level position positioning based on the edge contour features of the workpiece to be assembled and the workpiece to be assembled, and adjusts the relative assembly position to ensure the assembly accuracy requirements. The image acquisition module controls a 2D or 3D camera to acquire images via TCP communication; The screw-driving measurement module performs circle fitting and center calculation for the special circular structure of screw holes, guiding the robotic arm's screw-driving operation. The QR code recognition module scans the QR code on the materials entering the system to verify their identification number.

8. A control method for a multi-level automated assembly system, characterized in that, Includes the following steps: The task management system distributes assembly tasks to the electrical control system for process flow invocation; A single-point control system sends control commands to the electrical control system to perform single-point control of the actuator; The electrical control system drives each piece of equipment in the assembly to complete the corresponding actions in the process flow; In the accuracy requirement stage, the electronic control system triggers the measurement function of the measurement system for verification. The measurement system drives the image acquisition device to acquire images and uses the acquired images to calculate the measurement results, which are then fed back to the electronic control system.

9. The control method for a multi-level automated assembly system according to claim 8, characterized in that, Perform the following steps in the task management system: Establish a communication mechanism with the upstream management unit's Manufacturing Execution System (MES) to receive overall task control and report task execution status, material storage status, and site equipment status in real time; It establishes communication with the Automated Guided Vehicle (AGV) that transports external materials, receives materials and frames transported for assembly, and works with the AGV to transfer the assembled frames. The system manages and controls tasks entering the system in a unified manner, and also manages materials entering the system, supporting manual import of tasks from local storage. It can control the start, pause, or stop of tasks, and monitor and display the progress of the task process; The system records and displays the device status, basic device information, alarm logs, and device maintenance records within the system. The system stores and records completed and incomplete tasks within a certain period, and displays the statistics in the form of reports. The system is designed with three user permissions: "Administrator", "Operator" and "Maintenancer", with the level decreasing progressively. Personnel permissions are managed according to specific needs. Configure communication and database settings.

10. The control method for a multi-level automated assembly system according to claim 8, characterized in that, Perform the following steps in the measurement system: To address the errors generated during the material loading process, template matching and positioning are performed based on the image features of the assembly materials themselves to achieve coarse positioning for grasping; Based on the edge contour features of the assembled workpiece, sub-pixel level positioning is performed, and the relative position between the tool and the assembled workpiece is adjusted to the standard position to ensure consistent gripping and achieve precise gripping positioning. Based on the edge contour features of the workpiece to be assembled and the workpiece to be assembled, sub-pixel-level positioning is performed, and the relative assembly position is adjusted to ensure the assembly accuracy requirements and achieve precise assembly positioning. Control 2D or 3D cameras to acquire images via TCP communication; For the special circular structure of screw holes, we perform circular fitting and center calculation to guide the robotic arm in screw driving operations; Materials entering the system are numbered and verified by scanning the QR code on the material.

Images