Digital twinning-based large component installation and measurement integrated online regulation and control method and system

Abstract

The invention provides a digital twinning-based large component installation and measurement integrated online regulation and control method and system, and belongs to the technical field of intelligent manufacturing and digital twinning, and the method comprises the steps: carrying out the preprocessing of an original three-dimensional model of a large component, and carrying out the boundary point recognition, folding cost calculation and iteration simplification of the original three-dimensional model; constructing a multi-sensor measurement network, performing global coordinate system calibration and multi-sensor data fusion of the large component assembly scene, and constructing a digital twinborn model of the large component assembly scene; discretizing the digital twin model into a three-dimensional grid map, integrating global coordinate system data information, and performing dynamic path planning and collision detection; and comparing the actual pose of the large component measured by the laser tracker with the target pose of the large component of the digital twin model in real time, and when the deviation exceeds a threshold value, triggering a regulation and control mechanism to carry out real-time pose adjustment and closed-loop feedback control. According to the invention, closed loop of measurement, debugging and assembly of large components is realized, and the assembly efficiency and quality are improved.

Description

TECHNICAL FIELD

[0001] The present application relates to a kind of big component based on digital twin's assembly measurement integration online regulation method and system, belong to intelligent manufacturing and digital twin field. BACKGROUND

[0002] The assembly of large component product involves multi-variety cooperation and multi-stage process, and the traditional assembly method highly depends on manual operation and experience judgment, which is prone to cause measurement feedback delay, resulting in assembly dynamic adjustment lag, and it is difficult to realize real-time cooperation.In addition, human-computer cooperation relies on manual experience, lacks intelligent decision-making and data visualization, and has problems such as low integration degree, low assembly efficiency, poor precision, safety hazards and the like in assembly, which limits the ability of existing technology to integrate and display the assembly and measurement process.

[0003] Digital twin technology, as a new technology, is reconfiguring the method of assembly quality control in the manufacturing field.Digital twin can dynamically perceive the assembly and measurement of products by building a virtual twin model highly synchronized with the physical assembly system, combining multi-source data and intelligent algorithms, and can guide and correct the error behavior in the assembly process, realize closed-loop management and regulation, and greatly improve the efficiency and quality of assembly.

[0004] How to realize real-time and multi-device cooperation in large component assembly, realize closed-loop control of measurement, debugging and assembly, is a technical problem to be solved by the present application. SUMMARY

[0005] The present application aims to solve the above problems, and provides a kind of big component based on digital twin's assembly measurement integration online regulation method and system: In the first aspect, the present application provides a kind of big component based on digital twin's assembly measurement integration online regulation method, comprising the following steps: S1. Preprocessing the original three-dimensional model of large component, inputting complete original three-dimensional model data containing vertex set and face sheet set, identifying boundary points of the original three-dimensional model, calculating folding cost and iteratively simplifying, to obtain the simplified three-dimensional model of large component; S2. Constructing a multi-sensor measurement network containing laser tracker, three-dimensional scanner and numerical control positioner, calibrating the global coordinate system of large component assembly scene and fusing multi-sensor data, and constructing the digital twin model of large component assembly scene; S3. Discretize the digital twin model into three-dimensional grid map, integrate global coordinate system data information of static obstacles and dynamic devices, and perform dynamic path planning and collision detection; S4. In real time, compare the actual pose of the large component measured by the laser tracker with the target pose of the large component in the digital twin model. When the deviation exceeds the threshold, trigger the control mechanism to perform real-time pose adjustment and closed-loop feedback control.

[0006] Furthermore, the specific steps of step S1 are as follows: S11. Import the original 3D model of the large component, including the vertex set. V and the original set F Complete model data, the vertex set V and the original set F They are represented as follows:

[0007]

[0008] In the formula, Let i be the i-th vertex in the set. Let be the three-dimensional coordinates of the i-th vertex. Form a triangular facet for the i, j, and k vertices. The three vertices of the triangle facet are defined by the vertex set used to store the three-dimensional coordinates of the vertices. The facet set is composed of the vertex indices of the triangle facets, and each facet corresponds to the three vertices in the vertex set. S12. Calculate the number of adjacent edges of a triangle vertex using an improved triangle folding algorithm, identify key boundary points, and classify vertices with fewer than 6 adjacent edges as boundary points, forming a boundary point set. B :

[0009] In the formula, This represents the number of adjacent edges to the vertices of the triangular face. S13. Calculate the folding geometric error of the new vertex after folding each triangular unit. C By iterating the folding geometric error C The smallest triangle, dynamically updating the vertex set. V noodle set F This continues until the number of facets is reduced to a preset number, which is set according to the original 3D model, wherein the folding geometric error... C The calculation formula is:

[0010] In the formula, For triangular facets The number of adjacent triangles, The new vertex after folding These are the coordinates of the vertices of the adjacent triangles.

[0011] Further, step S2 comprises: S21. Arranging a laser tracker, a three-dimensional scanner and a numerical positioning device in a large component assembly scene, the laser tracker is used to measure the actual pose of the large component, the three-dimensional scanner is used to obtain point cloud data of the large component assembly scene, and the numerical positioning device is used to drive the AGV and the mechanical arm to assemble the large component; S22. Arranging M standard target balls in the large component assembly scene, the laser tracker measures the coordinates of the standard target balls, and a conversion matrix is solved by least squares method, the measured coordinates and the theoretical coordinates of the standard target balls are input, and the calibration of the global coordinate system is performed, wherein M is the number of standard target balls; S23. Establishing a time synchronization mechanism based on IEEE 1588 protocol, and mapping the point cloud data obtained by the three-dimensional scanner to the global coordinate system; S24. Using three-dimensional modeling software to create a virtual model of the large component assembly scene, setting the initial state of the model, inputting the data after time synchronization and coordinate conversion into the virtual model, and constructing a digital twin model of the large component assembly scene; S25. Real-time updating the data of the digital twin model, ensuring that the delay of physical and virtual mapping does not exceed the specified time, and the spatial registration error is controlled within the specified value, wherein the specified time and the specified value are set according to the actual situation, and the spatial pose data of the AGV and the mechanical arm is integrated in real time.

[0012] Further, the specific steps of step S3 are: S31. Using an improved A* algorithm for path planning, and improving the traditional cost function by introducing a position deviation and a timestamp penalty term , dynamically adjusting the path weight, and generating a preliminary path, the improved cost function expression is:

[0013] In the formula, represents the actual path cost from the starting point to the current node , which is dynamically calculated during planning; is the heuristic estimated cost from the current node to the end point, is the deviation of the actual position of the mechanical arm from the planned path, is the node timestamp penalty term, is the deviation weight, is the timestamp delay weight; , , is the coordinate value of the current node , , , coordinate value of the end point, , , coordinate value of the actual position of the mechanical arm; S32. adopt quintic polynomial interpolation on the preliminary path optimize the joint trajectory of the mechanical arm and perform accurate collision detection of the envelope surface of the mechanical arm link and the separation axis distance of the large component assembly scene model based on GPU parallel computing , if , mark as collision, the system re-plans the path, wherein, is a safety threshold, and the quintic polynomial interpolation is expressed as:

[0014] In the formula, is a polynomial coefficient, is a normalized time variable, , is the total motion time; the separation axis distance is expressed as:

[0015] In the formula, is the coordinate of the vertex of the envelope surface of the mechanical arm link, is the normal vector of the surface of the large component assembly scene model.

[0016] Further, the specific steps of step S4 are: S41. generate the pose adjustment command by using an adaptive PID algorithm , the proportional term coefficient is adjusted dynamically according to the assembly stage, the proportional gain is increased in the coarse adjustment stage to quickly converge, and the differential term is enhanced in the fine adjustment stage to suppress overshoot; the pose adjustment command is a six-dimensional pose adjustment amount, and is expressed as:

[0017] In the formula, is a proportional gain matrix, is a differential gain matrix, is a six-dimensional pose deviation vector, is a position deviation, is an attitude deviation; S42. the pose adjustment command is issued to the numerical control positioner through the Ethernet protocol, and the numerical control positioner drives the AGV and the mechanical arm to cooperatively complete the pose adjustment, and immediately performs secondary measurement verification after each pose adjustment to obtain the actual pose.

[0018] Further, in step S41: When the pose deviation is greater than a first threshold value , it is determined to enter a coarse adjustment stage; When the pose deviation continues to be less than the first threshold value and greater than a second threshold value , it is determined to switch to a fine adjustment stage; Wherein, the first threshold value is greater than the second threshold value , the first threshold value and the second threshold value are configured according to process requirements.

[0019] Further, the following steps are further included: Real-time assembly guidance is provided through an AR interface, combined with camera recognition of assembly actions, to trigger alarm, pause equipment, and permission lock of pose adjustment instruction issuance for error or dangerous behavior.

[0020] Further, the specific steps of providing real-time assembly guidance through an AR interface, combined with camera recognition of assembly actions, to trigger alarm, pause equipment, and permission lock of pose adjustment instruction issuance for error or dangerous behavior are as follows: Align the coordinate system of the AR device with the above-mentioned global coordinate system through a spatial positioning system, associate the three-dimensional model of the large component and the digital twin data, and superimpose virtual guidance information in the AR field of view; Deploy an industrial camera at the assembly station to cover the operating space of the operator; if the operator's arm is detected to move the robot motion area, the AR interface prompts, triggers an alarm, and pauses the equipment operation, locks the pose adjustment instruction issuance permission, and proceeds to the next assembly action after the on-site adjustment is completed.

[0021] In a second aspect, the embodiments of the present application also provide a large component assembly and measurement integrated online regulation and control system based on digital twin, comprising: A model preprocessing module pre-processes the original three-dimensional model of the large component, inputs complete original three-dimensional model data containing a vertex set and a face sheet set, performs boundary point identification, folding cost calculation, and iterative simplification on the original three-dimensional model, and obtains a simplified three-dimensional model of the large component; A model construction module is used to construct a multi-sensor measurement network containing a laser tracker, a three-dimensional scanner, and a numerical control positioner, calibrate a global coordinate system of the large component assembly scene, and fuse multi-sensor data to construct a digital twin model of the large component assembly scene; A path planning module is used to discretize the digital twin model into a three-dimensional grid map, integrate global coordinate system data information of static obstacles and dynamic equipment, and perform dynamic path planning and collision detection; The pose control module is used for comparing the actual pose of the large component measured by the laser tracker with the target pose of the large component of the digital twin model in real time, triggering a regulation mechanism when the deviation exceeds a threshold, and performing real-time pose regulation and closed-loop feedback control.

[0022] Further, the man-machine cooperation module is further included, which is used for providing real-time assembly guidance through an AR interface, combining camera recognition assembly actions, triggering alarm, suspending equipment and issuing permission lock for incorrect or dangerous behaviors.

[0023] From the above technical solutions, the present application has the following advantages: Through model lightening technology, key shape information is retained while the calculation amount is greatly reduced, laying an efficient data foundation for subsequent real-time regulation; a multi-sensor measurement network is combined with a standard target ball for global calibration, and is synchronized with the IEEE 1588 protocol, realizing high-frequency update of the physical and virtual scenes, low delay and 0.1mm-level registration accuracy, guaranteeing the real-time and accuracy of dynamic mapping; the A* algorithm is improved by introducing a position deviation and timestamp penalty term, combined with GPU parallel collision detection, so that the mechanical arm path planning is more suitable for dynamic environment and effectively avoids collision risks; the adaptive PID pose regulation dynamically adjusts parameters according to the assembly stage, coarsely adjusts fast convergence, finely adjusts to suppress overshoot, forms a closed loop of measurement, debugging and assembly, and improves the pose control precision; the man-machine cooperation mechanism combining the AR interface and the camera realizes real-time assembly guidance and dangerous behavior recognition, and guarantees the safety of man-machine cooperation through alarm, equipment suspension and permission lock; the present application supports the whole-process closed-loop control of measurement, regulation and assembly in complex scenes, greatly improving the efficiency and quality of large component assembly. BRIEF DESCRIPTION OF DRAWINGS

[0024] In order to more clearly illustrate the technical solutions of the present application, the drawings needed in the description will be briefly introduced below. Obviously, the drawings in the following description are only some embodiments of the present application, and other drawings can be obtained by those skilled in the art without creative labor.

[0025] Figure 1 It is a flowchart of the online regulation and control method of large component assembly and measurement integration based on digital twin of the present application.

[0026] Figure 2 It is an assembly scene diagram of the online regulation and control method of large component assembly and measurement integration based on digital twin of the present application.

[0027] Figure 3 It is a closed-loop control architecture diagram of the online regulation and control method of large component assembly and measurement integration based on digital twin of the present application.

[0028] Figure 4A schematic diagram of an online regulation system for digital-twin-based large component assembly and measurement integration of the present application. DETAILED DESCRIPTION

[0029] In the following detailed description of the specific steps of the digital-twin-based large component assembly and measurement integration online regulation method, various embodiments of the present application will be described more fully. The present application can have various embodiments, and adjustments and changes can be made therein. However, it should be understood that there is no intention to limit various embodiments of the present application to the specific embodiments disclosed herein, but the present application should be understood to cover all adjustments, equivalents and / or alternatives falling within the spirit and scope of various embodiments of the present application.

[0030] In order to make the inventive purpose, features and advantages of the present application more obvious and easy to understand, the technical solutions protected by the present application will be clearly and completely described below with specific embodiments and drawings. Obviously, the following described embodiments are only a part of the embodiments of the present application, not all the embodiments. Based on the embodiments in the present application, all other embodiments obtained by those skilled in the art without creative labor are within the scope of protection of the present application.

[0031] Please refer to Figure 1 The figure is a flowchart of a digital-twin-based large component assembly and measurement integration online regulation method in a specific embodiment, the method comprising: S1. Preprocessing the original three-dimensional model of the large component, inputting the complete original three-dimensional model data containing vertex set and patch set, identifying the boundary points of the original three-dimensional model, calculating the folding cost and iteratively simplifying to obtain the simplified three-dimensional model of the large component; It should be noted that by preprocessing the original three-dimensional model of the large component, the key shape information can be preserved while reducing the amount of calculation; after inputting the complete model data containing vertex set and patch set, the boundary points are identified, the folding cost is calculated and iteratively simplified, which significantly reduces the computational complexity and improves the system running efficiency on the premise of guaranteeing that the key features of the model are not lost, providing a lightweight and accurate model basis for subsequent multi-sensor data fusion, dynamic path planning and other operations; S2. Constructing a multi-sensor measurement network containing a laser tracker, a three-dimensional scanner and a numerical control positioner, calibrating the global coordinate system of the large component assembly scene and fusing multi-sensor data, and constructing a digital twin model of the large component assembly scene; It should be noted that constructing a multi-sensor measurement network and performing data fusion and global calibration can achieve high-precision synchronization of the physical and virtual scenes; S3. Discretizing the digital twin model into a three-dimensional grid map, integrating the global coordinate system data information of static obstacles and dynamic equipment, and performing dynamic path planning and collision detection; It should be noted that after the assembly scene is discretized into a three-dimensional grid map and the data is integrated, a safe path is dynamically planned, and collision detection is performed to avoid collision with static obstacles and dynamic equipment in real time, ensuring the safety of equipment such as robot arms and improving the path planning efficiency and environmental adaptability of the assembly process, thereby providing a dynamic and reliable path solution for large component assembly. S4. Real-time comparison of the actual pose of the large component measured by the laser tracker and the target pose of the large component of the digital twin model, triggering the regulation mechanism when the deviation exceeds the threshold, and performing real-time pose adjustment and closed-loop feedback control; It should be noted that by comparing the actual and target poses of the large component in real time, adaptive PID pose adjustment is triggered when the deviation exceeds the threshold, forming a "measurement-adjustment-verification" closed loop, achieving high-precision dynamic regulation of the large component assembly process, and improving assembly accuracy and efficiency.

[0032] Further, as a refinement and extension of the above embodiment, in order to fully describe the specific implementation process in this embodiment, another online regulation method for large component assembly based on digital twin is provided, please refer to Figure 3 The figure shows the closed-loop control architecture of the online regulation method for large component assembly based on digital twin, which includes the following steps: S1. Preprocessing the original three-dimensional model of the large component, inputting complete original three-dimensional model data containing vertex set and face set, performing boundary point identification, folding cost calculation and iterative simplification on the original three-dimensional model to obtain the simplified three-dimensional model of the large component; the specific steps of step S1 are: S11. Import the original three-dimensional model of the large component, which contains complete model data of vertex set V and original set F , wherein the vertex set V and the original set F are represented as:

[0033]

[0034] In the formula, is the i-th vertex in the set, is the three-dimensional coordinate of the i-th vertex, is the triangle face formed by the i-th, j-th and k-th vertices, are the three vertices of the triangle face; the vertex set is used to store the three-dimensional coordinates of the vertices, and the face set is composed of triangle face vertex indices, each face corresponds to three vertices in the vertex set; S12. Calculate the number of adjacent edges of a triangle vertex using an improved triangle folding algorithm, identify key boundary points, and classify vertices with fewer than 6 adjacent edges as boundary points, forming a boundary point set. B :

[0035] In the formula, This represents the number of adjacent edges to the vertices of the triangular face. S13. Calculate the folding geometric error of the new vertex after folding each triangular unit. C By iterating over the folding geometric error C The smallest triangle, dynamically updating the vertex set. V noodle set F This continues until the number of facets is reduced to a preset number, which is set according to the original 3D model, wherein the folding geometric error... C The calculation formula is:

[0036] In the formula, For triangular facets The number of adjacent triangles, The new vertex after folding The coordinates of the vertices of the adjacent triangles; For example, import the original 3D model of the cylindrical segment, and the model vertex set. Contains 5 million vertices, a set of faces It contains 10 million triangles; An improved triangle folding algorithm is used to calculate the number of adjacent edges at each vertex. ,Will The vertices are classified as boundary points, totaling 30,000 boundary points, forming a boundary point set. ; Calculate the new vertices after folding for each triangular facet. geometric error :

[0037] Taking one of the triangular facets as an example, this triangular facet has 4 adjacent triangles. The coordinates of the new vertex after folding are The coordinates of the adjacent vertices are respectively , , , The geometric error of the patch was calculated. ; After calculating the geometric error values ​​for all folded triangular facets, the geometric error values ​​must not exceed a threshold. The vertex set and the face set are dynamically adjusted at the same time, and finally the number of the retained faces is 5% of the original model, and the key curvature characteristics of the cylinder segment are completely retained; S2. Constructing a multi-sensor measurement network comprising a laser tracker, a three-dimensional scanner, and a numerical control positioner, calibrating a global coordinate system of a large component assembly scene and fusing multi-sensor data, and constructing a digital twin model of the large component assembly scene; step S2 comprises: S21. Arranging a laser tracker, a three-dimensional scanner, and a numerical control positioner in a large component assembly scene, the laser tracker being used for measuring the actual pose of a large component, the three-dimensional scanner being used for acquiring point cloud data of the large component assembly scene, and the numerical control positioner being used for driving the AGV and the mechanical arm to assemble the large component; S22. Arranging M standard target balls in the large component assembly scene, measuring the coordinates of the standard target balls by the laser tracker, and calibrating the global coordinate by inputting the measured coordinates and the theoretical coordinates of the standard target balls and solving the conversion matrix by the least square method, wherein M is the number of the standard target balls; S23. Establishing a time synchronization mechanism based on IEEE 1588 protocol, and mapping the point cloud data acquired by the three-dimensional scanner to the global coordinate system; S24. Creating a virtual model of the large component assembly scene by using a three-dimensional modeling software, setting the initial state of the model, inputting the data after time synchronization and coordinate conversion into the virtual model, and constructing a digital twin model of the large component assembly scene; S25. Real-time updating the data of the digital twin model, ensuring that the delay of the physical and virtual mapping does not exceed a specified time, and the spatial registration error is controlled within a specified value, wherein the specified time and the specified value are set according to the actual situation, and the spatial pose data of the AGV and the mechanical arm are integrated in real time; Exemplarily, Figure 2 The figure shows a schematic diagram of an assembly scene, a laser tracker, a three-dimensional scanner, and standard target balls are deployed in the assembly site, the laser tracker measures the coordinates of the target balls, the conversion matrix is solved by the least square method , the measured coordinates and the theoretical coordinates of the target balls are input, the error of the conversion matrix is calculated to be , the point cloud data of the three-dimensional scanner and the pose data of the mechanical arm are synchronized based on IEEE 1588 protocol and mapped to the global coordinate system, then the system updates the twin model in real time at a frequency of 100 Hz, the delay of the physical and virtual mapping is less than 30 ms, and the dynamic three-dimensional map integrates the spatial pose data of the AGV and the mechanical arm in real time; S3. Discretizing the digital twin model into a three-dimensional grid map, integrating the global coordinate system data information of static obstacles and dynamic devices, and performing dynamic path planning and collision detection; the specific steps of step S3 are: S31. Path planning is performed using an improved A* algorithm, and the traditional cost function is improved by introducing a position deviation and a timestamp penalty term , the path weight is dynamically adjusted, and a preliminary path is generated, and the improved cost function expression is:

[0038] wherein, represents the actual path cost from the starting point to the current node , which is dynamically calculated during the planning process; represents the heuristic estimated cost from the current node to the end point, represents the deviation of the actual position of the robot arm from the planned path, represents the node timestamp penalty term, represents the deviation weight, represents the timestamp delay weight; 、 、 represents the coordinate value of the current node , 、 、 represents the coordinate value of the end point, 、 、 represents the coordinate value of the actual position of the robot arm; S32. Five-degree polynomial interpolation is performed on the preliminary path to optimize the joint trajectory of the robot arm, and accurate curved surface collision detection is performed based on GPU parallel computing to calculate the separation axis distance between the robot arm link envelope surface and the large part assembly scene model , if , it is marked as collision, and the system re-plans the path, wherein, is a safety threshold, and the five-degree polynomial interpolation is expressed as:

[0039] wherein, is a polynomial coefficient, is a normalized time variable, , is the total motion duration; the separation axis distance is expressed as:

[0040] wherein, is the coordinate of the vertex of the robot arm link envelope surface, is the normal vector of the surface of the large part assembly scene model; Exemplarily, path planning is first performed by improving the A* algorithm, and an improved cost function is used to dynamically adjust the path weight to adapt to real-time environmental changes, and the improved cost function is represented as:

[0041] wherein:

[0042] from the starting point to the end point , the coordinates of a certain node n in the middle are (5m, 3m, 2m). Taking , the calculation gives , the deviation weight , the timestamp delay weight , the deviation of the actual position from the planned path , and the time delay . Substituting the data, the generated path length is 19.73m, and the planning time is 1.2s. A quintic polynomial interpolation is used to optimize the trajectory of the initially generated path:

[0043] Taking , the total duration , and the coefficient controls the acceleration, suppresses overshoot. After calculation, the maximum acceleration is 0.3m / s 2 , which is less than the theoretical limit of 0.5m / s 2 .

[0044] Collision detection is performed based on GPU parallel computing to calculate the separation axis distance between the envelope surface of the robot arm link and the scene model :

[0045] The coordinates of the vertex of the robot arm link , the surface normal vector of the obstacle , and the calculated separation axis distance is less than the safety threshold , the system triggers obstacle avoidance. After detecting the collision, path planning is performed again until the separation axis distance is greater than the safety threshold; S4. Real-time comparison of the actual pose of the large component measured by the laser tracker and the target pose of the large component of the digital twin model, and when the deviation exceeds the threshold, the regulation mechanism is triggered for real-time pose adjustment and closed-loop feedback control; the specific steps of step S4 are: S41. Generate a pose adjustment instruction using an adaptive PID algorithm , the proportional term coefficient is dynamically adjusted according to the assembly stage, the proportional gain is increased in the coarse adjustment stage to quickly converge, and the differential term is enhanced in the fine adjustment stage to suppress overshoot; the pose adjustment instruction is a six-dimensional pose adjustment amount, and is expressed as:

[0046] In the formula, is a proportional gain matrix, is a differential gain matrix, is a six-dimensional pose deviation vector, is a position deviation, is an attitude deviation; in step S41: when the pose deviation is greater than a first threshold value , it is determined to enter the coarse adjustment stage; when the pose deviation is continuously less than the first threshold value and greater than a second threshold value , it is determined to switch to the fine adjustment stage; wherein the first threshold value is greater than the second threshold value , the first threshold value and the second threshold value are configured according to process requirements; S42. The pose adjustment instruction is issued to the numerical positioning device through the Ethernet protocol, and the numerical positioning device drives the AGV and the mechanical arm to cooperatively complete the pose adjustment. After each pose adjustment, secondary measurement verification is immediately performed to obtain the actual pose; Exemplarily, first, the actual pose is measured by using a laser tracker , and the deviation is calculated:

[0047] The deviation is calculated by substituting the pose into the formula ; The adaptive PID control is divided into a coarse adjustment stage and a fine adjustment stage, and the pose adjustment control instruction generated by the PID is expressed as:

[0048] In the coarse adjustment stage, the proportional gain matrix and the differential gain matrix are taken, and the six-dimensional pose adjustment instruction is calculated by substituting various parameters: ; In the fine adjustment stage, the proportional gain matrix and the differential gain matrix are taken, and ​; after obtaining the first instruction, the instruction is sent to the numerical positioner through the Ethernet protocol to drive the AGV and the mechanical arm to complete the posture adjustment, the adjustment amount of the AGV is: , and the adjustment amount of the mechanical arm is: ; immediately after the adjustment, secondary measurement is performed to obtain the actual posture of the cylinder segment , the deviation , and the threshold requirement of is satisfied; It also includes the following steps: Real-time assembly guidance is provided through the AR interface, and camera recognition assembly actions are combined to trigger alarm, pause equipment and posture adjustment instruction issuing permission lock for error or dangerous behavior; the specific steps are as follows: Through the spatial positioning system, the coordinate system of the AR device is aligned with the above-mentioned global coordinate system, the three-dimensional model of the large component and the digital twin data are associated, and virtual guidance information is superimposed in the AR field of view; Industrial cameras are deployed at the assembly station to cover the operating space of the operator; if the operator's arm is detected to be in the mechanical arm movement area, the AR interface prompts, triggers alarm and pauses equipment operation, and locks the posture adjustment instruction issuing permission until the next assembly action is performed after the on-site adjustment is completed; Exemplarily, multiple cameras are deployed to capture the operator's actions in real time, and if the distance between the operator's arm and the mechanical arm movement area is detected to be <0.5m, the AR interface flashes red light, the system triggers alarm and pauses operation, and locks the posture adjustment instruction issuing permission until the next assembly action is performed after the on-site adjustment is completed; It should be noted that through the combination of the AR interface and the camera, human-machine collaborative safety control is realized, the AR interface superimposes virtual guidance such as posture deviation in real time, and assists the operator in precise assembly; when the camera recognizes dangerous actions, it triggers AR alarm, equipment pause and posture adjustment permission lock, which not only improves assembly efficiency, but also effectively avoids human-machine collaboration risks and ensures the safety and controllability of the assembly process; This embodiment uses model lightweight technology to greatly reduce the amount of calculation while retaining key shape information, laying a high-efficiency data foundation for subsequent real-time regulation and control; the multi-sensor measurement network combines standard target ball global calibration and IEEE 1588 protocol synchronization to realize high-frequency update of physical and virtual scenes, low delay and 0.1mm-level registration accuracy, ensuring the real-time and accuracy of dynamic mapping; the improved A* algorithm introduces position deviation and timestamp penalty terms, combined with GPU parallel collision detection, to make the mechanical arm path planning more adaptive to dynamic environments and effectively avoid collision risks; adaptive PID posture adjustment dynamically adjusts parameters according to the assembly stage, with coarse adjustment for rapid convergence and fine adjustment for overshoot suppression, forming a "measurement-adjustment-verification" closed loop to improve the accuracy of posture control; the human-machine collaborative mechanism combining the AR interface and the camera realizes real-time assembly guidance and dangerous behavior recognition, and ensures the safety of human-machine collaboration through alarm, equipment pause and permission lock.

[0049] As Figure 4 shown in the following is an embodiment of the online regulation and control system for large component assembly and measurement integration based on digital twinning provided by the embodiment of the present application. The system and the online regulation and control method for large component assembly and measurement integration based on digital twinning of each embodiment described above belong to the same inventive concept. Details not described in the embodiment of the online regulation and control system for large component assembly and measurement integration based on digital twinning can be referred to the embodiment of the online regulation and control method for large component assembly and measurement integration based on digital twinning described above.

[0050] The system comprises: a model preprocessing module, which pre-processes an original three-dimensional model of a large component, inputs complete original three-dimensional model data containing a vertex set and a sheet set, performs boundary point identification, folding cost calculation and iterative simplification on the original three-dimensional model, and obtains a simplified three-dimensional model of the large component; a model construction module, which is configured to construct a multi-sensor measurement network containing a laser tracker, a three-dimensional scanner and a numerical control positioner, calibrate a global coordinate system of a large component assembly scene and fuse multi-sensor data, and construct a digital twinning model of the large component assembly scene; a path planning module, which is configured to discretize the digital twinning model into a three-dimensional grid map, integrate global coordinate system data information of static obstacles and dynamic equipment, and perform dynamic path planning and collision detection; an attitude adjustment control module, which is configured to compare a large component actual attitude measured by the laser tracker with a large component target attitude of the digital twinning model in real time, trigger a regulation and control mechanism when a deviation exceeds a threshold, and perform real-time attitude adjustment and closed-loop feedback control; and further comprises a man-machine cooperation module, which is configured to provide real-time assembly guidance through an AR interface, recognize assembly actions in combination with a camera, trigger alarm, pause equipment and release permission lock for attitude adjustment instructions for error or dangerous behaviors.

[0051] The above description of disclosed embodiments enables a person skilled in the art to implement or use the present application. Various modifications to these embodiments will be apparent to those skilled in the art, and the general principles defined herein can be implemented in other embodiments without departing from the spirit or scope of the present application. Therefore, the present application will not be limited to these embodiments shown herein, but will conform to the widest scope consistent with the principles and novel features disclosed herein.

Claims

1. A method for integrated online control of large component assembly and testing based on digital twins, characterized in that, Includes the following steps: S1. Preprocess the original 3D model of the large component. Input the complete original 3D model data containing vertex set and face set. Perform boundary point identification, folding cost calculation and iterative simplification on the original 3D model to obtain the simplified 3D model of the large component. S2. Construct a multi-sensor measurement network including a laser tracker, a 3D scanner, and a CNC positioner; perform global coordinate system calibration and multi-sensor data fusion for the large component assembly scene; and construct a digital twin model of the large component assembly scene. S3. Discretize the digital twin model into a three-dimensional grid map, integrate the global coordinate system data information of static obstacles and dynamic equipment, and perform dynamic path planning and collision detection; S4. In real time, compare the actual pose of the large component measured by the laser tracker with the target pose of the large component in the digital twin model. When the deviation exceeds the threshold, trigger the control mechanism to perform real-time pose adjustment and closed-loop feedback control.

2. The integrated online control method for large component assembly and testing based on digital twins according to claim 1, characterized in that, The specific steps of step S1 are as follows: S11. Import the original 3D model of the large component, including the vertex set. V and the original set F Complete model data, the vertex set V and the original set F They are represented as follows: In the formula, Let i be the i-th vertex in the set. Let be the three-dimensional coordinates of the i-th vertex. Form a triangular facet for the i, j, and k vertices. The three vertices of the triangle facet are defined by the vertex set used to store the three-dimensional coordinates of the vertices. The facet set is composed of the vertex indices of the triangle facets, and each facet corresponds to the three vertices in the vertex set. S12. Calculate the number of adjacent edges of a triangle vertex using an improved triangle folding algorithm, identify key boundary points, and classify vertices with fewer than 6 adjacent edges as boundary points, forming a boundary point set. B : In the formula, This represents the number of adjacent edges to the vertices of the triangular face. S13. Calculate the folding geometric error of the new vertex after folding each triangular unit. C By iterating the folding geometric error C The smallest triangle, dynamically updating the vertex set. V noodle set F This continues until the number of facets is reduced to a preset number, which is set according to the original 3D model, wherein the folding geometric error... C The calculation formula is: In the formula, For triangular facets The number of adjacent triangles, The new vertex after folding These are the coordinates of the vertices of the adjacent triangles.

3. The integrated online control method for large component assembly and testing based on digital twins according to claim 1, characterized in that, Step S2 includes: S21. In the large component assembly scenario, a laser tracker, a 3D scanner, and a CNC positioner are deployed. The laser tracker is used to measure the actual pose of the large component, the 3D scanner is used to acquire point cloud data of the large component assembly scenario, and the CNC positioner is used to drive the AGV and the robotic arm to assemble the large component. S22. M standard target balls are arranged in the large component assembly scene. The laser tracker measures the coordinates of the standard target balls, solves the transformation matrix by the least squares method, inputs the measured coordinates and theoretical coordinates of the standard target balls, and performs global coordinate calibration. Here, M is the number of standard target balls. S23. Establish a time synchronization mechanism based on the IEEE 1588 protocol to map the point cloud data acquired by the 3D scanner to the global coordinate system; S24. Using 3D modeling software, create a virtual model of the large component assembly scene, set the initial state of the model, input the data after time synchronization and coordinate transformation into the virtual model, and construct a digital twin model of the large component assembly scene. S25. Update the data of the digital twin model in real time to ensure that the delay between physical and virtual mapping does not exceed the specified time and the spatial registration error is controlled within the specified value. The specified time and specified value are set according to the actual situation. Integrate the spatial pose data of AGV and robotic arm in real time.

4. The integrated online control method for large component assembly and testing based on digital twins according to claim 1, characterized in that, The specific steps of step S3 are as follows: S31. Use the improved A* algorithm for path planning and improve the traditional cost function by introducing positional bias. and timestamp penalty items The path weights are dynamically adjusted to generate an initial path. The improved cost function expression is as follows: In the formula, Indicates the distance from the starting point to the current node. The actual path cost is dynamically calculated during the planning process; The table shows the current node. Heuristic cost estimation to the destination This represents the deviation between the actual position of the robotic arm and the planned path. For node timestamp penalty items, For deviation weights, Timestamp delay weight; , , For the current node coordinates, , , The coordinates of the endpoint. , , These are the coordinates of the actual position of the robotic arm; S32. Apply fifth-order polynomial interpolation to the preliminary path. The robot arm joint trajectory was optimized, and accurate surface collision detection was performed based on GPU parallel computing to calculate the separation axis distance between the robot arm link envelope and the large component assembly scene model. ,like If a collision occurs, the system will re-plan the path. As a safety threshold, the fifth-order polynomial interpolation is expressed as: In the formula, For polynomial coefficients, For normalized time variables, , Total exercise duration; The separation axis distance Represented as: In the formula, Let be the coordinates of the vertices of the envelope surface of the robotic arm link. The normal vector of the surface of the assembly scene model for large components.

5. The integrated online control method for large component assembly and testing based on digital twins according to claim 1, characterized in that, The specific steps of step S4 are as follows: S41. Use an adaptive PID algorithm to generate attitude adjustment commands. The proportional gain coefficient is dynamically adjusted according to the assembly stage. During the coarse adjustment stage, the proportional gain is increased to accelerate convergence, and during the fine adjustment stage, the differential term is enhanced to suppress overshoot. The attitude adjustment command... The six-dimensional pose adjustment is represented as: In the formula, It is a proportional gain matrix. The differential gain matrix is... This is a six-dimensional pose deviation vector. For positional deviation, This is for attitude deviation; S42. The posture adjustment command is sent to the CNC positioner via the Ethernet protocol. The CNC positioner drives the AGV and the robotic arm to complete the posture adjustment in coordination. After each posture adjustment, a secondary measurement is immediately performed to verify the actual posture.

6. The integrated online control method for large component assembly and testing based on digital twins according to claim 5, characterized in that, In step S41: When the pose deviation is greater than the first threshold At that time, it is determined that the process has entered the coarse adjustment stage; When the pose deviation is consistently less than the first threshold And greater than the second threshold At that time, the system will switch to the fine-tuning stage. Among them, the first threshold Greater than the second threshold The first threshold With the second threshold Configure according to process requirements.

7. The integrated online control method for large component assembly and testing based on digital twins according to claim 1, characterized in that, It also includes the following steps: The AR interface provides real-time assembly guidance, and the camera recognizes assembly actions to trigger alarms, pause equipment, and lock permissions for issuing posture adjustment commands in case of errors or dangerous behavior.

8. The integrated online control method for large component assembly and testing based on digital twins according to claim 7, characterized in that, The AR interface provides real-time assembly guidance, and the camera recognizes assembly actions. The specific steps for triggering alarms, pausing equipment, issuing posture adjustment commands, and locking permissions for errors or dangerous behaviors are as follows: By using a spatial positioning system, the coordinate system of the AR device is aligned with the aforementioned global coordinate system, and the 3D model of the large component and the digital twin data are associated, and virtual guidance information is superimposed in the AR field of view. Deploy industrial cameras at assembly stations to cover the workers' workspace; If the operator's arm is detected moving within the robotic arm's movement area, the AR interface will issue a prompt, trigger an alarm, pause the equipment operation, and lock the authority to issue posture adjustment commands until the on-site adjustment is completed before proceeding to the next assembly step.

9. A large component assembly and testing integrated online control system based on digital twins, characterized in that, include: The model preprocessing module preprocesses the original 3D model of the large component. It takes the complete original 3D model data containing vertex set and face set as input, performs boundary point identification, folding cost calculation and iterative simplification on the original 3D model, and obtains the simplified 3D model of the large component. The model building module is used to build a multi-sensor measurement network that includes a laser tracker, a 3D scanner, and a CNC positioner, to perform global coordinate system calibration and multi-sensor data fusion for large component assembly scenarios, and to build a digital twin model of the large component assembly scenarios. The path planning module is used to discretize the digital twin model into a three-dimensional grid map, integrate global coordinate system data information of static obstacles and dynamic equipment, and perform dynamic path planning and collision detection. The attitude control module is used to compare the actual pose of the large component measured by the laser tracker with the target pose of the large component in the digital twin model in real time. When the deviation exceeds the threshold, the control mechanism is triggered to perform real-time attitude adjustment and closed-loop feedback control.

10. The integrated online control system for large component assembly and testing based on digital twins according to claim 9, characterized in that, It also includes a human-machine collaboration module, which provides real-time assembly guidance through an AR interface, combines camera recognition of assembly actions, triggers alarms for errors or dangerous behaviors, suspends equipment, and locks permissions for issuing posture adjustment commands.

Images