Simulation and virtual-real interaction method for composite material robot fiber placement machine

Abstract

The invention provides a simulation and virtual-real interaction method for a composite material robot fiber placement machine, and belongs to the technical field of automatic placement of composite materials. According to the method, a three-dimensional engine architecture is constructed on the basis of OpenGL and C # programming language according to simulation and virtual-real synchronization requirements of a composite robot fiber placement machine in the aircraft manufacturing industry, and orderly execution of model events is realized by adopting a mode of combining a serial process, a queue and recursion; integrating rendering optimization functions of point-line-plane rendering, OBJ model introduction and Von illumination and spotlight superposition; solving forward and inverse kinematics of the six-axis mechanical arm through a geometric method and an analytical method, and driving kinematics simulation of the fiber placement machine; data interaction and virtual-real synchronization of a virtual model and a physical model are realized based on industrial communication protocols such as ModBusRTU, TCP / IP and Siemens S7, and a man-machine interaction platform supporting a manual / automatic mode is established. The method can intervene in laying path and robot posture simulation in advance, safety accidents are avoided, dependence on foreign software is not needed, and the fiber laying efficiency and the intelligent level are improved.

Description

TECHNICAL FIELD

[0001] The application belongs to the technical field of composite material automatic laying, and more particularly relates to a method for simulating and interacting between virtuality and reality of a composite material robot fiber laying machine. BACKGROUND

[0002] In recent years, China's aircraft manufacturing industry has transformed from "Made in China" to "China's wisdom" and upgraded, and the composite material automatic laying technology has developed rapidly. The demand for domestic robot fiber laying machines is growing rapidly. The robot fiber laying machine is a device that lays multiple carbon fiber tows on the surface of a mold. It is applied to the inside of an aircraft barrel section or other structures that require certain strength, which can reduce the weight of the aircraft while ensuring structural strength.

[0003] At present, domestic fiber laying machines have achieved some results, and their carriers are usually six-axis mechanical arms or gantry-type large six-axis / multi-axis machine tools. For complex molds, a turntable and a transverse movement mechanism are added to form an eight-axis or more linkage system. Due to the risk of pressure failure, interlayer stacking, and collision during the fiber laying process, it is difficult to ensure the quality of the fiber laying through physical trial and error. Therefore, accurate robot pose simulation needs to be performed in software or platforms to verify the rationality of the laying path and robot pose in advance.

[0004] The closest prior art to the present application has obvious defects, as follows:

[0005] Using Unity as the simulation and virtual reality interaction carrier: Unity is a mature commercial game development engine that is used for industrial digital twins and dynamic displays in recent years. First, its performance is not as good as that of OpenGL, a native bottom-layer graphics engine library. Second, Unity is a highly encapsulated independent game engine with its own client software, which cannot be well integrated with target software and may have a future risk of charging.

[0006] Secondary development based on CATIA and other three-dimensional modeling engines: This solution is highly dependent on foreign platforms and software, has limited scalability, and cannot be independently published as a software or platform with independent intellectual property rights, which is not conducive to the development of domestic fiber laying machine technology.

[0007] In summary, the existing technology has performance deficiencies, relies on foreign software, has poor scalability, and cannot be independent and self-developed, which cannot meet the needs of high-precision simulation and virtual reality interaction of domestic robot fiber laying machines. SUMMARY

[0008] In order to solve the above technical problems, the application provides a composite material robot fiber placement machine simulation and virtual-real interaction method, which solves the technical problems in the related technology of the existing composite material robot fiber placement machine, such as relying on foreign software such as Unity or CATIA, needing to pass through physical machine trial and error in the early stage of placement, leading to low efficiency and difficult to avoid safety accidents such as collision, dispersed placement planning process needing additional software, and lack of low-latency virtual-real synchronization function.

[0009] A composite material robot fiber placement machine simulation and virtual-real interaction method, comprising the following steps:

[0010] S1: build a three-dimensional engine architecture based on a serial process and a queue recursion, submit events such as rendering, transformation, and deletion of the model to the task queue through the loop frame event driving of the OpenTK library, recursively traverse all models from the scene root node and execute the queue task;

[0011] S2: encapsulate the rendering data submission function, uniformly manage the vertex data, normal data, color data, and VAO required by the GPU through the IOC container, and realize the ordered submission of multi-model data;

[0012] S3: encapsulate the point-line-surface rendering function, based on the GL.DrawElements function, switch the rendering parameters according to user selection, automatically generate the VAO object of the standard model and the OBJ grid model and bind the GPU to complete the rendering;

[0013] S4: import the OBJ models of the fiber placement robot, mold, turntable, etc., extract the model data to form a model class through the Assimpt library, automatically generate the VAO and floating point data required for rendering, and join the scene rendering queue after specifying the parent level;

[0014] S5: build a virtual environment lighting system, adopt the way of superimposing the simplified variant of the Phong lighting and spotlight to optimize the model rendering effect;

[0015] S6: establish a forward and inverse kinematics model of a six-axis robot, solve the joint angle through the geometric method and the analytical method respectively, and drive the robot kinematics simulation by using the trajectory point coordinates and attitude data;

[0016] S7: based on the industrial communication protocol, establish the data interaction between the virtual model and the physical model, realize the virtual-real synchronization of the joint angle, sensor, motor, etc.

[0017] S8: integrate the data model, camera motion, and data flow interaction, and build a human-computer interactive program platform supporting manual mode and automatic mode.

[0018] Preferably, the three-dimensional engine architecture in step S1 adopts a consumer-production mode, each model independently configures rendering, rotation and translation, and deletion events, and the model needs to specify a parent when created to ensure that all models can be recursively found from the scene root node and execute queue tasks.

[0019] Preferably, in step S3, the triangular construction rule is adopted when rendering the surface, and the vertex data is stored in a structured array form, including vertex coordinates, normal and color.

[0020] The index array is generated in the order of 0, 1, 2, 3, 4, 5, …, n-3, n-2, n-1.

[0021] When rendering the line, the edge index array of each triangle patch is generated in the order of i, i+1, i, i+2, i+1, i+2, realizing the conversion of triangle patch to three edges.

[0022] Preferably, in step S4, when importing the OBJ model, only the model object needs to be created in the program and the file path and parent are specified, and the model can be automatically added to the scene rendering queue to complete the rapid deployment of the model.

[0023] Preferably, in step S5, the expression of the Phong illumination is:

[0024]

[0025] where K a , K d , K s are stored in the mtl material file, n is the vertex normal vector, l is the light source direction, v is the camera view direction, and r is the reflection vector.

[0026] The expression of the spotlight effect is:

[0027]

[0028] where φ is the inner cone angle, θ is the outer cone angle, and d is the distance from the fragment to the light source.

[0029] Preferably, in step S6, when solving the inverse kinematics of the six-axis mechanical arm, O0~O2 are responsible for the end displacement, and O3~O5 are responsible for the end filament head posture. The wrist intersection point position is obtained by the formula P wrist =P i -d tool N i , and θ1, θ2, θ3 are obtained through geometric relationship, and θ4, θ5, θ6 are obtained through the matrix:

[0030]

[0031]

[0032] ​Preferably, the industrial communication protocol in step S7 includes ModBus RTU, TCPIP, Siemens S7 communication protocol, the joint angle is bound with the front-end control through the WPF binding mechanism to realize the real-time display of the joint angle, in step S7, when the virtual and real synchronization, the joint angle of the physical model is collected through the encoder, and after being processed by the control system, it is transmitted to the OpenGL rendering system through the S7 communication interface, and is converted into a rotation matrix to update the posture of the virtual model, and the synchronization frame rate is not less than 60 frames per second.

[0033] Preferably, in step S8, in the manual mode, only the physical model data is allowed to flow to the virtual model, and the virtual model is prohibited from controlling the physical model.

[0034] In the automatic mode, based on the trajectory point coordinates and posture data in the G code file, the virtual model and the physical model are driven to move synchronously.

[0035] Preferably, it further includes a carbon fiber tows simulation drawing step: the end effector coordinates of the mechanical arm are recorded in real time, the adjacent coordinate differences are compared, the vertex array in the VAO is updated and re-registered to the GPU, and the dynamic tow trajectory is drawn by drawing a line between two points.

[0036] Compared with the prior art, the present application has the following beneficial effects:

[0037] Improve the pre-laying planning efficiency and avoid safety risks: the present application intervenes in the simulation of the laying path of the fiber laying machine and the robot posture design in advance, without the need of using physical machines to repeat trial and error, greatly improving the path planning and robot posture calculation efficiency; the collision risk of the robot and the mold can be observed intuitively in the simulation software, and safety accidents such as pressure failure and interlayer stacking can be avoided in advance to ensure the stable operation of the laying path and G code.

[0038] Realize technical self-determination and get rid of dependence on foreign software: based on the OpenGL native bottom layer graphics engine library and C# development, instead of Unity (highly encapsulated, potential charge) or CATIA (foreign platform, parasitic development) scheme, the development degree of freedom is high, and the software / platform with independent intellectual property rights can be independently published, solving the dependence on foreign software in the prior art and meeting the demand of Chinese intelligent self-determination.

[0039] Integrate the whole process function to reduce the dependence on additional software: the calculated path points and normal vectors of the fiber laying machine in the early laying planning are integrated with the whole process, including calculating the robot posture according to the path and normal vector, generating G code and simulation verification, forming a standard process, without the need of additional software assistance, from path calculation to simulation, which can be completed at one time, and the production efficiency is significantly improved.

[0040] High-precision rendering and model restoration ensure the realism of the simulation: Built-in point, line and surface drawing and OBJ mesh model import functions can recognize MTL material files and restore the appearance of real models one-to-one; the rendering method of using Von Tethys lighting and spotlight simplified variant overlay enhances the model's contour sense and realism. When importing a robot model containing two million triangles in a standard hardware environment, the frame rate can still be maintained above 140, meeting the needs of high-precision simulation.

[0041] Achieve low-latency virtual-real synchronization and enhance intelligence and visualization: Based on industrial communication protocols, achieve accurate mapping between physical equipment data and the virtual environment, with a virtual-real synchronization frame rate of over 60 frames per second, achieving ultra-low latency; Real-time display of joint angles through WPF binding mechanism, generation of tension motor speed change curves and recording of historical data, enhancing the intelligence and visualization of the wire laying process.

[0042] Safe and controllable human-computer interaction, adapted to actual production scenarios: The human-computer interaction platform supports manual and automatic modes. In manual mode, only physical model data is allowed to flow to the virtual model, and the virtual model is prohibited from controlling physical equipment to ensure operational safety. In automatic mode, virtual and physical models can be driven to move synchronously based on G-code, adapting to the needs of different scenarios in actual production and reducing the trial and error cost of physical equipment. Attached Figure Description

[0043] Figure 1 This is the core architecture diagram of the present invention;

[0044] Figure 2 This is a schematic diagram of data extraction from the OBJ model in this invention;

[0045] Figure 3 This is a geometric schematic diagram of the inverse kinematics solution of the six-axis robotic arm in this invention;

[0046] Figure 4 This is a schematic diagram illustrating the integration of the OpenGL rendering engine and functional modules in this invention;

[0047] Figure 5 This is a schematic diagram of the user-virtual-physical system data flow in the virtual-physical interaction of this invention;

[0048] Figure 6 This is a flowchart of the digital twin communication processing based on the S7 protocol in this invention. Detailed Implementation

[0049] The embodiments of the present invention will be described in further detail below with reference to the accompanying drawings and examples. The following examples are for illustrative purposes only and should not be construed as limiting the scope of the invention.

[0050] Please see Figures 1-6This invention provides a method for simulation and virtual-real interaction of composite material robot fiber placement machine, which is applicable to the simulation and virtual-real synchronous control of carbon fiber tow laying on the mold surface in the aircraft manufacturing industry. The specific implementation of this invention will be described in detail below with reference to specific technical details and accompanying drawings.

[0051] This implementation method is based on the OpenGL graphics library and the C# programming language to build the core technology framework. It relies on the loop frame event-driven mechanism provided by the OpenTK library (running at 60 frames / second by default, with a single frame event execution time of 0.016 seconds) and Microsoft's built-in IOC (Inversion of Control) container to achieve unified management of data and functions. At the same time, it integrates the Assimpt library for OBJ model data extraction and supports industrial communication protocols such as ModBusRTU, TCP / IP, and Siemens S7, ultimately realizing the simulation and virtual-real interaction functions of the wire laying machine.

[0052] The application scenario of this implementation is as follows: For composite material fiber placement machines with six-axis robotic arms or gantry-type multi-axis machine tools (complex mold scenarios require the use of turntables and transverse mechanisms to form an eight-axis or higher linkage system), the placement path and robot posture are verified in advance through simulation to avoid safety accidents such as pressure failure, interlayer stacking, and collision, and to realize real-time data interaction between the virtual model and the physical equipment.

[0053] Specific implementation steps:

[0054] Step 1: Build a 3D engine architecture based on serial flow and queue recursion.

[0055] This step aims to build the core architecture of the program, achieving ordered execution of model events and decoupling from the system, as detailed below:

[0056] Architecture design principles: Adopt a consumer-production model. Instead of directly calling model rendering, rotation, translation, deletion, and other events in OpenTK frame events, all events and tasks are submitted to a task queue. The queue tasks are executed in a loop in each frame through a serial process, ensuring the cleanliness and scalability of the program architecture.

[0057] Scene traversal mechanism: Before creating any model, a parent model must be specified. All models are recursively traversed from the scene root node downwards (regardless of the child-parent hierarchy depth). As long as the top-level parent points to the scene root node, it can be scanned by the task queue and the corresponding event (such as rendering, transformation) can be executed.

[0058] Core architecture process: After the process starts, the OpenGL environment is initialized first, and then the task queue executes tasks such as "model rendering, model transformation, model deletion, and mouse event response" in a loop. During the execution, the state of all models under the root node of the scene is continuously recursively checked until the simulation ends.

[0059] Step 2: Encapsulate the rendering data submission function

[0060] This step addresses the issue of orderly data submission during multi-model rendering through unified data management, as detailed below:

[0061] Data type definition: The data required for GPU rendering is collectively referred to as VAO (Vertex Array Object) and floating-point data, including the vertex data, normal data, and color data of the model.

[0062] IOC container application: Utilizing Microsoft's built-in IOC container, all the data required by the GPU is encapsulated in a unified class. This class is injected into the entire rendering system as a global service for all model objects to call, avoiding chaos when submitting multiple model data.

[0063] Data submission function: Calls OpenGL functions such as GL.BindVertexArray and GL.BindBuffer to register the encapsulated VAO and floating-point data to the GPU buffer, ensuring that all data has been submitted before rendering.

[0064] Step 3: Encapsulate point, line, and surface rendering functions

[0065] This step implements multi-mode rendering of the standard model and the OBJ mesh model, as detailed below:

[0066] Rendering function basics: Rendering is implemented based on the GL.DrawElements function (which supports passing formal parameters). Before calling it, you need to ensure that all model data and shader variables have been submitted to the GPU.

[0067] Rendering mode switching: The rendering functions for points, lines, and surfaces are uniformly encapsulated into a single class using an IOC container, and the parameters of GL.DrawElements are switched according to the user's selection.

[0068] When rendering a "face", pass in the "Triangles" parameter to index the "face VAO attribute data" of the model class; if this data does not exist, it will be created automatically.

[0069] Vertex data is stored in a structured array Vertex[n] = {V1, V2, V3, ...}, and each vertex Vn contains vertex coordinates, vertex normal, and vertex color;

[0070] The face is constructed using the triangle rule, and the index array is generated according to Index[m] = {0,1,2,3,4,5,…,n-3,n-2,n-1}. Finally, the VAO is bound to the GPU to complete the face rendering.

[0071] When rendering a "line", pass in the "LineStrip" parameter to automatically create an "edge VAO object". The index array is converted to {i,i+1,i+2} for each triangle facet (each triangle facet generates three edges).

[0072] When rendering a "point", a "point VAO object" is created in the same way, and rendering is completed after performing a non-null check.

[0073] Model type adaptation: Automatically identifies the model type (normal standard model / OBJ mesh model) during rendering and calls the corresponding VAO data to ensure rendering compatibility of different model types.

[0074] Step 4: Importing the OBJ model and constructing the virtual environment lighting

[0075] This step imports and optimizes the rendering of the core model of the filament placement machine (robot, mold, turntable), as detailed below:

[0076] OBJ model import process:

[0077] Use the Assimpt library to extract vertex, normal, and material data from the OBJ model and form an independent "model class";

[0078] Automatically generate the VAO and floating-point data required for rendering points, lines, and surfaces of this model class;

[0079] Within the program, create a model object, specify the OBJ file path and the parent model (e.g., the parent of the turntable model is the scene root node, and the parent of the robot model is the turntable model), and then add the model to the scene rendering queue.

[0080] Lighting system construction:

[0081] Using von Neumann lighting and spotlight simplified variant overlay, the model's contours and realism are enhanced:

[0082] Feng's formula for calculating illumination is:

[0083]

[0084] Where K a K d K s (Ambient light, diffuse light, specular light coefficients) are stored in model material files with the .mtl extension, and each represents an RGB value;

[0085] I ligtht Base lighting color;

[0086] n is the vertex normal vector, l is the custom light source direction, v is the camera's line of sight direction, and r is the reflection vector.

[0087] The formula for superimposing spotlight effects is:

[0088]

[0089] Where φ represents the inner cone angle, θ represents the outer cone angle, and d is the distance from the model segment to the light source;

[0090] K C K l K q As an attenuation coefficient, it ensures that the closer the segment is to the light source, the stronger the illumination, while the segment outside the outer cone angle appears black (without superimposing the spotlight's RGB values).

[0091] Engine performance verification: Under standard hardware conditions, when importing a filament-laying robot model containing 2 million triangular facets, the 3D engine frame rate is maintained above 140 frames per second, meeting the simulation performance requirements.

[0092] Step 5: Construct forward and inverse kinematics models of the robot

[0093] This step uses geometric and analytical methods to solve for the joint angles of the six-axis robotic arm, providing a data foundation for kinematic simulation, as detailed below:

[0094] The robotic arm has the following functional division: the laying head is fixed relative to the sixth joint O6, where O0 to O2 are responsible for the displacement control of the end effector, and O3 to O5 are responsible for the posture adjustment of the end-feeding head.

[0095] Inverse kinematics solution process:

[0096] Step 5.1: Determine the wrist crossing point position. Based on the coordinates of the trajectory points and the end effector posture, use formula P... wrist =P i -d tool N i Calculate the wrist crossing point P wrist (Relative to the base coordinate system), where:

[0097] P i The position vector of the trajectory point;

[0098] d tool This represents the Z-direction displacement of the wire-laying head.

[0099] N i It is the unit vector in the Z direction of the end coordinate system.

[0100] Step 5.2: Solve for the first three joint angles (θ1, θ2, θ3)

[0101] P wirst Projected onto a two-dimensional plane, θ1=Atan2(y wrist xwris );

[0102] In the formula, l1 and d4 are known, and a2 is known (mechanical arm structural parameters). The formula is:

[0103] Calculate l2;

[0104] Through the formula:

[0105] Solve for ψ, and then obtain

[0106] θ3 = α + β;

[0107] Where α = sin -1 [(Z wrist -Z o1 ) / l2];

[0108]

[0109] Step 5.3: Solve for the last three joint angles (θ4, θ5, θ6)

[0110] Based on the matrix derived from the forward kinematics The matrix expression is:

[0111]

[0112] Where Cn=cosθn, Sn=sinθn;

[0113] Based on the above piecewise matrix solution:

[0114]

[0115] Solution verification: The rationality of the joint angle calculation is verified by using the DH two-dimensional planar diagram to ensure that the movement of the robotic arm is free from interference.

[0116] Step 6: Kinematic Simulation of the Fiber Placement Robot

[0117] This step, based on the aforementioned kinematic model, realizes the dynamic simulation and end-point trajectory plotting of the fiber placement machine, as detailed below:

[0118] Model hierarchy management:

[0119] Import the filament-laying robot (six-axis) and filament-laying head model. Split the six-axis robotic arm into six independent OBJ models and establish a parent-child relationship in the order of "6-axis → 5-axis → 4-axis → 3-axis → 2-axis → 1-axis → base" (the base is the top-level parent, pointing to the root node of the scene) to ensure hierarchical linkage during axis movement.

[0120] Simulation-driven logic:

[0121] The trajectory point coordinates and attitude data in the G-code file are read, and the six joint angles (θ1~θ6) obtained from the inverse kinematics calculation are assigned to the corresponding axis models to drive the robotic arm to move along the preset path.

[0122] End trajectory drawing:

[0123] The coordinates of the robotic arm's end effector are recorded in real time, and the current coordinates are compared with the coordinates of the previous frame. If the coordinates are different, the VAO (rearranged vertex array) in the model data structure is updated, and the updated VAO is re-registered to the GPU. The line between the two points is drawn to form a dynamic end effector motion trajectory.

[0124] Step 7: Virtual-physical interaction between the virtual model and the physical model

[0125] This step utilizes industrial communication protocols to achieve data mapping and virtual-real synchronization, as detailed below:

[0126] Communication framework setup:

[0127] Based on the C# ecosystem, it supports communication protocols such as ModBusRTU, TCP / IP, and Siemens S7. Users can select the corresponding protocol according to the type of physical device (e.g., Siemens controllers should preferentially select the S7 protocol).

[0128] Communication process: OpenGL client creates S7 communication → opens put / get communication interface → waits for physical device connection → receives data such as joint angle, sensor, motor, etc. → synchronizes to virtual model → records historical data to log file.

[0129] Data mapping and synchronization:

[0130] Data acquisition: Each joint of the physical robotic arm is equipped with an encoder to collect joint angles in real time; the wire laying head is equipped with a temperature sensor and a tension motor (using a PID control strategy) to collect temperature and motor speed data.

[0131] Data transmission: The data collected by the physical devices is processed by the control system and then transmitted to the OpenGL rendering system through the S7 communication interface.

[0132] Virtual-real synchronization: The OpenGL system converts the received joint angles into rotation matrices and dynamically updates the joint pose of the virtual model; the synchronization frame rate is no less than 60 frames / second, achieving ultra-low latency synchronization.

[0133] Data visualization and recording:

[0134] By using the WPF binding mechanism, the joint angle is bound to the front-end control (such as a numerical display box or dashboard) to display the joint angle in real time.

[0135] The system reads the speed value of the tension motor in real time and generates a speed change curve; it also creates a Txt file to write joint data and sensor data to achieve historical data recording.

[0136] Step 8: Building a Human-Computer Interactive Program Platform

[0137] This step integrates all functional modules to form an operable human-computer interaction platform, as detailed below:

[0138] Pattern Design:

[0139] Manual mode: To ensure safety, only physical model data flow is allowed to the virtual model (the virtual model moves synchronously with the physical model), and the virtual model is prohibited from sending control commands to the physical model;

[0140] Automatic mode: Based on trajectory points and attitude data from G-code, it drives the virtual model and physical model to move synchronously, supporting fully automatic simulation and execution of path laying.

[0141] Function integration:

[0142] It integrates data models (joint, sensor, and motor data), camera motion (user-controllable virtual camera view), and data flow interaction (communication protocol), and provides operation buttons such as "model import, simulation start / pause, data viewing, and historical log export".

[0143] The three-dimensional display shows the motion process of the physical model and sensor data, including the motion simulation of the fiber placement robot and the simulation of carbon fiber bundles (dynamic line drawing).

[0144] Debugging support:

[0145] It supports the joint debugging of virtual and physical models, and can detect problems such as collisions and layer stacking in the laying path in advance through simulation. It can also optimize G code based on the virtual environment without the need for trial and error with physical devices.

[0146] Implementation effect verification:

[0147] Under standard hardware conditions (CPU: Intel Core i7-12700K, GPU: NVIDIA RTX 3070, memory: 32GB), the specific effects of this implementation method are as follows:

[0148] Simulation performance: When importing a wire-laying robot model with 2 million triangular facets, the rendering frame rate is stable at over 140 frames per second, meeting the requirements for high-precision simulation.

[0149] Virtual-physical synchronization: Joint angle synchronization delay <17ms (60 frames / second), and posture error between virtual and physical models <0.1°;

[0150] Functional completeness: It realizes a closed loop of the entire process from path planning (calculating path points and normals), attitude solving (calculating robot attitude), G-code generation to simulation verification, without the need for additional software support;

[0151] Safety: By using simulation to avoid collision accidents in advance, the number of trial and error attempts of physical equipment is reduced by more than 90%, and the pass rate of wire laying quality is increased to more than 98%.

[0152] The embodiments of the present invention are given for illustrative and descriptive purposes only, and are not intended to be exhaustive or to limit the invention to the forms disclosed. Many modifications and variations will be apparent to those skilled in the art. The embodiments were chosen and described in order to better illustrate the principles and practical application of the invention, and to enable those skilled in the art to understand the invention and to design various embodiments with various modifications suitable for a particular purpose.

Claims

1. A method for simulating and interacting with virtual reality in a composite material robot fiber placement machine, characterized in that, Includes the following steps: S1: Build a 3D engine architecture based on serial process and queue recursion. Through the loop frame event drive of the OpenTK library, submit the rendering, transformation, deletion and other events of the model to the task queue. Recursively traverse all models from the scene root node and execute queue tasks. S2: Encapsulates the rendering data submission function, and manages the vertex data, normal data, color data and VAO required by the GPU in a unified manner through the IOC container to achieve the orderly submission of multi-model data; S3: Encapsulates point, line, and surface rendering functions. Based on the GL.DrawElements function, it switches rendering parameters according to user selection, automatically generates VAO objects of standard models and OBJ mesh models, and binds them to the GPU to complete rendering. S4: Import OBJ models such as wire laying robot, mold, and turntable, extract model data through the Assimpt library to form model classes, automatically generate VAO and floating-point data required for rendering, and add them to the scene rendering queue after specifying the parent. S5: Construct a virtual environment lighting system and optimize the model rendering effect by overlaying a simplified variant of Von's lighting and spotlights. S6: Establish a forward and inverse kinematics model of a six-axis robotic arm, solve the joint angles using geometric and analytical methods respectively, and drive the robot's kinematics simulation using trajectory point coordinates and attitude data; S7: Establishes data interaction between the virtual model and the physical model based on industrial communication protocols to achieve virtual-real synchronization of data such as joint angles, sensors, and motors; S8: Integrates data models, camera motion, and data stream interaction to build a human-computer interactive program platform that supports manual and automatic modes.

2. The method according to claim 1, characterized in that, The 3D engine architecture described in step S1 adopts a consumer-production model. Each model is independently configured with rendering, rotation, translation, and deletion events. When a model is created, a parent level must be specified to ensure that all models can be recursively found from the scene root node and queue tasks can be executed.

3. The method according to claim 1, characterized in that, In step S3, the triangle construction rule is used when rendering the face, and the vertex data is stored in the form of a structured array, which includes vertex coordinates, normals and colors; The index array is generated in the order 0, 1, 2, 3, 4, 5, ..., n-3, n-2, n-1; When rendering lines, the edge index array of each triangle is generated according to i, i+1, i, i+2, i+1, i+2, realizing the transformation of the triangle into three edges.

4. The method according to claim 1, characterized in that, When importing the OBJ model in step S4, you only need to create the model object in the program and specify the file path and parent to automatically add it to the scene rendering queue and complete the rapid deployment of the model.

5. The method according to claim 1, characterized in that, The expression for the Von der Lune illumination in step S5 is: Where K a K d K s Stored in the MTL material file, where n is the vertex normal vector, l is the light source direction, v is the camera's view direction, and r is the reflection vector; The expression for the spotlight effect is: Where φ is the inner cone angle, θ is the outer cone angle, and d is the distance from the segment to the light source.

6. The method according to claim 1, characterized in that, In step S6, when solving the inverse kinematics of the six-axis robotic arm, O0 to O2 are responsible for the end effector displacement, and O3 to O5 are responsible for the end effector filament-laying head posture, using formula P. wrist =P i -d tool N i The wrist crossing point is determined, and then θ1, θ2, and θ3 are solved using geometric relationships, using the matrix: Solve for θ4, θ5, and θ6.

7. The method according to claim 1, characterized in that, In step S7, the industrial communication protocols include ModBusRTU, TCPIP, and Siemens S7 communication protocol. The joint angle is bound to the front-end control through the WPF binding mechanism to realize the real-time display of the joint angle.

8. The method according to claim 1, characterized in that, During virtual-real synchronization in step S7, the joint angles of the physical model are collected by the encoder, processed by the control system, and transmitted to the OpenGL rendering system through the S7 communication interface. The angles are then converted into rotation matrices to update the virtual model's pose, with a synchronization frame rate of no less than 60 frames per second.

9. The method according to claim 1, characterized in that, In step S8, in manual mode, only physical model data is allowed to flow to the virtual model, and the virtual model is prohibited from controlling the physical model. In automatic mode, the virtual model and the physical model move synchronously based on the trajectory point coordinates and attitude data in the G-code file.

10. The method according to claim 1, characterized in that, It also includes carbon fiber filament simulation drawing steps: real-time recording of the coordinates of the end effector of the robotic arm, comparison of the differences between adjacent coordinates, updating the vertex array in VAO and re-registering it to the GPU, and drawing the line connecting the two points to form a dynamic filament trajectory.

Images