Virtual simulation method, device and equipment for factory assembly line and readable storage medium

Abstract

The invention relates to a virtual simulation method, device and equipment for a factory assembly line and a readable storage medium. The method comprises the following steps: acquiring a plurality of technological processes and a technological diagram of each technological process; the current technological process is determined according to the execution sequence of the multiple technological processes, and based on the multiple position points corresponding to the current technological process, the virtual mechanical arm is controlled to move in the factory digital twinning scene model till execution of the multiple position points corresponding to the current technological process is completed; the factory digital twinning scene model is constructed based on a three-dimensional model of a virtual factory assembly line and a three-dimensional model of a virtual mechanical arm; the next technological process continues to be executed until the multiple technological processes are executed. By adopting the method, the process flow in the factory assembly line can be accurately simulated, the factory assembly line based on virtual simulation guides the layout and adjustment of the actual factory assembly line, and the safety of the factory assembly line can be improved.

Description

Technical Field

[0001] The present application relates to the field of virtual simulation technology, and in particular to a virtual simulation method, device, equipment and readable storage medium for a factory assembly line. Background Art

[0002] With the rapid development of the manufacturing industry and the intensification of market competition, traditional factory assembly lines can no longer meet the demand. Therefore, the introduction of robotic arms for automatic processing of process flows based on traditional assembly lines has been widely used.

[0003] However, the robotic arms in factory assembly lines are often pre-deployed. Due to the diversity of equipment in factory assembly lines and the complexity of process flows, when the process flows change, it is often necessary to redeploy the position of the robotic arms. Otherwise, interference with the robotic arms during movement may occur, resulting in damage to the robotic arms, assembly line equipment or process products. Therefore, there is an urgent need for a method to improve the safety of factory assembly lines. Summary of the invention

[0004] Based on this, it is necessary to provide a virtual simulation method, device, computer equipment, computer-readable storage medium and computer program product for a factory assembly line to address the above-mentioned technical problems, which can accurately simulate the process flow in the factory assembly line. The factory assembly line based on virtual simulation can guide the layout and adjustment of the actual factory assembly line, which is conducive to improving the safety of the factory assembly line.

[0005] In a first aspect, the present application provides a virtual simulation method for a factory assembly line, comprising:

[0006] Acquire multiple process flows and a process diagram of each process flow; the process diagram includes multiple location points;

[0007] Determine the current process flow according to the execution order of multiple process flows, and control the virtual robotic arm to move in the factory digital twin scene model based on multiple position points corresponding to the current process flow until the multiple position points corresponding to the current process flow are completed; the factory digital twin scene model is constructed based on the three-dimensional model of the virtual factory assembly line and the three-dimensional model of the virtual robotic arm;

[0008] The next process flow of the current process flow is determined according to the execution order of the multiple process flows, and the next process flow is continued to be executed until the multiple process flows are completed.

[0009] In one embodiment, based on multiple position points corresponding to the current process flow, controlling the movement of a virtual robot arm in a factory digital twin scene model includes:

[0010] Determine the motion path based on multiple position points corresponding to the current process flow;

[0011] Based on the motion path, the virtual robotic arm is controlled to move in the factory digital twin scene model.

[0012] In one embodiment, determining a motion path based on a plurality of position points corresponding to a current process flow includes:

[0013] Perform interpolation processing on multiple position points corresponding to the current process flow to obtain multiple interpolated position points;

[0014] Curve fitting is performed on multiple interpolated position points to obtain a motion path.

[0015] In one embodiment, based on the motion path, controlling the virtual robot arm to move in the factory digital twin scene model includes:

[0016] During the movement of the virtual robot arm, obstacles around the end device of the virtual robot arm are detected;

[0017] When the distance between the current position of the end device and the obstacle is less than the first preset distance, controlling the virtual robot arm to move a second preset distance in a direction away from the obstacle;

[0018] The updated position of the end device is obtained, and the virtual robot arm is controlled to move in the factory digital twin scene model according to the updated position and the remaining path in the motion path.

[0019] In one embodiment, the virtual simulation method of a factory assembly line further includes:

[0020] After receiving the target trigger event, obtain the starting point coordinates and the end point coordinates indicated by the target trigger event;

[0021] Determine the target motion path according to the starting point coordinates and the end point coordinates;

[0022] Based on the target motion path, the virtual robotic arm is controlled to move in the factory digital twin scene model.

[0023] In one embodiment, the steps of constructing the factory digital twin scenario model include:

[0024] Loading the 3D model of the virtual factory assembly line and the 3D model of the virtual robotic arm into the preset 3D engine;

[0025] A factory digital twin scene model is constructed based on the three-dimensional model of the virtual factory assembly line, the three-dimensional model of the virtual robotic arm, and the respective positions of the virtual factory assembly line and the virtual robotic arm in the virtual space.

[0026] In a second aspect, the present application also provides a virtual simulation device for a factory assembly line, comprising:

[0027] An acquisition module is used to acquire multiple process flows and a process diagram of each process flow; the process diagram includes multiple location points;

[0028] A control module is used to determine the current process flow according to the execution order of multiple process flows, and control the virtual robotic arm to move in the factory digital twin scene model based on multiple position points corresponding to the current process flow until the multiple position points corresponding to the current process flow are executed; the factory digital twin scene model is constructed based on the three-dimensional model of the virtual factory assembly line and the three-dimensional model of the virtual robotic arm;

[0029] The loop module is used to determine the next process flow of the current process flow according to the execution order of multiple process flows, and continue to execute the next process flow until the multiple process flows are completed.

[0030] In a third aspect, the present application further provides a computer device, including a memory and a processor, wherein the memory stores a computer program, and when the processor executes the computer program, the following steps are implemented:

[0031] Acquire multiple process flows and a process diagram of each process flow; the process diagram includes multiple location points;

[0032] Determine the current process flow according to the execution order of multiple process flows, and control the virtual robotic arm to move in the factory digital twin scene model based on multiple position points corresponding to the current process flow until the multiple position points corresponding to the current process flow are completed; the factory digital twin scene model is constructed based on the three-dimensional model of the virtual factory assembly line and the three-dimensional model of the virtual robotic arm;

[0033] The next process flow of the current process flow is determined according to the execution order of the multiple process flows, and the next process flow is continued to be executed until the multiple process flows are completed.

[0034] In a fourth aspect, the present application further provides a computer-readable storage medium having a computer program stored thereon, wherein when the computer program is executed by a processor, the following steps are implemented:

[0035] Acquire multiple process flows and a process diagram of each process flow; the process diagram includes multiple location points;

[0036] Determine the current process flow according to the execution order of multiple process flows, and control the virtual robotic arm to move in the factory digital twin scene model based on multiple position points corresponding to the current process flow until the multiple position points corresponding to the current process flow are completed; the factory digital twin scene model is constructed based on the three-dimensional model of the virtual factory assembly line and the three-dimensional model of the virtual robotic arm;

[0037] The next process flow of the current process flow is determined according to the execution order of the multiple process flows, and the next process flow is continued to be executed until the multiple process flows are completed.

[0038] In a fifth aspect, the present application further provides a computer program product, including a computer program, which implements the following steps when executed by a processor:

[0039] Acquire multiple process flows and a process diagram of each process flow; the process diagram includes multiple location points;

[0040] Determine the current process flow according to the execution order of multiple process flows, and control the virtual robotic arm to move in the factory digital twin scene model based on multiple position points corresponding to the current process flow until the multiple position points corresponding to the current process flow are completed; the factory digital twin scene model is constructed based on the three-dimensional model of the virtual factory assembly line and the three-dimensional model of the virtual robotic arm;

[0041] The next process flow of the current process flow is determined according to the execution order of the multiple process flows, and the next process flow is continued to be executed until the multiple process flows are completed.

[0042] The above-mentioned virtual simulation method, device, computer equipment, computer-readable storage medium and computer program product of the factory assembly line pre-construct a factory digital twin scene model based on the three-dimensional model of the virtual factory assembly line and the three-dimensional model of the virtual robotic arm, obtain multiple process flows and the process diagram of each process flow, determine the current process flow according to the execution order of multiple process flows, and control the virtual robotic arm to move in the factory digital twin scene model based on the multiple position points contained in the process diagram of the current process flow until the multiple position points corresponding to the current process flow are executed and then the next process flow is executed, thereby simulating each process flow in the factory digital twin scene model in turn. This method of accurately simulating multiple process flows in the factory assembly line can guide the layout and adjustment of the actual factory assembly line, which is conducive to improving the layout flexibility and safety of the factory assembly line. BRIEF DESCRIPTION OF THE DRAWINGS

[0043] In order to more clearly illustrate the technical solutions in the embodiments of the present application or related technologies, the drawings required for use in the embodiments of the present application or related technical descriptions will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other related drawings can be obtained based on these drawings without paying creative work.

[0044] FIG1 is an application environment diagram of a virtual simulation method for a factory assembly line in one embodiment;

[0045] FIG2 is a schematic diagram of a flow chart of a virtual simulation method for a factory assembly line in one embodiment;

[0046] FIG3 is a structural diagram of a virtual simulation system for a factory assembly line in one embodiment;

[0047] FIG4 is a digital twin process display effect diagram in one embodiment;

[0048] FIG5 is a block diagram of a virtual simulation device for a factory assembly line according to an embodiment;

[0049] FIG. 6 is a diagram showing the internal structure of a computer device in one embodiment. DETAILED DESCRIPTION

[0050] In order to make the purpose, technical solution and advantages of the present application more clearly understood, the present application is further described in detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present application and are not used to limit the present application.

[0051] The virtual simulation method of the factory assembly line provided in the embodiment of the present application can be applied to the application environment shown in Figure 1. Among them, the terminal 102 communicates with the server 104 through the network. The data storage system can store the data that the server 104 needs to process. The data storage system can be integrated on the server 104, or it can be placed on the cloud or other network servers. This embodiment is illustrated by applying the method to the terminal 102. It can be understood that the method can also be applied to the server, and can also be applied to the system including the terminal and the server, and is implemented through the interaction between the terminal and the server. The terminal 102 obtains multiple process flows and a process diagram of each process flow; the process diagram contains multiple position points; the current process flow is determined according to the execution order of the multiple process flows, and based on the multiple position points corresponding to the current process flow, the virtual robot arm is controlled to move in the factory digital twin scene model until the multiple position points corresponding to the current process flow are executed; the factory digital twin scene model is constructed based on the three-dimensional model of the virtual factory assembly line and the three-dimensional model of the virtual robot arm; the next process flow of the current process flow is determined according to the execution order of the multiple process flows, and the next process flow is continued to be executed until the multiple process flows are executed. The terminal 102 may be, but is not limited to, various personal computers, laptops, smart phones, tablet computers, IoT devices, and portable wearable devices. IoT devices may be smart speakers, smart TVs, smart air conditioners, smart vehicle-mounted devices, projection devices, etc. Portable wearable devices may be smart watches, smart bracelets, head-mounted devices, etc. Head-mounted devices may be virtual reality (VR) devices, augmented reality (AR) devices, smart glasses, etc.

[0052] In an exemplary embodiment, as shown in FIG2 , a virtual simulation method for a factory assembly line is provided, and the method is applied to the terminal 102 in FIG1 as an example for description, including the following steps 202 to 206. Among them:

[0053] Step 202, obtaining multiple process flows and a process diagram of each process flow; the process diagram includes multiple location points.

[0054] The process flow refers to the production process that needs to be processed and manufactured in the factory assembly line. The process flow is different for different process requirements. For example, the process flow can be cutting parts in steel plates or picking parts in a factory assembly line.

[0055] A process drawing is a drawing that indicates the specific content of a process flow. For example, a process drawing can be a part nesting drawing. The process drawings of multiple process flows can be the same or different.

[0056] The position point refers to the position point identified in the process drawing, which is used to indicate the movement of the robot arm. In some embodiments, the process drawing contains graphic outlines of multiple parts, and the terminal uses the key points in each graphic outline as the position point.

[0057] Step 204, determine the current process flow according to the execution order of multiple process flows, and based on multiple position points corresponding to the current process flow, control the virtual robotic arm to move in the factory digital twin scene model until the multiple position points corresponding to the current process flow are executed; the factory digital twin scene model is constructed based on the three-dimensional model of the virtual factory assembly line and the three-dimensional model of the virtual robotic arm.

[0058] Among them, multiple process flows have an execution order, and multiple process flows will be executed in sequence according to the execution order, thereby ensuring the orderly progress of the process flows, which is beneficial to the safety of the factory assembly line.

[0059] The current process flow refers to the process flow currently being executed. The multiple position points corresponding to the current process flow refer to the multiple position points included in the process diagram of the current process flow.

[0060] The virtual factory assembly line refers to the factory assembly line in the virtual environment. The 3D model of the virtual factory assembly line is obtained through 3D modeling according to the structural data and position data of each device in the actual factory assembly line. The virtual robotic arm is obtained through 3D modeling according to the structural data of the actual robotic arm.

[0061] The factory digital twin scene model is constructed based on the three-dimensional model of the virtual factory assembly line and the three-dimensional model of the virtual robotic arm. The factory digital twin scene model includes the virtual factory assembly line and the virtual robotic arm.

[0062] The terminal converts multiple location points to the virtual space where the factory digital twin scene model is located to obtain virtual location points. The virtual robotic arm can move according to the multiple virtual location points to realize the current process flow.

[0063] Step 206, determining the next process flow of the current process flow according to the execution order of the multiple process flows, and continuing to execute the next process flow until the multiple process flows are completed.

[0064] Among them, the next process flow refers to the next process flow of the current process flow determined in multiple process flows according to the execution order. The terminal takes the next process flow as the current process flow, and continues to execute the steps of controlling the movement of the virtual robot arm in the factory digital twin scene model in step 204 until multiple process flows are executed, thereby realizing the accurate simulation of multiple process flows in the factory digital twin scene model. If multiple process flows are successfully completed, it indicates that the layout of the factory assembly line and the robot arm in the factory digital twin scene model is reasonable. The layout of the actual factory assembly line is guided according to the layout method in the factory digital twin scene model, which is conducive to ensuring the safety of the factory assembly line. In addition, the process flow can be flexibly adjusted according to demand, and accordingly, the layout in the factory digital twin scene model can be flexibly adjusted, that is, before the actual factory assembly line is laid out, the above-mentioned virtual simulation method is used to simulate the corresponding process flow, which is conducive to pre-identifying safety hazards, thereby facilitating the safety of the actual factory assembly line.

[0065] In the above-mentioned virtual simulation method of the factory assembly line, a factory digital twin scene model is constructed in advance based on the three-dimensional model of the virtual factory assembly line and the three-dimensional model of the virtual robotic arm, multiple process flows and the process diagram of each process flow are obtained, the current process flow is determined according to the execution order of the multiple process flows, and the virtual robotic arm is controlled to move in the factory digital twin scene model based on the multiple position points contained in the process diagram of the current process flow until the multiple position points corresponding to the current process flow are executed and then the next process flow is executed, thereby simulating each process flow in the factory digital twin scene model in turn. This method of accurately simulating multiple process flows in the factory assembly line can guide the layout and adjustment of the actual factory assembly line, which is conducive to improving the layout flexibility and safety of the factory assembly line.

[0066] In an exemplary embodiment, based on multiple position points corresponding to the current process flow, a virtual robotic arm is controlled to move in a factory digital twin scene model, including: determining a motion path based on multiple position points corresponding to the current process flow; based on the motion path, controlling the virtual robotic arm to move in the factory digital twin scene model.

[0067] The terminal may perform curve fitting on multiple position points corresponding to the current process flow to determine the motion path. Alternatively, the terminal may determine the position sequence of multiple position points and use the path formed by connecting the multiple position points according to the position sequence as the motion path.

[0068] The virtual robotic arm moves in the factory digital twin scene model according to the motion path. For any pair of adjacent position points in the motion path, the position point with the earlier position sequence in the adjacent position points is taken as the starting point, and the position point with the later position sequence in the adjacent position points is taken as the end point. The virtual robotic arm is controlled to move to the starting point, and then the IK (Inverse Kinematics) algorithm of the robotic arm is used to use cyclic coordinate descent inverse kinematics to complete the driving of the virtual robotic arm through the specified number of iterations to ensure that the virtual robotic arm moves to the end point.

[0069] In this embodiment, the motion path is determined through multiple position points corresponding to the current process flow, and then the virtual robotic arm is controlled to run according to the motion path, which is conducive to ensuring that the virtual robotic arm accurately simulates the current process flow.

[0070] In an exemplary embodiment, based on multiple position points corresponding to the current process flow, a motion path is determined, including: interpolating the multiple position points corresponding to the current process flow to obtain multiple interpolated position points; and performing curve fitting on the multiple interpolated position points to obtain the motion path.

[0071] Among them, since the multiple position points are discrete points, the robot arm directly moves according to the multiple position points, which has the problem of poor path continuity. Therefore, the embodiment of the present application proposes to interpolate the multiple position points to obtain multiple interpolated position points. The terminal performs curve fitting on the multiple interpolated position points to obtain the motion path.

[0072] In this embodiment, interpolation processing is performed on multiple position points corresponding to the current process flow to increase the data volume of the position points, and curve fitting is performed based on the multiple interpolated position points, which is conducive to obtaining a smooth and continuous motion path.

[0073] In an exemplary embodiment, based on a motion path, a virtual robotic arm is controlled to move in a factory digital twin scenario model, including: during the movement of the virtual robotic arm, obstacles around the end device of the virtual robotic arm are detected; when the distance between the current position of the end device and the obstacle is less than a first preset distance, the virtual robotic arm is controlled to move a second preset distance away from the obstacle; and an updated position of the end device is obtained, and according to the updated position and the remaining path in the motion path, the virtual robotic arm is controlled to move in the factory digital twin scenario model.

[0074] The end device refers to the device installed at the end of the virtual robot arm. The end device is an operating device for the process flow, for example, it can be a magnetic gripper, a laser cutting head, a welding integrated head, etc. The terminal can determine the type of the end device according to the type of process flow.

[0075] During the movement of the virtual robotic arm, if the installation position of the virtual robotic arm is unreasonable, it may cause the end device to collide with surrounding objects. In some embodiments, the virtual robotic arm is installed with a virtual radar to simulate the obstacle detection capability of a real radar. The terminal obtains the detection result of the virtual radar on the obstacles around the end device. If the detection result indicates that there are obstacles around the end device, the terminal obtains the distance between the current position of the end device and the obstacle. If the distance is less than the first preset distance, it indicates that the end device is close to the obstacle. If it continues to approach the obstacle, it may collide with the obstacle. Therefore, the terminal controls the virtual robotic arm to move a second preset distance away from the obstacle to avoid the risk of collision.

[0076] The remaining path refers to the path after the current position in the motion path. The updated position refers to the position of the end device after the virtual robot moves away from the obstacle.

[0077] The terminal uses the updated position as the starting point of the remaining path and continues to control the movement of the virtual robot arm according to the remaining path.

[0078] In this embodiment, obstacle detection is performed during the movement of the virtual robotic arm, and when it is detected that the distance between the current position of the end device and the obstacle is less than a first preset distance, the virtual robotic arm is moved to adjust the layout of the virtual robotic arm. This is beneficial for guiding the layout of the actual robotic arm in the factory assembly line, avoiding collision interference, and thus improving the safety of the factory assembly line.

[0079] In an exemplary embodiment, the virtual simulation method of a factory assembly line also includes: after receiving a target trigger event, obtaining the starting point coordinates and the end point coordinates indicated by the target trigger event; determining the target motion path according to the starting point coordinates and the end point coordinates; and based on the target motion path, controlling the virtual robotic arm to move in the factory digital twin scene model.

[0080] Among them, the target trigger event refers to an event triggered upstream of the virtual factory assembly line. In some embodiments, after multiple process flows are executed, a target trigger event is generated in response to user needs. For example, multiple process flows are part cutting processes, and the target trigger event is to pick the cut parts into the box. The starting point coordinates and the end point coordinates indicated by the target trigger event are often different from the position points corresponding to the multiple process flows. The terminal can determine the target motion path based on the starting point coordinates and the end point coordinates. The target motion path may be different from the motion path corresponding to each process flow. The terminal controls the movement of the virtual robot arm in the factory digital twin scene model based on the target motion path corresponding to the target trigger event, and can simulate the execution process of the target trigger event, thereby determining whether the layout of the virtual factory assembly line and the virtual robot arm is reasonable. If there is a collision interference, the layout can be adjusted directly in the factory digital twin scene model, which improves the layout flexibility and layout efficiency.

[0081] In this embodiment, by receiving a target trigger event and determining the target motion path according to the starting point coordinates and end point coordinates indicated by the target trigger event, the movement of the virtual robotic arm is controlled. In this way, the execution process of the target trigger event can be simulated. The layout of the virtual factory assembly line and the virtual robotic arm can be flexibly adjusted according to the situation of the target trigger event, which greatly improves the layout flexibility and layout efficiency. The layout in the factory digital twin scenario model is used to guide the layout and adjustment of the actual factory assembly line, which is conducive to improving the layout flexibility and safety of the factory assembly line.

[0082] In an exemplary embodiment, the steps of constructing a factory digital twin scene model include: loading a three-dimensional model of a virtual factory assembly line and a three-dimensional model of a virtual robotic arm in a preset three-dimensional engine; and constructing a factory digital twin scene model according to the three-dimensional model of the virtual factory assembly line, the three-dimensional model of the virtual robotic arm, and the respective positions of the virtual factory assembly line and the virtual robotic arm in the virtual space.

[0083] The preset 3D engine refers to a software framework for creating and rendering 3D models, such as THreeJS (Javascript 3D library, a WebGL engine developed based on JavaScript), Unity (a game development tool), UE4 (Unreal Engine 4), etc. After the terminal loads the 3D model of the virtual factory assembly line and the 3D model of the virtual robotic arm in the preset 3D engine, it will combine the positions of the 3D models in the virtual space to obtain a combined 3D scene model, i.e., the factory digital twin scene model.

[0084] In some embodiments, a variety of end devices are loaded into a preset three-dimensional engine, and the corresponding end device can be determined according to the requirements of each process flow, and then the end device can be assembled into the corresponding virtual robotic arm.

[0085] In this embodiment, by loading the three-dimensional models of the virtual factory assembly line and the virtual robotic arm into a preset three-dimensional engine, and then constructing a factory digital twin scene model according to the positions of each three-dimensional model, it is beneficial to accurately simulate the factory assembly line.

[0086] To illustrate the virtual simulation method and effect of the factory assembly line in this solution in detail, the following is a most detailed embodiment:

[0087] For the scenario of setting up factory assembly lines, the virtual simulation method of the factory assembly line proposed in the embodiment of the present application is adopted, and digital twin technology, factory simulation technology and robotic arm technology are often used in combination to improve overall production efficiency and safety.

[0088] As shown in FIG3, a structure diagram of the virtual simulation system of a factory assembly line in one embodiment. The virtual simulation system of the factory assembly line includes a three-dimensional model loading part, a motion processing part, and a digital twin process display part. The three-dimensional model loading part includes the loading of the three-dimensional model of the virtual factory assembly line, the loading of the three-dimensional model of the virtual robotic arm, and the relative motion space points of the virtual robotic arm. The operation processing part includes the robotic arm IK algorithm and the target trigger event. The digital twin process display part includes a process flow driving the virtual robotic arm. As shown in FIG4, a digital twin process display effect diagram in one embodiment. The figure includes multiple virtual factory assembly lines, multiple virtual robotic arms, and a variety of virtual factory assembly line equipment.

[0089] Among them, in the 3D model loading part, the loaded 3D model of the virtual factory assembly line and the 3D model of the virtual robotic arm need to run on the preset 3D engine, such as THreeJS, Unity, UE4, etc. When the system is initialized, the existing 3D model of the virtual factory assembly line and the 3D model of the virtual robotic arm are loaded to build a factory digital twin scene model.

[0090] The 3D model of the virtual robot arm needs to have the corresponding joint information and the information of the end device. The end device can be a magnetic gripper, a laser cutting head, a welding integrated head, etc. After loading the virtual robot arm model, initialize the names of each component so that the corresponding components can be found in subsequent processing.

[0091] The relative motion space points of the virtual robot are a set of discrete coordinates that are precisely related to the process flow. The production of this set of coordinates needs to be set in combination with conditions such as collision, shortest distance, process flow, etc. For example, the process diagram of each process flow contains multiple position points.

[0092] The operation processing part can determine the motion path based on the above-mentioned discrete coordinate points. In some embodiments, the specific coordinate value of each time point can be obtained by path interpolation. For example, multiple position points are interpolated to obtain multiple interpolated position points, and curve fitting is performed on the multiple interpolated position points to obtain the motion path. Through the coordinate values ​​of each point in the motion path, the changes in the data of each joint in the virtual robot are calculated by the cyclic coordinate descent inverse kinematics in the robot IK algorithm to complete the corresponding joint changes. The accuracy of each change is freely set according to the CPU computing power, and the number of loop recursions is set to obtain more accurate joint motion data.

[0093] After receiving the target trigger event, the starting point coordinates and the end point coordinates indicated by the target trigger event are obtained, and the end device coordinates of the virtual robotic arm are used to determine the target motion path. Based on the target motion path, the virtual robotic arm is controlled to move in the factory digital twin scene model.

[0094] Refer to Figure 4, which shows the movement process of the virtual robot arm in the factory digital twin scene model, realizing the digitalization and visualization transformation of physical resources to three-dimensional virtual resources, as well as the three-dimensional visualization simulation of the production process. Through these technologies, the production process, resources and operations can be modeled and simulated without the need for physical prototypes or pilot plants, from equipment layout to work units to production lines, realizing real changes in the production process.

[0095] The above-mentioned virtual simulation method of the factory assembly line pre-builds a factory digital twin scene model based on the three-dimensional model of the virtual factory assembly line and the three-dimensional model of the virtual robotic arm, obtains multiple process flows and the process diagram of each process flow, determines the current process flow according to the execution order of multiple process flows, and controls the virtual robotic arm to move in the factory digital twin scene model based on multiple position points contained in the process diagram of the current process flow until the multiple position points corresponding to the current process flow are executed and then the next process flow is executed, thereby simulating each process flow in the factory digital twin scene model in turn. This method of accurately simulating multiple process flows in the factory assembly line can guide the layout and adjustment of the actual factory assembly line, which is conducive to improving the layout flexibility and safety of the factory assembly line. At the same time, through the deep integration of virtual and reality, production efficiency is greatly improved, costs are reduced, and the flexibility and safety of the system are enhanced.

[0096] It should be understood that, although the various steps in the flowcharts involved in the above-mentioned embodiments are displayed in sequence according to the indication of the arrows, these steps are not necessarily executed in sequence according to the order indicated by the arrows. Unless there is a clear explanation in this article, the execution of these steps does not have a strict order restriction, and these steps can be executed in other orders. Moreover, at least a part of the steps in the flowcharts involved in the above-mentioned embodiments can include multiple steps or multiple stages, and these steps or stages are not necessarily executed at the same time, but can be executed at different times, and the execution order of these steps or stages is not necessarily to be carried out in sequence, but can be executed in turn or alternately with other steps or at least a part of the steps or stages in other steps.

[0097] Based on the same inventive concept, the embodiment of the present application also provides a virtual simulation device for a factory assembly line for implementing the virtual simulation method for a factory assembly line involved above. The implementation scheme for solving the problem provided by the device is similar to the implementation scheme recorded in the above method, so the specific limitations of the virtual simulation device embodiments of one or more factory assembly lines provided below can refer to the limitations of the virtual simulation method for a factory assembly line above, and will not be repeated here.

[0098] In an exemplary embodiment, as shown in FIG5 , a virtual simulation device 500 for a factory assembly line is provided, comprising: an acquisition module 520, a control module 540 and a circulation module 560, wherein:

[0099] The acquisition module 520 is used to acquire multiple process flows and a process diagram of each process flow; the process diagram includes multiple location points;

[0100] The control module 540 is used to determine the current process flow according to the execution order of the multiple process flows, and control the virtual robotic arm to move in the factory digital twin scene model based on the multiple position points corresponding to the current process flow until the multiple position points corresponding to the current process flow are executed; the factory digital twin scene model is constructed based on the three-dimensional model of the virtual factory assembly line and the three-dimensional model of the virtual robotic arm;

[0101] The loop module 560 is used to determine the next process flow of the current process flow according to the execution order of the multiple process flows, and continue to execute the next process flow until the multiple process flows are completed.

[0102] The virtual simulation device of the above-mentioned factory assembly line pre-constructs a factory digital twin scene model based on the three-dimensional model of the virtual factory assembly line and the three-dimensional model of the virtual robotic arm, obtains multiple process flows and the process diagram of each process flow, determines the current process flow according to the execution order of multiple process flows, and controls the virtual robotic arm to move in the factory digital twin scene model based on the multiple position points contained in the process diagram of the current process flow until the multiple position points corresponding to the current process flow are executed and then the next process flow is executed, thereby simulating each process flow in the factory digital twin scene model in turn. This method of accurately simulating multiple process flows in the factory assembly line can guide the layout and adjustment of the actual factory assembly line, which is conducive to improving the layout flexibility and safety of the factory assembly line.

[0103] In one embodiment, based on multiple position points corresponding to the current process flow, the virtual robotic arm is controlled to move in the factory digital twin scene model. The control module 540 is also used to: determine the motion path based on the multiple position points corresponding to the current process flow; based on the motion path, control the virtual robotic arm to move in the factory digital twin scene model.

[0104] In one embodiment, the motion path is determined based on multiple position points corresponding to the current process flow, and the control module 540 is also used to: interpolate the multiple position points corresponding to the current process flow to obtain multiple interpolated position points; and perform curve fitting on the multiple interpolated position points to obtain the motion path.

[0105] In one embodiment, based on the motion path, the virtual robotic arm is controlled to move in the factory digital twin scene model, and the control module 540 is also used to: detect obstacles around the end device of the virtual robotic arm during the movement of the virtual robotic arm; when the distance between the current position of the end device and the obstacle is less than the first preset distance, control the virtual robotic arm to move a second preset distance away from the obstacle; obtain the updated position of the end device, and control the virtual robotic arm to move in the factory digital twin scene model according to the updated position and the remaining path in the motion path.

[0106] In one embodiment, the virtual simulation device 500 of the factory assembly line also includes a trigger module, which is used to: after receiving a target trigger event, obtain the starting point coordinates and the end point coordinates indicated by the target trigger event; determine the target motion path according to the starting point coordinates and the end point coordinates; based on the target motion path, control the virtual robotic arm to move in the factory digital twin scene model.

[0107] In one embodiment, in terms of constructing a factory digital twin scene model, the control module 540 is also used to: load the three-dimensional model of the virtual factory assembly line and the three-dimensional model of the virtual robotic arm in a preset three-dimensional engine; and construct a factory digital twin scene model based on the three-dimensional model of the virtual factory assembly line, the three-dimensional model of the virtual robotic arm, and the respective positions of the virtual factory assembly line and the virtual robotic arm in the virtual space.

[0108] Each module in the virtual simulation device of the factory assembly line can be implemented in whole or in part by software, hardware and their combination. Each module can be embedded in or independent of the processor in the computer device in the form of hardware, or can be stored in the memory of the computer device in the form of software, so that the processor can call and execute the operations corresponding to each module.

[0109] In an exemplary embodiment, a computer device is provided, which may be a terminal, and its internal structure diagram may be shown in FIG6. The computer device includes a processor, a memory, an input / output interface, a communication interface, a display unit, and an input device. The processor, the memory, and the input / output interface are connected via a system bus, and the communication interface, the display unit, and the input device are connected to the system bus via the input / output interface. The processor of the computer device is used to provide computing and control capabilities. The memory of the computer device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system and a computer program. The internal memory provides an environment for the operation of the operating system and the computer program in the non-volatile storage medium. The input / output interface of the computer device is used to exchange information between the processor and an external device. The communication interface of the computer device is used to communicate with an external terminal in a wired or wireless manner, and the wireless manner may be implemented through WIFI, a mobile cellular network, near field communication (NFC) or other technologies. When the computer program is executed by the processor, a virtual simulation method of a factory assembly line is implemented. The display unit of the computer device is used to form a visually visible picture, which may be a display screen, a projection device, or a virtual reality imaging device. The display screen can be a liquid crystal display screen or an electronic ink display screen, and the input device of the computer device can be a touch layer covering the display screen, or a button, trackball or touchpad set on the computer device shell, or an external keyboard, touchpad or mouse.

[0110] Those skilled in the art will understand that the structure shown in FIG. 6 is merely a block diagram of a partial structure related to the solution of the present application, and does not constitute a limitation on the computer device to which the solution of the present application is applied. The specific computer device may include more or fewer components than shown in the figure, or combine certain components, or have a different arrangement of components.

[0111] In an exemplary embodiment, a computer device is provided, including a memory and a processor, wherein a computer program is stored in the memory, and when the processor executes the computer program, the following steps are implemented:

[0112] Acquire multiple process flows and a process diagram for each process flow; the process diagram contains multiple position points; determine the current process flow according to the execution order of the multiple process flows, and control the virtual robotic arm to move in the factory digital twin scene model based on the multiple position points corresponding to the current process flow until the multiple position points corresponding to the current process flow are executed; the factory digital twin scene model is constructed based on the three-dimensional model of the virtual factory assembly line and the three-dimensional model of the virtual robotic arm; determine the next process flow of the current process flow according to the execution order of the multiple process flows, and continue to execute the next process flow until the multiple process flows are executed.

[0113] In one embodiment, when the processor executes the computer program, the processor further implements the following steps:

[0114] Based on multiple position points corresponding to the current process flow, the motion path is determined; based on the motion path, the virtual robotic arm is controlled to move in the factory digital twin scene model.

[0115] In one embodiment, when the processor executes the computer program, the processor further implements the following steps:

[0116] Interpolation processing is performed on multiple position points corresponding to the current process flow to obtain multiple interpolated position points; curve fitting is performed on the multiple interpolated position points to obtain a motion path.

[0117] In one embodiment, when the processor executes the computer program, the processor further implements the following steps:

[0118] During the movement of the virtual robotic arm, obstacles around the end device of the virtual robotic arm are detected; when the distance between the current position of the end device and the obstacle is less than the first preset distance, the virtual robotic arm is controlled to move a second preset distance away from the obstacle; the updated position of the end device is obtained, and according to the updated position and the remaining path in the motion path, the virtual robotic arm is controlled to move in the factory digital twin scene model.

[0119] In one embodiment, when the processor executes the computer program, the processor further implements the following steps:

[0120] After receiving the target trigger event, the starting point coordinates and the end point coordinates indicated by the target trigger event are obtained; the target motion path is determined according to the starting point coordinates and the end point coordinates; based on the target motion path, the virtual robotic arm is controlled to move in the factory digital twin scene model.

[0121] In one embodiment, when the processor executes the computer program, the processor further implements the following steps:

[0122] The three-dimensional model of the virtual factory assembly line and the three-dimensional model of the virtual robotic arm are loaded into the preset three-dimensional engine; a factory digital twin scene model is constructed according to the three-dimensional model of the virtual factory assembly line, the three-dimensional model of the virtual robotic arm, and the respective positions of the virtual factory assembly line and the virtual robotic arm in the virtual space.

[0123] In one embodiment, a computer readable storage medium is provided, on which a computer program is stored, and when the computer program is executed by a processor, the following steps are implemented:

[0124] Acquire multiple process flows and a process diagram for each process flow; the process diagram contains multiple position points; determine the current process flow according to the execution order of the multiple process flows, and control the virtual robotic arm to move in the factory digital twin scene model based on the multiple position points corresponding to the current process flow until the multiple position points corresponding to the current process flow are executed; the factory digital twin scene model is constructed based on the three-dimensional model of the virtual factory assembly line and the three-dimensional model of the virtual robotic arm; determine the next process flow of the current process flow according to the execution order of the multiple process flows, and continue to execute the next process flow until the multiple process flows are executed.

[0125] In one embodiment, when the computer program is executed by a processor, the following steps are also implemented:

[0126] Based on multiple position points corresponding to the current process flow, the motion path is determined; based on the motion path, the virtual robotic arm is controlled to move in the factory digital twin scene model.

[0127] In one embodiment, when the computer program is executed by a processor, the following steps are also implemented:

[0128] Interpolation processing is performed on multiple position points corresponding to the current process flow to obtain multiple interpolated position points; curve fitting is performed on the multiple interpolated position points to obtain a motion path.

[0129] In one embodiment, when the computer program is executed by a processor, the following steps are also implemented:

[0130] During the movement of the virtual robotic arm, obstacles around the end device of the virtual robotic arm are detected; when the distance between the current position of the end device and the obstacle is less than the first preset distance, the virtual robotic arm is controlled to move a second preset distance away from the obstacle; the updated position of the end device is obtained, and according to the updated position and the remaining path in the motion path, the virtual robotic arm is controlled to move in the factory digital twin scene model.

[0131] In one embodiment, when the computer program is executed by a processor, the following steps are also implemented:

[0132] After receiving the target trigger event, the starting point coordinates and the end point coordinates indicated by the target trigger event are obtained; the target motion path is determined according to the starting point coordinates and the end point coordinates; based on the target motion path, the virtual robotic arm is controlled to move in the factory digital twin scene model.

[0133] In one embodiment, when the computer program is executed by a processor, the following steps are also implemented:

[0134] The three-dimensional model of the virtual factory assembly line and the three-dimensional model of the virtual robotic arm are loaded into the preset three-dimensional engine; according to the three-dimensional model of the virtual factory assembly line, the three-dimensional model of the virtual robotic arm, and the respective positions of the virtual factory assembly line and the virtual robotic arm in the virtual space, a factory digital twin scene model is constructed.

[0135] In one embodiment, a computer program product is provided, comprising a computer program, which, when executed by a processor, implements the following steps:

[0136] Acquire multiple process flows and a process diagram for each process flow; the process diagram contains multiple position points; determine the current process flow according to the execution order of the multiple process flows, and control the virtual robotic arm to move in the factory digital twin scene model based on the multiple position points corresponding to the current process flow until the multiple position points corresponding to the current process flow are executed; the factory digital twin scene model is constructed based on the three-dimensional model of the virtual factory assembly line and the three-dimensional model of the virtual robotic arm; determine the next process flow of the current process flow according to the execution order of the multiple process flows, and continue to execute the next process flow until the multiple process flows are executed.

[0137] In one embodiment, when the computer program is executed by a processor, the following steps are also implemented:

[0138] Based on multiple position points corresponding to the current process flow, the motion path is determined; based on the motion path, the virtual robotic arm is controlled to move in the factory digital twin scene model.

[0139] In one embodiment, when the computer program is executed by a processor, the following steps are also implemented:

[0140] Interpolation processing is performed on multiple position points corresponding to the current process flow to obtain multiple interpolated position points; curve fitting is performed on the multiple interpolated position points to obtain a motion path.

[0141] In one embodiment, when the computer program is executed by a processor, the following steps are also implemented:

[0142] During the movement of the virtual robotic arm, obstacles around the end device of the virtual robotic arm are detected; when the distance between the current position of the end device and the obstacle is less than the first preset distance, the virtual robotic arm is controlled to move a second preset distance away from the obstacle; the updated position of the end device is obtained, and according to the updated position and the remaining path in the motion path, the virtual robotic arm is controlled to move in the factory digital twin scene model.

[0143] In one embodiment, when the computer program is executed by a processor, the following steps are also implemented:

[0144] After receiving the target trigger event, the starting point coordinates and the end point coordinates indicated by the target trigger event are obtained; the target motion path is determined according to the starting point coordinates and the end point coordinates; based on the target motion path, the virtual robotic arm is controlled to move in the factory digital twin scene model.

[0145] In one embodiment, when the computer program is executed by a processor, the following steps are also implemented:

[0146] The three-dimensional model of the virtual factory assembly line and the three-dimensional model of the virtual robotic arm are loaded into the preset three-dimensional engine; a factory digital twin scene model is constructed according to the three-dimensional model of the virtual factory assembly line, the three-dimensional model of the virtual robotic arm, and the respective positions of the virtual factory assembly line and the virtual robotic arm in the virtual space.

[0147] It should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, stored data, displayed data, etc.) involved in this application are all information and data authorized by the user or fully authorized by all parties, and the collection, use and processing of relevant data must comply with relevant regulations.

[0148] A person of ordinary skill in the art can understand that all or part of the processes in the above-mentioned embodiment method can be completed by instructing the relevant hardware through a computer program, and the computer program can be stored in a non-volatile computer-readable storage medium. When the computer program is executed, it can include the processes of the embodiments of the above-mentioned methods. Among them, any reference to the memory, database or other medium used in the embodiments provided in the present application can include at least one of non-volatile memory and volatile memory. Non-volatile memory can include read-only memory (ROM), magnetic tape, floppy disk, flash memory, optical memory, high-density embedded non-volatile memory, resistive random access memory (ReRAM), magnetic random access memory (MRAM), ferroelectric random access memory (FRAM), phase change memory (PCM), graphene memory, etc. Volatile memory can include random access memory (RAM) or external cache memory, etc. As an illustration and not limitation, RAM can be in various forms, such as static random access memory (SRAM) or dynamic random access memory (DRAM). The database involved in each embodiment provided in this application may include at least one of a relational database and a non-relational database. Non-relational databases may include distributed databases based on blockchains, etc., but are not limited to this. The processor involved in each embodiment provided in this application may be a general-purpose processor, a central processing unit, a graphics processor, a digital signal processor, a programmable logic device, a data processing logic device based on quantum computing, an artificial intelligence (AI) processor, etc., but are not limited to this.

[0149] The technical features of the above embodiments may be combined arbitrarily. To make the description concise, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this application.

[0150] The above-described embodiments only express several implementation methods of the present application, and the descriptions thereof are relatively specific and detailed, but they cannot be understood as limiting the scope of the present application. It should be pointed out that, for a person of ordinary skill in the art, several variations and improvements can be made without departing from the concept of the present application, and these all belong to the protection scope of the present application. Therefore, the protection scope of the present application shall be subject to the attached claims.

Claims

1. A virtual simulation method for a factory assembly line, characterized in that: The method comprises: Acquire multiple process flows and a process diagram of each process flow; the process diagram includes multiple location points; Determine the current process flow according to the execution order of multiple process flows, and control the virtual robotic arm to move in the factory digital twin scene model based on multiple position points corresponding to the current process flow until the multiple position points corresponding to the current process flow are completed; the factory digital twin scene model is constructed based on the three-dimensional model of the virtual factory assembly line and the three-dimensional model of the virtual robotic arm; The next process flow of the current process flow is determined according to the execution order of the multiple process flows, and the next process flow is continuously executed until the multiple process flows are completed.

2. The method according to claim 1, characterized in that The controlling the virtual robot arm to move in the factory digital twin scene model based on the multiple position points corresponding to the current process flow includes: Determining a motion path based on a plurality of position points corresponding to the current process flow; Based on the motion path, the virtual robotic arm is controlled to move in the factory digital twin scene model.

3. The method according to claim 2, characterized in that The determining of the motion path based on the multiple position points corresponding to the current process flow includes: Performing interpolation processing on a plurality of position points corresponding to the current process flow to obtain a plurality of interpolated position points; Curve fitting is performed on multiple interpolated position points to obtain a motion path.

4. The method according to claim 2, characterized in that: The controlling the virtual robot arm to move in the factory digital twin scene model based on the motion path includes: During the movement of the virtual robotic arm, detecting obstacles around the end device of the virtual robotic arm; When the distance between the current position of the end device and the obstacle is less than the first preset distance, controlling the virtual robotic arm to move a second preset distance in a direction away from the obstacle; The updated position of the end tool is obtained, and the virtual robot arm is controlled to move in the factory digital twin scene model according to the updated position and the remaining path in the motion path.

5. The method according to claim 1, characterized in that The method further comprises: After receiving the target trigger event, obtaining the starting point coordinates and the ending point coordinates indicated by the target trigger event; Determine a target motion path according to the starting point coordinates and the end point coordinates; Based on the target motion path, the virtual robotic arm is controlled to move in the factory digital twin scene model.

6. The method according to claim 1, characterized in that The steps of constructing the factory digital twin scenario model include: Loading the 3D model of the virtual factory assembly line and the 3D model of the virtual robotic arm into the preset 3D engine; A factory digital twin scene model is constructed based on the three-dimensional model of the virtual factory assembly line, the three-dimensional model of the virtual robotic arm, and the respective positions of the virtual factory assembly line and the virtual robotic arm in the virtual space.

7. A virtual simulation device for a factory assembly line, characterized in that: The device comprises: An acquisition module, used to acquire multiple process flows and a process diagram of each process flow; the process diagram includes multiple location points; A control module, used to determine the current process flow according to the execution order of multiple process flows, and control the virtual robotic arm to move in the factory digital twin scene model based on multiple position points corresponding to the current process flow until the multiple position points corresponding to the current process flow are executed; the factory digital twin scene model is constructed based on the three-dimensional model of the virtual factory assembly line and the three-dimensional model of the virtual robotic arm; The loop module is used to determine the next process flow of the current process flow according to the execution order of multiple process flows, and continue to execute the next process flow until the multiple process flows are completed.

8. A computer device comprising a memory and a processor, wherein the memory stores a computer program, wherein: When the processor executes the computer program, the steps of the method according to any one of claims 1 to 6 are implemented.

9. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 6 are implemented.

10. A computer program product, comprising a computer program, characterized in that When the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 6 are implemented.

Images