Unmanned aerial vehicle electrical equipment real-time action simulation system and method based on joint disassembly

Abstract

The invention discloses a real-time motion simulation system and method for unmanned aerial vehicle electrical equipment based on joint disassembly. The unmanned aerial vehicle electrical equipment comprises a plurality of motion joints. The simulation system comprises a display device, a sensor, a controller and a high-speed calculation module, wherein the sensor, the controller and the high-speed calculation module are arranged on the unmanned aerial vehicle electrical equipment; the display device is in wireless communication with the unmanned aerial vehicle electrical equipment; an equipment data communication transmission protocol is arranged between the controller and the high-speed calculation module; the high-speed calculation module can generate driving data based on the equipment data; a driving data communication transmission protocol is arranged between the high-speed calculation module and the display equipment; and the display equipment can perform real-time action simulation of the unmanned aerial vehicle electrical equipment on a built-in three-dimensional model based on the driving data. According to the invention, real-time action simulation of the electromechanical equipment driven by low-bandwidth data can be realized; an operator observes the real-time action of the equipment on the display screen, so that the state of the equipment is known, and remote monitoring is carried out.

Description

Technical Field

[0001] The present invention belongs to the technical field of digital twins of electromechanical equipment, and particularly relates to a real-time motion simulation system and method for unmanned electromechanical equipment based on joint disassembly. Background Art

[0002] Electromechanical equipment is widely used in various industries. With the development of informatization and intelligent technologies, electromechanical equipment is gradually developing towards unmanned operation, such as devices like unmanned aerial vehicles and unmanned ships. In a manned working environment, personnel can directly observe the actions of electromechanical equipment, understand its status, and then operate the equipment to perform actions. However, in an unmanned usage scenario, operators usually observe the data uploaded by the equipment through forms such as display screens at a remote end. Users cannot directly observe the status of the equipment; this will result in a lack of remote supervision by personnel during the automated operation process of the equipment, and operators cannot discover and handle abnormalities in a timely manner when the equipment malfunctions. In addition, when users remotely control the operation of the equipment at a remote end, it is difficult to correctly remotely control the equipment to perform actions if they cannot intuitively and effectively understand the status of the equipment.

[0003] To address the above problems, usually, a camera is installed to collect video images in real time, and the video images of the site are sent to the remote end for monitoring through a display screen at the remote end side. This solution is applicable to usage scenarios with wired data transmission such as optical fibers or high-frequency wireless data transmission such as 5G. However, in scenarios such as the ocean and high altitude, due to environmental limitations and a relatively long remote control operation range, there is a lack of efficient data transmission means, and data transmission is usually carried out through satellite communication. At this time, the transmission of video images will occupy a large amount of transmission resources and cannot be effectively transmitted. Summary of the Invention

[0004] The purpose of the present invention is to provide a real-time motion simulation system and method for unmanned electromechanical equipment based on joint disassembly to solve the problems raised in the background art.

[0005] To achieve the above purpose, the present invention provides the following technical solutions:

[0006] A real-time motion simulation system for unmanned electromechanical equipment based on joint disassembly, where the unmanned electromechanical equipment includes a plurality of motion joints, and each motion joint includes an active component controlled by a controller and at least one driven component driven by the active component;

[0007] The system includes a display device, and sensors, a controller, and a high-speed computing module provided on the unmanned electromechanical equipment. The display device communicates wirelessly with the unmanned electromechanical equipment;

[0008] A device data communication transmission protocol with the motion joint as the core is set between the controller and the high-speed computing module, and the device data includes sensor data and control instructions of each motion joint;

[0009] The high-speed computing module can generate drive data based on device data, and the drive data includes the motion data of each component in each motion joint.

[0010] A drive data communication transmission protocol with the motion joint as the core is set between the high-speed computing module and the display device; the display device can perform real-time motion simulation of the unmanned electromechanical device on the built-in 3D model based on the drive data.

[0011] Preferably, the high-speed computing module includes:

[0012] A model construction unit for constructing a motion relationship model of the driven component and the driving component in each motion joint under the same motion parameters.

[0013] A power calculation unit for calculating drive data based on the motion relationship model and device data.

[0014] Preferably, the sensor data includes motion amount data and / or motion speed data.

[0015] In the model construction unit, a motion amount relationship model and a motion speed relationship model of the driven component and the driving component in each motion joint are constructed by means of data fitting, modeling and simulation, or theoretical derivation.

[0016] Preferably, in the high-speed computing module, a power analysis unit is further provided. When the sensor only measures the motion speed data or the motion amount data of the component, the corresponding motion amount or motion speed is calculated through the motion amount analysis model or the motion speed analysis model, and is respectively expressed as:

[0017]

[0018] In the formula, X(t m ) is the displacement or rotation amount at time t m , V(t i-1 ) is the speed or rotational speed measured by the sensor at time t i-1 , V(t n ) is the speed or rotational speed at time t n , X(t n ), X(t n-1 ) respectively represent the displacement or rotation amount at time t n , t n-1 .

[0019] Preferably, in the display device, the drive data at different times is expanded by the method of expanding the data frequency through time linear interpolation.

[0020] Preferably, the method of expanding the data frequency through time linear interpolation is: at two times (t i,t i+1 ) to obtain the new driving data f(t new ), the calculation formula is:

[0021]

[0022] In the formula, f(t i ) indicates t i Driving data at the moment.

[0023] Preferably, during data driving, the overall driving data moment is delayed by a starting time interval.

[0024] Preferably, the motion data of each of the components are the motion parameters of each component in the active motion direction relative to its own coordinate system; in the motion simulation, the parent-child object relationship between the components is determined by analyzing the hinge and translation connection relationships between all the components of the action joints, and the motion of all the action joint components is analyzed to the same global coordinate system through the coordinate system transformation method, so as to drive the entire electromechanical equipment to perform real-time motion simulation.

[0025] Preferably, the coordinate system conversion between multiple components is achieved by multiplying the coordinate system conversion matrices between different parent and child object components, which is expressed as:

[0026]

[0027] Among them, M is the coordinate transformation matrix, and the j-1th component is the parent object of the jth construction.

[0028] A method for real-time motion simulation of unmanned aerial vehicle electrical equipment with disassembled joints, using the motion simulation system, the motion simulation method comprising:

[0029] The display device performs real-time motion simulation based on the drive data sent by the drone's electrical and mechanical equipment and the built-in 3D model;

[0030] The driving data is calculated by the high-speed calculation module using the device data sent by the controller.

[0031] Compared with the prior art, the present invention has the following beneficial effects:

[0032] The present invention uses non-image data that occupies less transmission resources, such as sensor data and device control instructions, combined with a pre-built physical model of the electromechanical equipment, to calculate the motion of each component of the electromechanical equipment, thereby realizing real-time motion simulation of the electromechanical equipment driven by low-bandwidth data. The operator can understand the equipment status and perform remote monitoring by observing the real-time motion of the equipment on the display screen. BRIEF DESCRIPTION OF THE DRAWINGS

[0033] Figure 1It is a schematic structural diagram of the real-time action simulation system for the unmanned electromechanical equipment of the present invention.

[0034] Figure 2 It is a key component diagram of the equipment data communication transmission protocol.

[0035] Figure 3 It is a key component diagram of the drive data communication transmission protocol.

[0036] Figure 4 It is a schematic diagram of the linear interpolation time axis.

[0037] Figure 5 It is a schematic diagram of the Unity3d parent-child objects and six degrees of freedom. Specific implementation manner

[0038] Next, the technical solutions in the embodiments of the present invention will be clearly and completely described in conjunction with the accompanying drawings. Obviously, the described embodiments are only a part of the embodiments of the present invention, rather than all the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those of ordinary skill in the art without making creative efforts belong to the protection scope of the present invention.

[0039] Refer to Figure 1 As shown, the present invention provides a real-time action simulation system for unmanned electromechanical equipment based on joint disassembly. The unmanned electromechanical equipment includes a number of action joints. Each action joint includes active components such as an oil cylinder, a motor, a hydraulic motor or an electric push rod controlled by a controller to act, and a driven component driven by the active component; the action simulation system includes a display device, a sensor, a controller, and a high-speed calculation module. The sensor, the controller, and the high-speed calculation module are all installed on the local side of the electromechanical equipment, and the display device is installed at the remote operator. And a wireless communication method such as satellite communication is used between the display device and the electromechanical equipment.

[0040] An equipment data communication transmission protocol with the action joint as the core is set between the controller and the high-speed calculation module, and the controller sends equipment data to the high-speed calculation module. Specifically, refer to Figure 2 As shown, the data transmission content under the equipment data communication transmission protocol includes the number of joints and the data of each action joint. The data of each action joint includes joint sensor data and joint control instructions. The joint sensor data includes at least one of the process sensing value and the speed / rotation speed sensing value of the joint, the positive direction in-place sensing value of the joint, and the negative direction in-place sensing value of the joint; the process sensing value is also the amount of movement, which is used to indicate the process displacement / rotation amount information of the current action joint. The in-place sensing values in the positive and negative directions are used to indicate whether the current action joint reaches the extreme limit position in the positive and negative directions; the joint control instruction is the control output opening amount of the joint active component, which indicates the speed of the current action of the action joint.

[0041] In the present invention, the moving joints of the unmanned electromechanical device can be obtained by disassembling the physical model of the unmanned electromechanical device.

[0042] In the present invention, in order to adapt to the design of joint disassembly, the moving joints are used as the core to transmit information in the communication protocol, and the sensor data and control data related to each moving joint are integrated and merged into joint device data, which is convenient for subsequent data analysis and data-driven model actions.

[0043] The high-speed calculation module includes:

[0044] A model construction unit, which is used to construct a motion relationship model of the driven component and the driving component in each moving joint under the same motion parameters through methods such as data fitting, modeling and simulation, or theoretical derivation, that is, the motion quantity relationship model between the driven component and the driving component, and the motion speed relationship model between the driven component and the driving component;

[0045] A power calculation unit, which calculates the driving data based on the motion relationship model and the device data, and the driving data includes the motion speed data and motion quantity data of each component.

[0046] Furthermore, in the high-speed calculation module of the present invention, a power analysis unit is also provided. When the displacement / rotation amount of the joint is not measured in the joint sensing data, the measured speed / rotation speed sensing value and the motion quantity analysis model are used to calculate the corresponding displacement / rotation amount. The motion quantity analysis model combines the speed / rotation speed multiplied by the sampling point time interval to accumulate to obtain the displacement / rotation amount, as shown in the following formula:

[0047]

[0048] In the formula, X(t m ) is the displacement or rotation amount at time t m , and V(t i-1 ) is the speed or rotation speed measured by the sensor at time t i-1 ;

[0049] When the speed / rotation speed sensing value of the joint is not measured in the joint sensing data, the measured displacement / rotation amount value and the motion speed analysis model are used to calculate the corresponding speed / rotation speed. The motion speed analysis model uses the subtraction of the displacement / rotation amount at adjacent times divided by the time interval to obtain the speed / rotation speed, as shown in the following formula:

[0050]

[0051] In the formula, V(t n ) is the speed or rotation speed at time t n , X(t n ), X(t n-1 ) respectively represent the displacement / rotation amount at time tn , t n-1 The displacement or rotation amount at a moment;

[0052] In the present invention, through experiments and other means, one of the motion parameter values of the active component in the motion joint is measured by a sensor, and the other motion parameter value is derived by differential and integral methods.

[0053] As one specific embodiment of the present invention, the relationship between the same motion parameters of the driven component and the active component is obtained by data fitting methods such as polynomial regression and least squares method as shown in the following formula:

[0054]

[0055] In the formula, X represents the displacement or rotation amount, V represents the speed or rotational speed, and n represents the index of the driven component in the motion joint.

[0056] Of course, through multi-body dynamics simulation software such as Adams, the dynamic models of each motion joint can be constructed, and the relationships between the displacement / rotation amount and speed / rotational speed of each driven component and the active component in the motion joint can be obtained through simulation calculation.

[0057] In this power calculation unit, specifically, the measured displacement / rotation amount and speed / rotational speed of the motion joint with respect to the sensor monitoring component are used as known quantities, and the displacement / rotation amount or speed / rotational speed of the associated component is deduced through the motion quantity relationship model and motion speed relationship model between the active component and the driven component. For example, if the displacement / rotation amount and speed / rotational speed of the driven component are known, the displacement / rotation amount and speed / rotational speed of the associated active component are obtained by solving its inverse function, as shown in the following formula:

[0058]

[0059] In the formula, are respectively the inverse functions of f n , g n .

[0060] In the present invention, since the communication distance between the remote side and the device local side is relatively long, the overall communication is not stable, and problems such as communication interruption will occur; when data is frequently lost, the calculation accuracy of calculating speed based on displacement or calculating displacement based on speed will decrease; therefore, a high-speed calculation module is added on the local side to receive device data in real time, calculate and analyze the positions and speeds of each component as driving data, so that the display device on the remote side only performs action visualization display and does not involve excessive model solving.

[0061] A drive data communication transmission protocol with the motion joint as the core is set between the display device and the unmanned electric device, and the unmanned electric device can obtain the drive data sent by the display device. Specifically, refer toFigure 3 As shown, the transmission content under the drive data communication transmission protocol includes the number of moving joints, the number of components of each moving joint, and the motion data of any component. The motion data includes the amount of motion and the motion speed. The amount of motion is the displacement or rotation amount, and the motion speed is the speed corresponding to the displacement amount or the rotation speed corresponding to the rotation amount.

[0062] The display device has a built-in 3D model and can drive the 3D model to perform motion simulation based on the received motion data of each component of the device.

[0063] It should be noted here that the motion data of each component of the device are all the motion parameters of each component in the active motion direction relative to its own coordinate system.

[0064] In the motion simulation of the display device based on the drive data, by analyzing the hinge and translational connection relationships between the components of all moving joints, the parent-child object relationship between the components is determined (the motion of the parent object will drive the child object and the coordinate system of the child object to move together). By means of coordinate system transformation, the motions of all moving joint components are resolved to the same global coordinate system, so as to realize the serial connection of the motions of all components and drive the overall electromechanical device to perform real-time motion simulation.

[0065] Here, if the first component is the parent object of the second component, the second component is the parent object of the third component, and so on, the (j - 1)th component is the parent object of the jth component. By multiplying the coordinate system transformation matrices between different parent-child object components, the coordinate system transformation between multiple components is realized, which is expressed as:

[0066]

[0067] Among them, M is the coordinate transformation matrix, which is composed of translation and rotation, and its calculation formula is as follows:

[0068]

[0069] Among them, α, β, and γ are the rotation amounts of the child object coordinate system around the x, y, and z axes in the parent object coordinate system respectively, and x, y, and z are the displacement amounts of the child object coordinate system relative to the parent object coordinate system. Of course, the coordinate system transformation relationship can also be solved through a game engine. For example, in the actual use of Unity3d, only the parent and child objects need to be set, and then the parent object is driven to move, and the child object will follow the movement, which can simplify the program calculation. Refer to Figure 5 As shown, in the linear motor moving joint, the push rod is the parent object and the push plate is the child object. When the push rod moves, it will drive the push plate to move. In Unity3d, each component has its six-degree-of-freedom coordinates (3 moving coordinates x, y, z and 3 rotation coordinates α, β, γ), such as Figure 5As shown in the red box at the upper right corner. By changing the six - degree - of - freedom coordinates of the component, the position and attitude of the component can be changed, realizing the translational and rotational motions of the component.

[0070] The drive data will include the displacement / rotation amount of each joint component. In Unity3d, the specific degrees of freedom in the six - degree - of - freedom coordinates for changing the motion amount of each joint component are set in advance. When receiving the real - time drive data, adding the received motion amount on the preset degrees of freedom can drive the corresponding component to move.

[0071] In this display device, since action simulation requires a high frame rate, while remote data communication usually cannot maintain a high communication frequency, the data frequency is expanded through time - linear interpolation. Refer to Figure 5 As shown, if the drive data received by the display device at different times is represented by f(T), T = {t0, t1, t2,..., t I}, at this time, new drive data f(t i , t i+1 ) can be calculated between two times (t new ), thereby expanding the drive data. Here, the calculation formula for the new drive data is:

[0072]

[0073] In the formula, f(t i ) represents the drive data at time t i , and i = 0, 1, 2,…, I.

[0074] However, through the above - mentioned linear interpolation method, the data points at the previous and next times need to be known. Therefore, the overall drive data time is delayed by a starting time interval during data driving. When the drive data of t1 is received at time t1, linear interpolation is performed on the drive data between t0 and t1, and the model is driven by the actual drive data at time t0. As Figure 4 shown, the long vertical lines in the figure represent the drive data sent by the back - end server, and the short vertical lines represent the drive data calculated by the linear interpolation method. The data acquisition time is the real - world time, and the drive data usage time refers to the occurrence time of the drive data used.

[0075] The method of expanding the data frequency through linear interpolation in the present invention linearly interpolates the drive data received at the previous and next times according to time, thereby generating a large amount of drive data between the two times, improving the fluency of real - time action simulation.

[0076] In the present invention, through the method of joint disassembly, the actions of a complex multi-component device are disassembled into multiple action joints with the active component as the core. The component motions of each joint are respectively simulated, and finally, through methods such as coordinate system transformation, the component motions of multiple joints are integrated together. This solves the problem that it is difficult to simulate the action combinations of multiple active components in a single model for a complex electromechanical device composed of multiple active components to realize the simulation of the motion of the whole machine.

[0077] Moreover, in the device data communication transmission protocol and the drive data communication transmission protocol of the present invention, by setting the number of joints and the number of components in the action joints, the user can determine the data lengths of each part of the message. When the electromechanical device needs to add action joints, only the number information in the protocol needs to be modified, and new joint device data information is added to the message. Both communication parties can normally parse it, thus realizing the expandability of the communication protocol.

[0078] Based on the above real-time action simulation system, the present invention also provides a real-time action simulation method for an unmanned electromechanical device with joint disassembly. The specific action simulation method is as follows: The display device performs real-time action simulation based on the drive data sent by the unmanned electromechanical device and the three-dimensional model built in the display device. And this drive data is calculated by the high-speed calculation module using the device data sent by the controller.

[0079] The above-described embodiments are only preferred solutions of the present invention, and do not impose any form of limitation on the present invention. There are other variations and modifications without exceeding the technical solutions recorded in the claims.

Claims

1. A real-time motion simulation system for UAV electrical equipment based on joint disassembly, characterized in that: The unmanned aerial vehicle electromechanical device comprises a plurality of motion joints, each motion joint comprising an active component controlled by a controller and at least one driven component driven by the active component; The system includes a display device, and a sensor, a controller, and a high-speed computing module arranged on the unmanned aerial vehicle electrical and mechanical equipment. The display device and the unmanned aerial vehicle electrical and mechanical equipment communicate wirelessly. A device data communication transmission protocol with action joints as the core is arranged between the controller and the high-speed computing module. The device data includes sensor data and control instructions of each action joint. The high-speed computing module can generate drive data based on the device data. The drive data includes motion data of each component in each action joint. A drive data communication transmission protocol with action joints as the core is set between the high-speed computing module and the display device; the display device can perform real-time action simulation of the drone's electrical and mechanical equipment on the built-in three-dimensional model based on the drive data.

2. A real-time motion simulation system for electrical equipment of a drone with disassembled joints as claimed in claim 1, characterized in that: The high-speed computing module comprises: A model building unit, used for building a motion relationship model under the same motion parameters for the driven component and the active component in each action joint; The power calculation unit calculates the drive data based on the motion relationship model and the equipment data.

3. A real-time motion simulation system for electrical equipment of a drone with disassembled joints as claimed in claim 2, characterized in that: The sensor data includes movement amount data and / or movement speed data; In the model building unit, a motion quantity relationship model and a motion speed relationship model are built for the driven component and the active component in each action joint through data fitting, modeling simulation or theoretical deduction method.

4. A real-time motion simulation system for electrical equipment of a drone with disassembled joints as claimed in claim 3, characterized in that: The high-speed computing module is also provided with a power analysis unit. When the sensor only measures the motion speed data or motion amount data of the component, the corresponding motion amount or motion speed is calculated by the motion amount analysis model or the motion speed analysis model, which are respectively expressed as: In the formula, X(t m ) is t m The displacement or rotation at a certain moment, V(t i-1 ) is the t measured by the sensor i-1 The speed or rotation speed at the moment, V(t n ) is t n The speed or rotation speed at the moment, X(t n )、X(t n-1 ) represent t n ,t n-1 The amount of displacement or rotation at a given moment.

5. The real-time motion simulation system for the electrical equipment of a drone with disassembled joints as claimed in claim 1, characterized in that: In the display device, the driving data at different moments are expanded by using a time linear interpolation method to expand the data frequency.

6. A real-time motion simulation system for electrical equipment of a drone with disassembled joints as claimed in claim 5, characterized in that: The method of expanding the data frequency by time linear interpolation is as follows: at two moments (t i , t i+1 ) to obtain the new driving data f(t new ), the calculation formula is: In the formula, f(t i ) indicates t i Driving data at the moment.

7. A real-time motion simulation system for electrical equipment of a drone with disassembled joints as claimed in claim 6, characterized in that: When data is driven, the overall driving data is always delayed by a starting time interval.

8. The real-time motion simulation system for the electrical equipment of a drone with disassembled joints as claimed in claim 1, characterized in that: The motion data of each of the components are the motion parameters of each component in the active motion direction relative to its own coordinate system; in the motion simulation, the parent-child object relationship between the components is determined by analyzing the hinge and translation connection relationship between all the components of the action joints, and the motion of all the action joint components is analyzed to the same global coordinate system through the coordinate system transformation method, so as to drive the entire electromechanical equipment to perform real-time motion simulation.

9. A real-time motion simulation system for electrical equipment of a drone with disassembled joints as claimed in claim 8, characterized in that: By multiplying the coordinate system transformation matrices between different parent and child object components, the conversion of coordinate systems between multiple components is achieved, which is expressed as: Among them, M is the coordinate transformation matrix, and the j-1th component is the parent object of the jth construction.

10. A method for simulating the real-time motion of an unmanned aerial vehicle electrical device with disassembled joints, characterized in that: Applying the action simulation system according to any one of claims 1 to 9, the action simulation method comprises: The display device performs real-time motion simulation based on the drive data sent by the drone's electrical and mechanical equipment and the built-in 3D model; The driving data is calculated by the high-speed calculation module using the device data sent by the controller.

Images