Control system of mechanical equipment
By identifying gestures and generating control commands by virtual reality devices, the problem of wireless remote control is solved that the wireless remote control method is not intuitive and flexible in complex scenarios, and more flexible, convenient and accurate mechanical equipment control is achieved.
Patent Information
- Application Number
- CN202420315141.1
- Authority / Receiving Office
- CN · China
- Patent Type
- Utility models(China)
- Current Assignee / Owner
- Filing Date
- 2024-02-20
- Publication Date
- 2025-02-11
- Estimated Expiration
- 2034-02-20
AI Technical Summary
The existing wireless remote control method has problems such as intuition and inflexibility in some complex scenarios, making it difficult to achieve more accurate and flexible mechanical equipment control.
Virtual reality devices are used to obtain the object's hand position data in real time, identify gestures and generate corresponding control commands, and send commands to mechanical devices through wireless communication to achieve accurate control of mechanical devices.
Through gesture control, the mechanical equipment is free from the button restrictions of remote control equipment, and more flexible, convenient and accurate mechanical equipment control is achieved.
Smart Images
Figure CN222462011U_ABST
Abstract
Description
Technical Field
[0001] The present application relates to the field of virtual reality technology, and in particular to a control system for mechanical equipment. Background Art
[0002] Engineering machinery is widely used in the fields of construction, excavation, transportation, processing, and repair, such as excavators and cranes. In the control of such engineering machinery, in addition to manual operation in the cockpit, wireless remote control can also be used.
[0003] Although wireless remote control can control the mechanical end without being in the cockpit, this operation method is not intuitive and flexible. In some rescue, disaster relief, explosion protection, fine control and other scenarios, more flexible and precise mechanical control methods are needed to replace the operations that should be performed manually.
[0004] Therefore, it is necessary to redesign the control system of mechanical equipment to control the mechanical equipment more flexibly, intuitively and accurately. Utility Model Content
[0005] The embodiment of the present application provides a control system for mechanical equipment to control the mechanical equipment more flexibly and accurately.
[0006] The embodiment of the present application provides a control system for a mechanical device, including: a mechanical device, a data acquisition device disposed on the mechanical device, and a virtual reality device connected to the mechanical device for communication, wherein:
[0007] The data acquisition device acquires the status data of the mechanical device in real time and sends the status data to the mechanical device;
[0008] The mechanical device receives the status data reported by the data acquisition device, sends the status data to the virtual reality device, receives the control command sent by the virtual reality device, and performs corresponding actions based on the control command;
[0009] The virtual reality device updates the position and state of a pre-created mechanical device model based on the received status data sent by the mechanical device, obtains the hand position data of the object, recognizes the gesture of the object based on the hand position data, obtains a corresponding control command based on the gesture of the object, and sends the control command to the mechanical device.
[0010] Optionally, the virtual reality device includes: a wearable data acquisition device for acquiring hand position data of the object.
[0011] Optionally, the wearable data acquisition device includes a data glove.
[0012] Optionally, the mechanical device includes a chassis, and the virtual reality device includes: a positioning device, a control device, and a communication device;
[0013] The control device is used to control the positioning device to obtain the movement amplitude of the object's hand when controlling the chassis of the mechanical device to move, and determine the movement speed when controlling the movement of the mechanical device based on the movement amplitude;
[0014] The communication device is used to send the moving speed to the mechanical device to control the mechanical device to move at the moving speed.
[0015] Optionally, the mechanical equipment includes a boom;
[0016] The control device is further used to control the positioning device to obtain the position change and angle change of the object's hand when controlling the arm of the mechanical device to move, and determine the movement amount and rotation angle of the arm based on the position change and the angle change;
[0017] The communication device is also used to send the determined movement amount and rotation angle of the arm to the mechanical device to control the movement of the arm of the mechanical device.
[0018] Optionally, the virtual reality device includes: a wearable display device for displaying a model of the mechanical device and showing the position and status of the mechanical device.
[0019] Optionally, the wearable display device includes virtual reality glasses.
[0020] Optionally, the data acquisition device includes one or more of the following: a satellite positioning sensor, a direction sensor, an inclination sensor, and one or more sensors for collecting working status parameters of the mechanical equipment.
[0021] Optionally, the mechanical equipment includes a boom and a chassis, and the control command includes one or more of the following: a boom control command of the mechanical equipment, a chassis rotation control command of the mechanical equipment, and a walking control command of the mechanical equipment.
[0022] Optionally, the mechanical device includes: a connection detection device, used to determine whether the network connection with the virtual reality device is normal through heartbeat detection, and control the mechanical device to stop working when it is determined that the network connection with the virtual reality device is abnormal.
[0023] The beneficial effects of this application are as follows:
[0024] The control system of the mechanical equipment provided in the embodiment of the present application utilizes a virtual reality device to obtain the hand position data of the object in real time, determines the hand gesture of the object based on the hand position data, and then controls the mechanical equipment based on the gesture of the object. Compared with the existing wireless remote control method, the mechanical equipment is controlled by gestures, which gets rid of the button restrictions of the remote control device, and more gestures or gesture groups can be used to control the mechanical equipment, thereby making the control of the mechanical equipment more flexible, convenient and accurate.
[0025] Other features and advantages of the present application will be described in the following description, and partly become apparent from the description, or be understood by practicing the present application. The purpose and other advantages of the present application can be realized and obtained by the structures specifically pointed out in the written description, claims, and drawings. BRIEF DESCRIPTION OF THE DRAWINGS
[0026] The drawings described herein are used to provide a further understanding of the present application and constitute a part of the present application. The illustrative embodiments of the present application and their descriptions are used to explain the present application and do not constitute an improper limitation on the present application. In the drawings:
[0027] Figure 1 A schematic diagram of an application scenario in an embodiment of the present application;
[0028] Figure 2 This is a schematic diagram of the structure of a control system of a mechanical device in an embodiment of the present application. DETAILED DESCRIPTION
[0029] In order to make the purpose, technical solution and advantages of the embodiments of the present application clearer, the technical solution of the present application will be clearly and completely described below in conjunction with the drawings in the embodiments of the present application. Obviously, the described embodiments are part of the embodiments of the technical solution of the present application, rather than all of the embodiments. Based on the embodiments recorded in the application documents, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the technical solution of the present application.
[0030] The following is an introduction to some concepts involved in the embodiments of the present application.
[0031] Data gloves: a multi-mode virtual reality hardware that can obtain hand position data through sensors and can also be programmed with software to complete actions such as grabbing, moving, and rotating objects in the virtual scene.
[0032] Virtual Reality: Virtual Reality (VR) is a computer simulation system that can create and experience a virtual world. It uses computers to generate a simulated environment and immerse objects in it. Virtual reality technology uses real-life data and electronic signals generated by computer technology, and combines them with various output devices to transform them into phenomena that people can feel. These phenomena can be real objects or substances that are invisible to the naked eye, and are expressed through three-dimensional models. Because these phenomena are not what we can see directly, but the real world simulated by computer technology, they are called virtual reality.
[0033] Mechanical equipment, the mechanical equipment in the embodiments of the present application, generally refers to engineering machinery, and the mechanical equipment includes at least one component of a boom, a rotatable chassis or a travelable chassis.
[0034] The following is a brief introduction to the design concept of the embodiment of the present application:
[0035] Engineering machinery is widely used in the fields of construction, excavation, transportation, processing, and repair, such as excavators and cranes. In the control of such engineering machinery, in addition to manual operation in the cockpit, wireless remote control can also be used.
[0036] Although wireless remote control can control the mechanical end without being in the cockpit, this operation method is not intuitive and flexible. In some rescue, disaster relief, explosion protection, fine control and other scenarios, more flexible and precise mechanical control methods are needed to replace the operations that should be performed manually.
[0037] In view of this, an embodiment of the present application provides a control system for a mechanical device, which uses a virtual reality device to obtain the hand position data of an object in real time, determines the hand gesture of the object based on the hand position data, and then controls the mechanical device based on the gesture of the object. Compared with the existing wireless remote control method, the mechanical device is controlled by gestures, which gets rid of the button restrictions of the remote control device, and can use more gestures or gesture groups to control the mechanical device, thereby making the control of the mechanical device more flexible, convenient and accurate.
[0038] The preferred embodiments of the present application are described below in conjunction with the drawings in the specification. It should be understood that the preferred embodiments described herein are only used to illustrate and explain the present application, and are not used to limit the present application. In addition, the embodiments and features in the embodiments of the present application may be combined with each other if there is no conflict.
[0039] like Figure 1As shown, it is a schematic diagram of an application scenario of an embodiment of the present application. The application scenario diagram includes an object 10, a virtual reality device 11 and a mechanical device 12. Among them, the virtual reality device 11 is connected to the mechanical device 12 in communication. The mechanical device 12 can be various engineering mechanical equipment, including but not limited to: excavators, pump trucks, tower cranes, cranes, aerial work machinery, etc.
[0040] In the embodiment of the present application, the virtual reality device 11 can collect the hand position data of the object 10, and then recognize the gesture of the object 10, and then use the gesture to control the mechanical device 12. Among them, the virtual reality device 11 can also display the position and status of the mechanical device to the object 10 in real time through modeling based on the status data reported by the mechanical device 12.
[0041] The following describes the control system of the mechanical equipment provided by the exemplary embodiment of the present application in combination with the application scenarios described above and with reference to the accompanying drawings. It should be noted that the above application scenarios are only shown to facilitate understanding of the spirit and principles of the present application, and the implementation methods of the present application are not limited in this regard.
[0042] It should be noted that the control system of the mechanical equipment provided in the embodiment of the present application can be divided into a mechanical equipment receiving end and a remote operation end as a whole. The mechanical equipment receiving end includes the mechanical equipment and a data acquisition device arranged on the mechanical equipment, and the remote operation end includes a virtual reality device. Of course, both the mechanical equipment and the virtual reality device include a wireless communication module for communication between the mechanical equipment and the virtual reality device.
[0043] The present application embodiment provides a control system for a mechanical device, such as Figure 2 As shown, it includes: a mechanical device 20, a data acquisition device 21 arranged on the mechanical device, and a virtual reality device 22 communicatively connected to the mechanical device 20.
[0044] Data acquisition device 21, such as Figure 2 The data acquisition device 1, the data acquisition device 2, etc. shown in the figure collect the status data of the mechanical device 20 in real time and send the status data to the mechanical device 20.
[0045] The mechanical device 20 receives the status data reported by the data acquisition device 21, sends the status data to the virtual reality device 22, and receives the control command sent by the virtual reality device 22, and performs corresponding actions based on the control command.
[0046] Among them, when the mechanical equipment includes a boom and a chassis, the control command includes one or more of the following: a boom control command of the mechanical equipment 20, a chassis rotation control command of the mechanical equipment 20, and a walking control command of the mechanical equipment 20.
[0047] The virtual reality device 22 updates the position and state of the pre-created mechanical device model based on the status data received from the mechanical device 21, obtains the hand position data of the object, recognizes the gesture of the object based on the hand position data, obtains the corresponding control command based on the gesture of the object, and sends the control command to the mechanical device 20 to control the mechanical device 20 to perform the corresponding action.
[0048] It should be noted that the mechanical device 20 may also include a controller for receiving data reported by the data acquisition device 21, sending the data reported by the data acquisition device 21 to the virtual reality device 22 through the wireless communication module, receiving and executing control commands sent by the virtual reality device 22, and driving the arm and chassis of the mechanical device 20 to perform corresponding actions. The control logic of the specific controller can be implemented by existing technologies, and can be further developed and implemented on the basis of the hardware platform provided in the embodiment of the present application.
[0049] In practical applications, the data acquisition device 21 includes one or more of the following: a satellite positioning sensor, a direction sensor, a tilt sensor, and one or more sensors for collecting working status parameters of the mechanical equipment 20 .
[0050] The virtual reality device 22 includes: a wearable data acquisition device for acquiring hand position data of the subject. Specifically, the wearable data acquisition device includes a data glove.
[0051] During specific implementation, the virtual reality device 22 also includes: a positioning device, a control device, and a communication device.
[0052] In one possible embodiment, when the mechanical device 20 includes a chassis, a control device is used to control the positioning device to obtain the movement amplitude of the object's hand when controlling the chassis of the mechanical device 20 to move, and determine the movement speed when controlling the mechanical device 20 to move based on the movement amplitude; and a communication device is used to send the movement speed to the mechanical device 20 to control the mechanical device 20 to move.
[0053] In another possible embodiment, when the mechanical device 20 includes an arm, the control device is also used to control the positioning device to obtain the position change and angle change of the object's hand when controlling the movement of the arm of the mechanical device 20, and determine the movement amount and rotation angle of the arm based on the position change and angle change; the communication device is also used to send the determined movement amount and rotation angle of the arm to the mechanical device 20 to control the movement of the arm of the mechanical device 20.
[0054] It should be noted that the control logic of the control device in the virtual reality device 22 can be implemented through existing technologies and can be further developed and implemented based on the hardware platform provided in the embodiment of the present application.
[0055] In some possible implementations, the virtual reality device 22 includes: a wearable display device, which is used to display the established three-dimensional model of the mechanical device 20 and show the position and state of the mechanical device 20. The wearable display device includes virtual reality glasses.
[0056] In some possible implementations, the mechanical device 20 includes: a connection detection device for determining whether the network connection with the virtual reality device 22 is normal through heartbeat detection, and controlling the mechanical device 20 to stop working when it is determined that the network connection with the virtual reality device 22 is abnormal, for example, when the mechanical device 20 is disconnected from the virtual reality device 22.
[0057] It should be noted that, based on the virtual reality device provided in the embodiment of the present application, the control device in the virtual reality device is further developed. When the mechanical device is controlled based on gestures, the mechanical device can be controlled based on gestures or based on gesture groups. In order to avoid the recognition conflict problem between gestures and gesture groups, when a gesture group is recognized, only the mechanical device is controlled to execute the control command corresponding to the gesture group, and the control command corresponding to the current gesture is no longer executed.
[0058] Although the preferred embodiments of the present application have been described, those skilled in the art may make additional changes and modifications to these embodiments once they have learned the basic creative concept. Therefore, the appended claims are intended to be interpreted as including the preferred embodiments and all changes and modifications falling within the scope of the present application.
[0059] Obviously, those skilled in the art can make various changes and modifications to the present application without departing from the spirit and scope of the present application. Thus, if these modifications and variations of the present application fall within the scope of the claims of the present application and their equivalents, the present application is also intended to include these modifications and variations.
Claims
1. A control system for mechanical equipment, characterized in that: include: A mechanical device, a data acquisition device arranged on the mechanical device, and a virtual reality device connected to the mechanical device for communication, wherein: The data acquisition device acquires the status data of the mechanical device in real time and sends the status data to the mechanical device; The mechanical device receives the status data reported by the data acquisition device, sends the status data to the virtual reality device, receives the control command sent by the virtual reality device, and performs corresponding actions based on the control command; The virtual reality device updates the position and state of the pre-created mechanical device model based on the received state data sent by the mechanical device, obtains the hand position data of the object, recognizes the gesture of the object based on the hand position data, obtains a corresponding control command based on the gesture of the object, and sends the control command to the mechanical device; Wherein, the virtual reality device includes a data glove for collecting hand position data of the object.
2. The system according to claim 1, characterized in that The mechanical device includes a chassis, and the virtual reality device includes: a positioning device, a control device, and a communication device; The control device is used to control the positioning device to obtain the movement amplitude of the object's hand when controlling the chassis of the mechanical device to move, and determine the movement speed when controlling the movement of the mechanical device based on the movement amplitude; The communication device is used to send the moving speed to the mechanical device to control the mechanical device to move at the moving speed.
3. The system according to claim 2, characterized in that The mechanical equipment includes a boom; The control device is further used to control the positioning device to obtain the position change and angle change of the object's hand when controlling the arm of the mechanical device to move, and determine the movement amount and rotation angle of the arm based on the position change and the angle change; The communication device is also used to send the determined movement amount and rotation angle of the arm to the mechanical device to control the movement of the arm of the mechanical device.
4. The system according to claim 1, characterized in that The virtual reality device includes: a wearable display device, which is used to display the model of the mechanical device and show the position and status of the mechanical device.
5. The system according to claim 4, characterized in that The wearable display device includes virtual reality glasses.
6. The system according to any one of claims 1 to 5, characterized in that: The data acquisition equipment includes one or more of the following: a satellite positioning sensor, a direction sensor, an inclination sensor, and one or more sensors for collecting working status parameters of the mechanical equipment.
7. The system according to any one of claims 1 to 5, characterized in that: The mechanical equipment includes a boom and a chassis, and the control command includes one or more of the following: a boom control command of the mechanical equipment, a chassis rotation control command of the mechanical equipment, and a walking control command of the mechanical equipment.
8. The system according to any one of claims 1 to 5, characterized in that: The mechanical device includes: a connection detection device, which is used to determine whether the network connection with the virtual reality device is normal through heartbeat detection, and control the mechanical device to stop working when it is determined that the network connection with the virtual reality device is abnormal.