A human-machine collaborative system based on rapid communication

Abstract

The present invention discloses a human-machine collaborative system based on rapid communication, comprising an imaging module, a control module and an execution module. The human-machine collaborative system also comprises a remote collaborative control mode, an autonomous cruise mode and an auxiliary following mode. The cooperation module of the remote collaborative control mode comprises an imaging module, an execution module and a control module. The cooperation module of the auxiliary following mode comprises an imaging module and an execution module. The cooperation module of the autonomous cruise mode comprises an execution module. The imaging module comprises an imaging helmet and imaging glasses. The control module comprises a control magic cube and a control gyroscope. The execution module comprises a robot dog and a drone. The present invention can be accessed by multiple intelligent devices and can cover multiple scene applications in an area.

Description

Technical Field

[0001] The present invention relates to the technical field of remote cooperative control, and more specifically, to a human-machine system based on rapid communication. Background Art

[0002] With the development of advanced intelligent technology, there are more diversified choices for inspection methods in various occasions. Using intelligent auxiliary means to solve the problems existing in manual inspections is a more common application direction.

[0003] At present, wheeled or track robots are relatively popular intelligent inspection equipment, but due to technical limitations, they cannot pass between equipment rooms or floors. Each equipment room needs to be equipped with one robot, which causes a lot of resource waste and generates a lot of additional maintenance and debugging work. The above problems prevent the patrol robot from fully utilizing its strong mobility, causing a lot of additional burden on operation and maintenance personnel, which needs to be solved urgently.

[0004] At the same time, due to the problems that the various systems of smart devices cannot be connected to each other and data cannot be shared, resources cannot be fully utilized. It is necessary to consider how a smart device can participate in multiple scene applications to improve the economic and social benefits of smart devices. Therefore, it is urgent to develop a system that can be accessed by multiple smart devices and smart devices that can cover multiple scene applications in the region. Summary of the invention

[0005] In view of the deficiencies in the prior art, an object of the present invention is to provide a human-machine system based on rapid communication.

[0006] To achieve the above-mentioned purpose, the present invention provides the following technical solutions: a human-machine collaborative system based on rapid communication, comprising an imaging module, a control module and an execution module. The human-machine collaborative system also includes a remote collaborative control mode, an autonomous cruise mode and an auxiliary following mode. The cooperation module of the remote collaborative control mode includes an imaging module, an execution module and a control module. In the remote collaborative control mode, the user synchronizes the field of view of the execution module through the imaging module, and implements the action operation of the execution module through the control module. The remote collaborative control mode is used for high-risk operations and emergency operations and other engineering work that poses a certain threat to personal safety. The cooperation module of the auxiliary following mode includes an imaging module and an execution module. In the auxiliary following mode, the user completes the collection of multi-dimensional data information and the visualization of inspection depth information through the data interaction between the imaging module and the sensor on the execution module. The cooperation module of the autonomous cruise mode includes an execution module. In the autonomous cruise mode, the user presets the route in advance, and the execution module completes daily line patrol through the preset route.

[0007] The present invention is further configured as follows: the imaging module includes an imaging helmet and imaging glasses, the imaging helmet includes a helmet frame, a helmet imaging screen, a helmet communication module and a helmet control chip, the helmet imaging screen, the helmet communication module and the helmet control chip are all arranged on the helmet frame, the helmet imaging screen is used to perform imaging display according to the information transmitted by the execution module, the helmet control chip is used to calculate and control internal data, the helmet communication module is used to interconnect the data of the imaging helmet with that of the execution module, and is used to transmit the quantitative head motion data to the execution module, the imaging glasses include a glasses frame, imaging lenses, a glasses communication module and a glasses control chip, the imaging lenses, the glasses communication module and the glasses control chip are all arranged on the glasses frame, the imaging lenses are used to project virtual images according to the information transmitted by the execution module, the glasses control chip is used to calculate and control the internal data, the glasses communication module is used to interconnect the data of the imaging glasses with that of the execution module, and is used to transmit the received device data collected by the execution module to the imaging lenses.

[0008] The present invention is further configured as follows: the control module includes a control magic cube and a control gyroscope, the control magic cube includes a shell and a gravity sensing device arranged in the shell, the gravity sensing device is used to complete relative position adjustment according to the gravity line to determine the operation reference plane, the shell is provided with a pressure sensor, a displacement sensor and a magic cube communication unit, the pressure sensor is configured to control the movement speed of the execution module based on the pressure on the shell surface, and the speed control of the execution module is completed by the magnitude of the pressure borne by the plane sensed by the pressure sensor, the displacement sensor is used to control the movement direction of the execution module on the horizontal plane, and the control of the execution module is completed by the coordinates of the plane relative to the origin sensed by the displacement sensor, the control gyroscope includes a hollow spherical shell, a suspension ball arranged in the hollow spherical shell and a spherical shell medium for filling the hollow spherical shell, the spherical shell medium is coated on the outside of the suspension ball, the suspension ball is used to calculate the movement of the control gyroscope through the force relationship and motion relationship when the suspension ball is in the spherical shell medium, and a reset sensor and a speed sensor are provided on the hollow spherical shell, the reset sensor is used to complete the reset operation of the control gyroscope, and the speed sensor is used to control the speed and gear position of the execution module.

[0009] The present invention is further configured as follows: the execution module includes a robot dog and a drone, the robot dog includes a robot dog body, a robot dog control chip, a robot dog communication module and a robot dog camera, the number of the robot dog cameras is set to be several, and they are arranged around the robot dog body to obtain the environment in which the robot dog is located, the robot dog control chip and the robot dog communication module are arranged in a cavity formed inside the robot dog body, the robot dog control chip is used to calculate the motion path of the robot dog and control the limbs of the robot dog, the robot dog communication module is used to communicate and transmit data with the imaging module and the control module, the drone includes a drone body, a drone control chip, a drone communication module and a drone camera, the number of the drone cameras is set to be several, and they are arranged around the drone body to obtain the environment in which the drone is located, the drone control chip and the drone communication module are arranged in a cavity formed inside the drone body, the drone control chip is used to calculate the motion path of the drone and control the speed of the drone, and the drone communication module is used to communicate and transmit data with the imaging module and the control module.

[0010] By adopting the above technical solution, when the user selects the remote cooperative control mode, the imaging module, the control module and the execution module work simultaneously, and the data information of the surrounding environment is obtained according to the execution module, and this information is sent to the imaging module, and the imaging module projects the data information to the user. The user can complete the picture adjustment and parameter selection of the data information according to the motion sensor on the imaging module, and complete the acquisition of the motion data of the execution module according to the various sensor units in the control module. The motion data is realized through the execution motor on the execution module.

[0011] When the user selects the auxiliary following mode, the execution module and the imaging module work at the same time, and the execution module and the imaging module are position bound. When the user wears the imaging module, the execution module will start the following mode. The execution module reasonably avoids obstacles while maintaining a certain distance from the imaging module. According to the data information in the visual area collected by the imaging module, after the execution module recognizes the data information, it calls various sensors on the execution module to complete the scanning sensing of the identification device and obtains digital sensor data. The sensor data is sent to the imaging module through the communication unit. The imaging module converts the above data into a visual image through a built-in algorithm and projects it onto the user's visual interface.

[0012] When the user selects the autonomous cruise mode, the execution module starts working. The execution module completes the line inspection according to the preset path, and completes the equipment scan on the inspection line by calling various sensors on the execution module to complete the inspection task. BRIEF DESCRIPTION OF THE DRAWINGS

[0013] Figure 1 This is a framework diagram of a human-machine collaborative system based on rapid communication according to the present invention;

[0014] Figure 2 It is a schematic diagram of the structure of the imaging helmet;

[0015] Figure 3 is a schematic diagram of the structure of imaging glasses;

[0016] Figure 4 This is a schematic diagram of the structure for controlling the Rubik's Cube;

[0017] Figure 5 This is a schematic diagram of the structure for controlling the gyroscope;

[0018] Figure 6 This is a schematic diagram of the structure of the robot dog;

[0019] Figure 7 This is a schematic diagram of the structure of the UAV;

[0020] 1. Helmet frame; 2. Imaging lens; 3. Glasses communication module; 4. Control chip of glasses; 5. Gravity sensing device; 6. Pressure sensor; 7. Displacement sensor; 8. Cube communication unit; 9. Hollow spherical shell; 10. Suspended sphere; 11. Spherical shell medium; 12. Reset sensor; 13. Speed ​​sensor; 14. Robot dog body; 15. Robot dog control chip; 16. Robot dog communication module; 17. Robot dog camera; 18. Drone body; 19. Drone control chip; 20. Drone communication module; 21. Drone control chip; 22. Drone communication module; 23. Drone camera. DETAILED DESCRIPTION

[0021] Reference Figures 1 to 7 An embodiment of a human-machine collaborative system based on rapid communication of the present invention is further described.

[0022] For ease of explanation, spatial relative terms such as "upper", "lower", "left", "right" and the like are used in the embodiments to illustrate the relationship of one element or feature shown in the figure relative to another element or feature. It should be understood that, in addition to the orientation shown in the figure, the spatial terms are intended to include different orientations of the device in use or operation. For example, if the device in the figure is inverted, the element described as being "under" other elements or features will be positioned "on" other elements or features. Therefore, the exemplary term "under" can include both upper and lower orientations. The device can be positioned in other ways (rotated 90 degrees or in other orientations), and the spatial relative descriptions used here can be interpreted accordingly.

[0023] Furthermore, relational terms such as “first” and “second” and the like are merely used to distinguish one component from another having the same name, but do not necessarily require or imply any such actual relationship or order between these components.

[0024] A human-machine collaborative system based on rapid communication includes an imaging module, a control module and an execution module. The human-machine collaborative system also includes a remote collaborative control mode, an autonomous cruise mode and an auxiliary following mode. The cooperation module of the remote collaborative control mode includes an imaging module, an execution module and a control module. In the remote collaborative control mode, the user synchronizes the field of view of the execution module through the imaging module and implements the action operation of the execution module through the control module. The remote collaborative control mode is used for engineering work that poses a certain threat to personal safety, such as high-risk operations and emergency operations. The cooperation module of the auxiliary following mode includes an imaging module and an execution module. In the auxiliary following mode, the user completes the collection of multi-dimensional data information and the visualization of inspection depth information through data interaction between the imaging module and the sensor on the execution module. The cooperation module of the autonomous cruise mode includes an execution module. In the autonomous cruise mode, the user presets the route in advance, and the execution module completes daily line patrol through the preset route.

[0025] The imaging helmet includes a helmet frame 10, a helmet imaging screen 11, a helmet gyroscope, a helmet capture camera, a helmet screen projection device, a helmet communication device and a helmet control chip 13, wherein the helmet imaging screen 11, the helmet gyroscope, the helmet capture camera, the helmet screen projection device, the helmet communication device and the helmet control chip 13 are all arranged on the helmet frame 10, the helmet capture camera is located on the front side of the helmet frame 10, and is used to capture the hand image on the front side of the helmet, the helmet imaging screen 11 and the helmet screen projection device are located on the front side of the helmet frame 10, and are used for imaging display, the helmet gyroscope is used to synchronize head movements, such as rotation, and convert them into quantitative motion data, the helmet control chip 13 will realize the communication, calculation and control of the internal data of the immersive imaging helmet, the helmet communication device will realize the data interconnection between the immersive imaging helmet and the execution module, after the helmet communication device transmits the quantitative head motion data to the execution module, the helmet camera motor installed on the execution module realizes the synchronization of the camera direction, and at the same time the helmet communication device transmits the picture received from the helmet camera of the execution module to the helmet screen projection device, and the helmet screen projection device images the received picture on the imaging screen.

[0026] The imaging glasses include a glasses frame 20, an imaging lens 21, a glasses communication module 22, a glasses front camera, a glasses screen projection module and a glasses control chip 23. The imaging lens 21, the glasses communication module 22, the glasses front camera, the glasses screen projection module and the glasses control chip 23 are arranged on the glasses frame 20. The glasses front camera is located on the front side of the glasses frame 20 for capturing images of the front device and the hand. The glasses imaging lens 21 is embedded in the glasses frame 20. The glasses screen projection module is located on both sides of the glasses frame 20 for projecting a virtual image at the imaging lens 21. The glasses control chip 23 realizes imaging. The glasses' internal data communication, calculation and control, the glasses control chip 23 includes positioning components, and the execution module is followed at all times through the positioning components. The glasses communication module 22 will realize the data interconnection between the imaging glasses and the execution module. The glasses communication module 22 will transmit the device data collected by the sensor of the execution module to the glasses projection module, and the glasses projection module will image the received device data on the imaging lens 21. At the same time, the glasses communication module 22 will transmit the device information recognized by the front camera of the glasses to the execution module, and the execution module will control the sensor thereon to complete the data collection of the corresponding device.

[0027] The control cube has twelve plane frames, including a touch sensor, two pressure sensors 31, a displacement sensor 32 and a data processing and communication unit. The touch sensor is built into the inner surface of the control cube to obtain touch data on its outer surface. The touch sensor establishes a data connection with the data processing and communication unit to transmit the acquired touch data to the data processing and communication unit, which processes the data and transmits the processed data to the execution module. The pressure sensors 31 are respectively built into the inner surface of the control cube to obtain pressure data on the outer surface of the control cube. The pressure sensors 31 are connected to the data processing and communication unit. The unit establishes a data connection, transmits the acquired pressure data to the data processing and communication unit, which processes the data and transmits the processed data to the execution module. The displacement sensor 32 is embedded in the outer surface of the control cube, and is used to obtain the displacement data between the outer surface of the control cube and the plane contacted by the outer surface. The displacement sensor 32 establishes a data connection with the data processing and communication unit, transmits the acquired displacement data to the data processing and communication unit, which processes the data and transmits the processed data to the execution module. The data processing and communication unit is arranged inside the space enclosed by the plane frame of the control cube.

[0028] The control gyroscope is set as a sphere, and sensors are set on the surface and cavity of the sphere. The movement of the execution module is controlled by the movement of the entire sphere. The control of the execution module is completed by transmitting the data obtained by the sensor through remote communication. The control gyroscope completes the acquisition of the movement data of the execution module through the acceleration sensor 44 or the gravity sensor or the pressure sensor 31. The control gyroscope includes a hollow spherical shell 40, a spherical shell medium 42, a speed sensor 44, a reset sensor 43, a data processing and communication unit, a suspension ball 41 and an acceleration sensor 44. The interior of the hollow spherical shell 40 is filled with a spherical shell medium 42, and a suspension ball 41 is suspended in the center of the hollow spherical shell 40. The material of the hollow spherical shell 40 is designed to be The spherical shell medium 42 is a magnetic material, the spherical shell medium 42 is a gas medium, the material of the suspension ball 41 is a material that can generate magnetic force with the material of the hollow spherical shell 40, the suspension ball 41 is suspended and placed in the center of the cavity of the hollow spherical shell 40, the acceleration sensor 44 is located on the suspension ball 41, and the movement of the control gyroscope is calculated by obtaining the force relationship and motion relationship of the suspension ball 41 in the spherical shell medium 42. The reset sensor 43 and the speed sensor 44 are set on the surface of the hollow spherical shell 40. The reset operation of the control gyroscope is completed by the reset sensor 43, that is, the accumulated displacement data is cleared, and the spatial position of the control gyroscope when the reset sensor 43 is triggered is used as the origin, and the displacement data is accumulated again.

[0029] The robot dog includes a robot dog body 50, a robot dog control chip 51, a robot dog communication device, and a robot dog camera 53. The robot dog control chip 51, the robot dog communication device, and the robot dog camera 53 are arranged on the robot dog body 50. The robot dog cameras 53 are arranged in 7 numbers and are evenly distributed on both sides and the tail of the robot dog body 50. The robot dog control chip 51 and the robot dog communication device are located in a cavity wrapped by the shell of the robot dog body 50. The robot dog control chip 51 is used for motion calculation and limb control. The robot dog communication device is used for communication and data transmission with the imaging module and the control module. Sensors can be installed on the main body of the robot dog body 50 according to specific needs, and modules can be combined with various sensors to realize specific scene applications. The sensor types may include electromagnetic sensors, infrared temperature sensors, infrared distance sensors, video sensors, radar sensors, sound wave sensors, three-dimensional depth of field sensors, etc. Mechanical equipment can be installed on the main body of the robot dog body 50 according to specific needs, and modules can be combined with various mechanical equipment to realize specific scene applications. The mechanical equipment types may include operating robotic arms, sensing robotic arms, cleaning robotic arms, etc.

[0030] The drone includes a drone body 60, a drone control chip 61, a drone communication device and a drone camera 63. The drone control chip 61, the drone communication device and the drone camera 63 are arranged on the drone body 60. The drone cameras 63 are arranged in five numbers. The front camera of the drone is arranged at the head of the drone body 60. The remaining four drone cameras 63 are evenly distributed on both sides and the tail of the drone body 60. The drone camera 63 realizes video acquisition of the environment in which the drone is located. The drone bottom camera is arranged at the bottom of the drone body 60 for acquiring the video picture below the drone and calculating the corresponding height. The drone control chip 61 and the drone communication device are located outside the drone body 60. In the cavity wrapped by the shell, the drone control chip 61 is used to calculate the motion and control the blade speed. The drone communication device is used to communicate and transmit data with the imaging module and the control module. Sensors can be installed on the drone body 60 according to specific needs, and modules can be combined with various sensors to achieve specific scene applications. Sensor types may include electromagnetic sensors, infrared temperature sensors, infrared distance sensors, video sensors, radar sensors, sound wave sensors, three-dimensional depth of field sensors, etc. Mechanical equipment can be installed on the drone body 60 according to specific needs, and modules can be combined with various mechanical equipment to achieve specific scene applications. Mechanical equipment types may include operating robotic arms, sensing robotic arms, cleaning robotic arms, etc.

[0031] When the user selects the remote collaborative control mode, the imaging module, the control module and the execution module work simultaneously. The video acquisition unit in the execution module collects video data of the surrounding environment of the execution module in real time, and the video data is sent to the imaging module through the communication unit. The imaging unit of the imaging module projects the video data to the user visual interface. The user can complete the picture adjustment and parameter selection of the video data according to the motion sensor on the imaging module, and complete the acquisition of the motion data of the execution module according to the various sensor units in the control module, including speed data, direction data, etc. The motion data is sent to the execution module through the communication unit, and the execution motor on the execution module completes the implementation of the motion data.

[0032] When the user selects the auxiliary following mode, the execution module and the imaging module work at the same time. The execution module and the imaging module complete the position binding through the positioning unit. When the user wears the imaging module, the execution module will start the following mode due to the position binding. The execution module reasonably avoids obstacles while maintaining a certain distance from the imaging module. The video acquisition unit of the imaging module collects the user's real-time visual area in real time, identifies the equipment in the visual area, and collects video data in the visual area at the same time. The video data and the identification data are sent to the execution module through the communication unit. The execution module adjusts the viewing angle of the execution module to the same viewing angle as the video data by adjusting the direction, and calls various sensors on the execution module according to the identification data to complete the scanning sensing of the identification equipment, and obtains digital sensor data, including charged data, infrared thermal imaging data, etc. The sensor data is sent to the imaging module through the communication unit. The imaging module converts the above data into a visual image through a built-in algorithm and projects it onto the user's visual interface.

[0033] When the user selects the autonomous cruise mode, the execution module works, the execution module completes the line inspection according to the preset path, and completes the equipment scan on the inspection line by calling various sensors on the execution module to complete the inspection task. The above is only a preferred embodiment of the present invention and is not intended to limit the present invention. The usual changes and substitutions made by technicians in this field within the scope of the technical solution of the present invention should be included in the protection scope of the present invention.

Claims

1. A human-machine collaborative system based on rapid communication, Features: The human-machine collaborative system includes an imaging module, a control module and an execution module. The human-machine collaborative system also includes a remote cooperative control mode, an autonomous cruise mode and an auxiliary following mode. The cooperation module of the remote cooperative control mode includes an imaging module, an execution module and a control module. In the remote cooperative control mode, the user synchronizes the field of view of the execution module through the imaging module and implements the action operation of the execution module through the control module. The remote cooperative control mode is used to perform engineering work that threatens personal safety. The cooperation module of the auxiliary following mode includes an imaging module and an execution module. In the auxiliary following mode, the user completes the collection of multi-dimensional data information and the visualization of inspection depth information through the data interaction between the imaging module and the sensor on the execution module. The cooperation module of the autonomous cruise mode includes an execution module. In the autonomous cruise mode, the user presets the route in advance, and the execution module completes daily line patrol through the preset route. The imaging module includes an imaging helmet and imaging glasses. The imaging helmet includes a helmet frame, a helmet imaging screen, a helmet communication module and a helmet control chip. The helmet imaging screen, the helmet communication module and the helmet control chip are all arranged on the helmet frame. The helmet imaging screen is used to perform imaging display according to the information transmitted by the execution module. The helmet control chip is used to calculate and control internal data. The helmet communication module is used to interconnect the data of the imaging helmet with that of the execution module, and is used to transmit quantitative head motion data to the execution module. The imaging glasses include a glasses frame, an imaging lens, a glasses communication module and a glasses control chip. The imaging lens, the glasses communication module and the glasses control chip are all arranged on the glasses frame. The imaging lens is used to project a virtual image according to the information transmitted by the execution module. The glasses control chip is used to calculate and control internal data. The glasses communication module is used to interconnect the data of the imaging glasses with that of the execution module, and is used to transmit the received device data collected by the execution module to the imaging lens. The control module includes a control magic cube and a control gyroscope. The control magic cube includes a shell and a gravity sensing device arranged in the shell. The gravity sensing device is used to complete relative position adjustment according to the gravity line to determine the operation reference plane. The shell is provided with a pressure sensor, a displacement sensor and a magic cube communication unit. The pressure sensor is configured to control the movement speed of the execution module based on the pressure on the shell surface. The speed control of the execution module is completed by the magnitude of the pressure on the shell surface sensed by the pressure sensor. The displacement sensor is used to control the movement direction of the execution module on the horizontal plane. The control of the execution module is completed by the coordinates of the shell surface relative to the origin sensed by the displacement sensor. The control gyroscope includes a hollow spherical shell, a suspension ball arranged in the hollow spherical shell and a spherical shell medium for filling the hollow spherical shell. The spherical shell medium is coated on the outside of the suspension ball. The suspension ball is used to calculate the movement of the control gyroscope through the force relationship and motion relationship when the suspension ball is in the spherical shell medium. A reset sensor and a speed sensor are provided on the hollow spherical shell. The reset sensor is used to complete the reset operation of the control gyroscope. The speed sensor is used to control the speed and gear position of the execution module.

2. A human-machine collaborative system based on rapid communication according to claim 1, Features: The execution module includes a robot dog and a drone. The robot dog includes a robot dog body, a robot dog control chip, a robot dog communication module and a robot dog camera. The number of the robot dog cameras is set to be several and is arranged around the robot dog body to obtain the environment in which the robot dog is located. The robot dog control chip and the robot dog communication module are arranged in a cavity formed inside the robot dog body. The robot dog control chip is used to calculate the motion path of the robot dog and control the limbs of the robot dog. The robot dog communication module is used to communicate and transmit data with the imaging module and the control module. The drone includes a drone body, a drone control chip, a drone communication module and a drone camera. The number of the drone cameras is set to be several and is arranged around the drone body to obtain the environment in which the drone is located. The drone control chip and the drone communication module are arranged in a cavity formed inside the drone body. The drone control chip is used to calculate the motion path of the drone and control the speed of the drone. The drone communication module is used to communicate and transmit data with the imaging module and the control module.

Images