A multi-unmanned device cluster real-time control method and system based on a data glove

Abstract

This invention discloses a real-time control method and system for a cluster of multiple unmanned driving devices based on a data glove, belonging to the field of unmanned driving control technology. It addresses the problem of controlling a cluster of multiple unmanned driving devices based on a data glove. The invention utilizes an inertial measurement unit and a tension sensor in the smart glove, which are connected to a wireless network via a processor, and then to the host computers of multiple unmanned driving devices. The host computers of these devices are connected to the chassis development boards of the unmanned driving devices. The inertial measurement unit in the smart glove captures the dynamic trajectory of the hand using an accelerometer and gyroscope, while the tension sensor collects real-time finger bending information and identifies static gestures. A server is created on the processor, and its monitoring function is enabled. The host computers of the unmanned driving devices send real-time control commands from the smart glove to the chassis development boards, achieving real-time control of the cluster of multiple unmanned driving devices using the data glove.

Description

Technical Field

[0001] This invention belongs to the field of unmanned driving control technology, specifically relating to a real-time control method and system for a cluster of multiple unmanned driving devices based on data gloves. Background Technology

[0002] With the rapid development of computer technology and the increasing maturity of the human-computer interaction field, wearable devices are becoming increasingly popular in people's daily lives. Gesture recognition technology, as an important branch of human-computer interaction, has not only achieved significant theoretical results but has also been widely applied in various fields such as virtual reality (VR), augmented reality (AR), smart homes, game control, and medical devices. Wearable data gloves combined with gesture recognition technology are gradually being applied to the swarm control of various unmanned devices such as drones and autonomous vehicles through gesture control devices. However, existing technologies are mostly limited to one-to-one control or fixed formation control, lacking flexible multi-target dynamic switching capabilities. Summary of the Invention

[0003] The problem this invention aims to solve is to realize the cluster control of multiple unmanned driving devices based on data gloves, and proposes a real-time control method and system for multiple unmanned driving devices based on data gloves.

[0004] To achieve the above objectives, the present invention provides the following technical solution:

[0005] A real-time control system for a cluster of multiple unmanned driving devices based on a data glove includes a smart glove, a wireless network, and multiple unmanned driving devices;

[0006] The inertial measurement unit and tensile sensor in the smart glove are connected to a wireless network via a processor. The wireless network is connected to the host computer of multiple unmanned driving devices, and the host computer of multiple unmanned driving devices is connected to the chassis development board of multiple unmanned driving devices.

[0007] The inertial measurement unit in the smart glove is used to capture the dynamic trajectory of the hand through an accelerometer and a gyroscope. The stretch sensor is used to collect the finger bending degree in real time and recognize static gestures. A server is created on the processor to create server sockets and bind them to a preset fixed IP address and port number, and to enable the server's listening function to realize the connection between the smart glove and the unmanned driving device.

[0008] The host computer of the unmanned vehicle is used to send real-time control commands of the smart glove to the chassis development board, so as to realize real-time control of multiple unmanned vehicle clusters of data gloves.

[0009] Furthermore, the host computer of the autonomous driving device creates a ROS node publisher for communicating with the chassis ROS node on the chassis development board; the ROS node of the host computer of the autonomous driving device creates a client socket for connecting to the server of the smart glove via a wireless network and sending the device name of the autonomous driving device to the server of the smart glove to complete the registration on the server.

[0010] Furthermore, the tensile sensor is a 5-dimensional tensile sensor, and the inertial measurement unit is a 6-dimensional inertial measurement unit.

[0011] A control method for a real-time control system for a cluster of multiple unmanned vehicles based on data gloves includes the following steps:

[0012] S1. Develop a gesture switching strategy for controlling multiple autonomous driving devices with smart gloves;

[0013] S2. Construct a gesture control strategy for intelligent gloves to control multiple autonomous driving devices;

[0014] S3. Establish initial connections between the smart glove and multiple autonomous driving devices via wireless network;

[0015] S4. Initiate a start gesture command. The smart glove recognizes the gesture and sends the recognized start gesture command to multiple autonomous driving devices via wireless network, thus starting the multiple autonomous driving devices.

[0016] S5. Use a gesture switching strategy to determine the autonomous driving device to be controlled among multiple autonomous driving devices that have been started, and then use a gesture control strategy to control the driving status of the autonomous driving device.

[0017] Furthermore, in step S1, the gesture is mapped to a state lock, and the state lock is set as a binary bit mask, which corresponds to the currently controlled unmanned driving device. For n unmanned driving devices to be controlled, 1≤n≤10, the state lock corresponds to an n-bit binary number. The state lock includes a global lock and a device lock.

[0018] Set gesture 0 as a global lock to control all autonomous driving devices. The corresponding on / off value of gesture 0 is G0=0, and the binary number is all 0.

[0019] Set gesture k i This is a device lock used to control the k-th autonomous vehicle, where k is any one of n, corresponding to the gesture k. i switch value The (k-1)th bit of a binary number is 1;

[0020] If the user makes m gestures in sequence, and the index of each gesture is k1, k2, ..., k i ...,km Then, the state lock Sm after m gesture transitions is the bitwise OR operation of the corresponding gesture's on / off value, with the formula:

[0021] ;

[0022] For calculating the state lock in real-time control, the state lock after the previous switch of the m-th gesture is superimposed with the current gesture, using the following formula:

[0023]

[0024] in, This is the state lock after the previous toggle of m gestures. The current on / off value for the gesture switching;

[0025] The obtained current state lock is then mapped to different autonomous driving devices to complete cluster control. The mapping logic is as follows:

[0026] right Calculate n times, where j is an integer. The j-th bit of the binary representation, 0≤j≤n-1, result= ( j);

[0027] in, This is the state lock after the current gesture switch, and result is the result; when the obtained result is 1, the j-th unmanned vehicle is added to the control list;

[0028] The gesture switching strategy is set to a mutual exclusion priority mechanism between global lock and device lock.

[0029] Furthermore, in step S2, the gesture control strategy for the smart glove to control multiple autonomous driving devices is based on wrist rotation to control the autonomous driving devices. Raising the wrist corresponds to the autonomous driving device moving forward, lowering the wrist corresponds to the autonomous driving device moving backward, rotating the wrist to the left corresponds to the autonomous driving device turning left, and rotating the wrist to the right corresponds to the autonomous driving device turning right, thus realizing natural and convenient gesture control commands.

[0030] Furthermore, the specific implementation method of step S3 includes the following steps:

[0031] S3.1. The smart glove establishes a network connection. The smart glove first automatically connects to the preset wireless network, obtains a local area network IP address, and performs hardware initialization to start the hardware environment for gesture recognition function.

[0032] S3.2. Create a server on the processor of the smart glove, create a server socket, bind it to a preset fixed IP address and port number, and enable the server's listening function to wait for the autonomous driving device to initiate a connection request;

[0033] S3.3. The unmanned vehicle establishes a network connection, initializes the host computer ROS node, creates a ROS node publisher and a ROS topic for publishing commands to the host computer ROS node, and uses it to communicate with the chassis ROS node of the unmanned vehicle.

[0034] S3.4. The host ROS node of the autonomous driving device creates a client socket, connects to the server of the smart glove, and sends the device name of the autonomous driving device to the server of the smart glove to complete the registration process on the server side.

[0035] S3.5. The unmanned vehicle starts the chassis ROS node, subscribes to the ROS topic that issues commands to the host computer ROS node, and establishes a connection with the chassis through the bus.

[0036] Furthermore, the specific implementation method of step S4 includes the following steps:

[0037] S4.1. The operator raises their wrist to indicate that the unmanned vehicle is moving forward as a start gesture command. The smart glove recognizes the gesture and, upon recognizing the start gesture command, sends the command to the ROS node of the unmanned vehicle through the server.

[0038] S4.2. The host computer ROS node of the unmanned vehicle parses the gesture commands and converts the parsed string commands into ROS message format, and publishes them to the ROS topic of the host computer ROS node publishing command through the ROS node publisher;

[0039] S4.3. After receiving the ROS topic message issued by the host computer ROS node, the chassis ROS node of the autonomous driving device converts it into a bus command of the underlying controller of the autonomous driving device and sends it to the chassis through the bus to start the autonomous driving device.

[0040] Furthermore, the specific implementation method of step S5 is as follows: after the gesture switching strategy is matched to the autonomous driving device to be controlled, the wrist is rotated through the gesture control strategy. The glove processor calculates the pitch angle and roll angle according to the gesture control strategy and sends the gesture control command to the autonomous driving device to realize real-time control of multiple autonomous driving device clusters based on the data glove.

[0041] The beneficial effects of this invention are:

[0042] This invention discloses a real-time control system for a cluster of multiple unmanned vehicles based on a data glove. This system achieves precise control of multiple unmanned vehicles through gesture recognition. The glove acts as the control center, integrating advanced sensor technology and a high-efficiency communication module. It can capture hand gestures in real time and convert them into control commands, which are then sent to the unmanned vehicles via a wireless network. The unmanned vehicles establish a connection with the glove through pre-registration, receive and parse the commands, and then drive the unmanned vehicles to complete the corresponding actions. The entire system design fully considers real-time performance, multi-device compatibility, and ease of operation, aiming to provide an efficient and intuitive intelligent control solution.

[0043] This invention discloses a real-time control method for a cluster of multiple unmanned vehicles based on a data glove. It achieves dynamic and flexible "one-to-many" control by leveraging gesture switching strategies and one-to-many interaction technology. This fundamentally overcomes the limitations of existing technologies (one-to-one control or fixed formations), enabling a single operator to control a cluster of multiple (≥3) unmanned vehicles in real time and dynamically. The operator can freely select the specific target unmanned vehicle (or subset) to be controlled and switch control objects instantly, without needing to pre-set fixed formations or complex mode switching operations.

[0044] The present invention provides a real-time control method for a cluster of multiple unmanned vehicles based on a data glove, which is particularly suitable for complex scenarios that require frequent adjustment of the control focus and assignment of different tasks to different unmanned vehicles (such as collaborative search, reconnaissance, transportation, formation change, etc.).

[0045] This invention discloses a real-time control method for a cluster of multiple unmanned vehicles based on data gloves. The method offers intuitive and efficient interaction, significantly reducing cognitive load and operational complexity. By utilizing data gloves to capture natural hand movements, control intentions are directly mapped to gesture commands, resulting in an interaction method highly aligned with human intuition and with low learning costs. "Rich control" is achieved with "few gestures": Through a carefully designed gesture switching strategy, control commands for any selected target can be created using a limited number of basic gestures, avoiding the need to design separate gestures for each target or command, greatly simplifying the operation process and reducing memory burden. Operation is convenient and fast; gesture switching is typically faster than selecting targets and switching modes on traditional interfaces (such as keyboards, mice, and multi-joysticks), improving operational efficiency and response speed.

[0046] This invention discloses a real-time control method for a cluster of multiple unmanned vehicles based on data gloves, enhancing the system's scalability and universality. The core of the control strategy lies in "target designation + command transmission," theoretically not limited by the number of specific unmanned vehicles (limited by recognition accuracy and communication bandwidth), thus providing stronger system scalability. Adding or removing unmanned vehicle nodes only requires registration / deregistration at the system level, without altering the core interaction logic. Gesture control commands are abstract (e.g., "forward," "left turn," "stop," "select target N"), independent of the underlying drivers of specific unmanned vehicles, making the solution easily adaptable to different types of unmanned vehicle platforms (ground, air, water).

[0047] The present invention discloses a real-time control method for a cluster of multiple unmanned vehicles based on a data glove. Typically, target selection and basic control can be completed with only one hand wearing the data glove, freeing up the other hand for other tasks (such as operating other devices or viewing map information). This reduces the time operators spend looking down at traditional control screens or complex consoles, allowing them to focus more on the task environment and the real-time status of the unmanned vehicle cluster, thus improving overall situational awareness and safety.

[0048] This invention discloses a real-time control system for a cluster of multiple unmanned vehicles based on data gloves. A single data glove can replace multiple remote controls or dedicated control stations (required for traditional one-to-one control), reducing hardware investment costs. Intuitive gesture interaction simplifies and speeds up operator training, lowering the training threshold and time cost for professional operators. This invention creatively utilizes data gloves and an efficient gesture switching strategy, enabling operators to perform real-time, dynamic, and flexible target assignment and control of a cluster of multiple unmanned vehicles using only a few intuitive and convenient gestures. This completely overcomes the bottlenecks of traditional methods in terms of flexibility, efficiency, and scalability, providing a powerful and easy-to-use human-machine interaction method for multi-agent collaborative operations in complex environments. Its advantages are reflected in multiple dimensions, including dynamic control capabilities, interaction efficiency, system scalability, operator experience, and cost-effectiveness. Attached Figure Description

[0049] Figure 1 This is a schematic diagram of the structure of a real-time control system for a cluster of multiple unmanned driving devices based on data gloves, as described in this invention.

[0050] Figure 2 This is a flowchart of a real-time control method for a cluster of multiple unmanned driving devices based on data gloves, as described in this invention.

[0051] Figure 3 This is a diagram illustrating the gesture switching strategy framework for one glove used on three autonomous vehicles in this embodiment. Detailed Implementation

[0052] To make the objectives, technical solutions, and advantages of this invention clearer, the invention will be further described in detail below with reference to the accompanying drawings and specific embodiments. It should be understood that the specific embodiments described herein are only for explaining the invention and are not intended to limit the invention; that is, the described specific embodiments are merely a part of the embodiments of the invention, and not all of them. The components of the specific embodiments of the invention described and shown in the accompanying drawings can generally be arranged and designed in various different configurations, and the invention may also have other embodiments.

[0053] Therefore, the following detailed description of specific embodiments of the invention provided in the accompanying drawings is not intended to limit the scope of the claimed invention, but merely to illustrate selected specific embodiments of the invention. All other specific embodiments obtained by those skilled in the art based on these specific embodiments without inventive effort are within the scope of protection of this invention.

[0054] To further understand the invention's content, features, and effects, the following specific embodiments are provided, along with accompanying drawings. Figure 1 - Appendix Figure 3 Detailed explanation is as follows:

[0055] Example 1:

[0056] A real-time control system for a cluster of multiple unmanned driving devices based on a data glove includes a smart glove, a wireless network, and multiple unmanned driving devices;

[0057] The inertial measurement unit and tensile sensor in the smart glove are connected to a wireless network via a processor. The wireless network is connected to the host computer of multiple unmanned driving devices, and the host computer of multiple unmanned driving devices is connected to the chassis development board of multiple unmanned driving devices.

[0058] The inertial measurement unit in the smart glove is used to capture the dynamic trajectory of the hand through an accelerometer and a gyroscope. The stretch sensor is used to collect the finger bending degree in real time and recognize static gestures. A server is created on the processor to create server sockets and bind them to a preset fixed IP address and port number, and to enable the server's listening function to realize the connection between the smart glove and the unmanned driving device.

[0059] The host computer of the unmanned vehicle is used to send real-time control commands of the smart glove to the chassis development board, so as to realize real-time control of multiple unmanned vehicle clusters of data gloves.

[0060] Furthermore, the host computer of the autonomous driving device creates a ROS node publisher for communicating with the chassis ROS node on the chassis development board; the ROS node of the host computer of the autonomous driving device creates a client socket for connecting to the server of the smart glove via a wireless network and sending the device name of the autonomous driving device to the server of the smart glove to complete the registration on the server.

[0061] Furthermore, the tensile sensor is a 5-dimensional tensile sensor, and the inertial measurement unit is a 6-dimensional inertial measurement unit.

[0062] Example 2:

[0063] A control method for a real-time control system for a cluster of multiple unmanned driving devices based on the data glove described in Embodiment 1 includes the following steps:

[0064] S1. Develop a gesture switching strategy for controlling multiple autonomous driving devices with smart gloves;

[0065] Furthermore, in step S1, the gesture is mapped to a state lock, and the state lock is set as a binary bit mask, which corresponds to the currently controlled unmanned driving device. For n unmanned driving devices to be controlled, 1≤n≤10, the state lock corresponds to an n-bit binary number. The state lock includes a global lock and a device lock.

[0066] Set gesture 0 as a global lock to control all autonomous driving devices. The corresponding on / off value of gesture 0 is G0=0, and the binary number is all 0.

[0067] Set gesture k i This is a device lock used to control the k-th autonomous vehicle, where k is any one of n, corresponding to the gesture k. i switch value The (k-1)th bit of a binary number is 1;

[0068] If the user makes m gestures in sequence, and the index of each gesture is k1, k2, ..., k i ...,k m Then, the state lock Sm after m gesture transitions is the bitwise OR operation of the corresponding gesture's on / off value, with the formula:

[0069] ;

[0070] For calculating the state lock in real-time control, the state lock after the previous switch of the m-th gesture is superimposed with the current gesture, using the following formula:

[0071]

[0072] in, This is the state lock after the previous toggle of m gestures. The current on / off value for the gesture switching;

[0073] The obtained current state lock is then mapped to different autonomous driving devices to complete cluster control. The mapping logic is as follows:

[0074] right Calculate n times, where j is an integer. The j-th bit of the binary representation, 0≤j≤n-1, result= ( j);

[0075] in, This is the state lock after the current gesture switch, and result is the result; when the obtained result is 1, the j-th unmanned vehicle is added to the control list;

[0076] The gesture switching strategy is set to a mutual exclusion priority mechanism between global lock and device lock.

[0077] Furthermore, assuming there are three autonomous driving devices, we define a 3-bit binary integer to represent a lock. The mapping relationship between the integer and the lock is as follows:

[0078] Integer 0 (binary 000): Global lock, controlling all autonomous driving equipment;

[0079] Integer 1 (binary 001): Device lock, controlling unmanned device 1;

[0080] Integer 2 (binary 010): Device lock, controlling unmanned device 2;

[0081] Integer 3 (binary 011): Device lock, controlling unmanned device 1 and unmanned device 2;

[0082] Integer 4 (binary 100): Device lock, controlling unmanned device 3;

[0083] Integer 4 (binary 101): Device lock, controlling unmanned device 1 and unmanned device 3;

[0084] Integer 6 (binary 110): Device lock, controlling unmanned device 2 and unmanned device 3;

[0085] Integer 7 (binary 111): Device lock, controls all unmanned equipment.

[0086] Since there are three autonomous driving devices, the mapping relationship between gestures and status lock switches is as follows:

[0087] Gesture 0 corresponds to the binary number of the switch (000);

[0088] Gesture 1 corresponds to the switch binary number (001);

[0089] Gesture 2 corresponds to the switch binary number (010);

[0090] Gesture 3 corresponds to the switch binary number (100).

[0091] Furthermore, the mapping relationship of gestures is that gesture 0 corresponds to the global lock, which is used to control all devices; gesture k corresponds to the device lock [k], which can only control the unmanned device k; gesture m + n corresponds to the device lock m + n, realizing gesture superposition to control the cluster m + n. The core strategy of gesture switching is the mutual exclusion priority mechanism between the global lock and the device lock. The global lock can forcibly reset the device lock, and the device lock can also forcibly reset the global lock; when the global lock forcibly resets the device lock, all other device locks become invalid and can only control all devices; when the device lock forcibly resets the global lock, the global lock becomes invalid and starts single control or cluster control; the lock variable update and instruction distribution are completed within a single transaction cycle to avoid control conflicts. Through lock superposition, it supports multiple control modes such as 1→N, 1→1, 1→M (M < N); eliminates the need for physical button switching, and gestures directly trigger the lock state migration. The one-to-three gesture switching strategy is as Figure 3 shown.

[0092] S2. Construct a gesture control strategy for an intelligent glove to control multiple unmanned devices;

[0093] Furthermore, the gesture control strategy for the intelligent glove to control multiple unmanned devices in step S2 is to control the unmanned devices by flipping the wrist. Lifting the wrist corresponds to the forward movement of the unmanned device, lowering the wrist corresponds to the backward movement of the unmanned device, left flipping of the wrist corresponds to the left turn of the unmanned device, and right flipping of the wrist corresponds to the right turn of the unmanned device, realizing natural and convenient gesture control instructions.

[0094] Furthermore, design a more natural and intuitive gesture control paradigm to control multiple unmanned devices. Use the inertial measurement unit on the intelligent glove to estimate the hand posture, predict the pitch angle and roll angle of the wrist, and map the wrist posture to the unmanned device control instructions to realize more natural interactive action recognition.

[0095] Use the quaternion attitude estimation method with gyroscope gravity error elimination to achieve more accurate attitude estimation.

[0096] First, for the sake of simplicity in description, the coordinate system corresponding to the gyroscope is called n, and the coordinate system where the virtual mouse is located is called b. It is necessary to convert the coordinate system corresponding to the virtual mouse to the coordinate system corresponding to the gyroscope. According to Euler's theory, the pitch angle , the heading angle and the roll angle These are a series of attitude angles from system b to system n, and the attitude transition matrix is ​​shown below:

[0097] ;

[0098] The above rotation matrix The motion data based on system b was transferred to system n.

[0099] in The definition is as follows:

[0100] ;

[0101] In addition, the quaternion Q needs to be referenced, whose mathematical definition is shown in the following formula. The quaternion also contains rotation information from the b system to the n system:

[0102] ;

[0103] Here, The numbers are real numbers, and i, j, and k are all unit vectors.

[0104] Rotation matrix This can be equivalently represented using quaternions:

[0105] ;

[0106] Ultimately, quaternions can be represented as angle information:

[0107] ;

[0108] The relative stability of the gyroscope and accelerometer is used to determine the state. If both are relatively stable, the device is considered stationary, and the bias is updated only when the device is stationary to avoid errors caused by gravity. For the gyroscope and accelerometer data within the sliding window, the standard deviations of the gyroscope and accelerometer are calculated separately.

[0109] When the standard deviation of the gyroscope and the standard deviation of the accelerometer are both less than the preset threshold, the unmanned driving equipment is considered to be in a stationary state.

[0110] When a stationary state is detected, update the gyroscope bias:

[0111] ;

[0112] Then, gyroscope data correction is performed, and the correction formula is as follows:

[0113]

[0114] in, This is the data after gyroscope calibration. This is the current data from the gyroscope. For gyroscope bias;

[0115] After obtaining the gyroscope data after eliminating gravity, the pitch and roll angles after eliminating gravity errors are calculated using quaternions. This prevents the attitude angles from jumping due to the influence of gravity and obtains more accurate attitude angles under motion interference environments.

[0116] Pitch angle: describes the "forward" or "backward" movement of the device. Positive angle: usually defined as the front of the device tilting downward ("forward"). Negative angle: usually defined as the front of the device tilting upward ("backward").

[0117] Roll angle: Describes the angle at which the equipment turns "left" or "right". Positive angle: usually defined as the equipment turning left. Negative angle: usually defined as the equipment turning right.

[0118] Dead zone: An angular range. When the device tilts within this range, the system considers it a "neutral" position and does not generate any motion commands. This is to prevent accidental movement caused by minor device vibrations or calibration errors.

[0119] Maximum effective angle: An angle threshold. When the device tilt angle reaches or exceeds this value, the command strength reaches its maximum value (100%).

[0120] S3. Establish initial connections between the smart glove and multiple autonomous driving devices via wireless network;

[0121] Furthermore, the specific implementation method of step S3 includes the following steps:

[0122] S3.1. The smart glove establishes a network connection. The smart glove first automatically connects to the preset wireless network, obtains a local area network IP address, and performs hardware initialization to start the hardware environment for gesture recognition function.

[0123] S3.2. Create a server on the processor of the smart glove, create a server socket, bind it to a preset fixed IP address and port number, and enable the server's listening function to wait for the autonomous driving device to initiate a connection request;

[0124] S3.3. The unmanned vehicle establishes a network connection, initializes the host computer ROS node, creates a ROS node publisher and a ROS topic for publishing commands to the host computer ROS node, and uses it to communicate with the chassis ROS node of the unmanned vehicle.

[0125] S3.4. The host ROS node of the autonomous driving device creates a client socket, connects to the server of the smart glove, and sends the device name of the autonomous driving device to the server of the smart glove to complete the registration process on the server side.

[0126] S3.5. The unmanned vehicle starts the chassis ROS node, subscribes to the ROS topic that issues commands to the host computer ROS node, and establishes a connection with the chassis through the bus.

[0127] S4. Initiate a start gesture command. The smart glove recognizes the gesture and sends the recognized start gesture command to multiple autonomous driving devices via wireless network, thus starting the multiple autonomous driving devices.

[0128] Furthermore, the specific implementation method of step S4 includes the following steps:

[0129] S4.1. The operator raises their wrist to indicate that the unmanned vehicle is moving forward as a start gesture command. The smart glove recognizes the gesture and, upon recognizing the start gesture command, sends the command to the ROS node of the unmanned vehicle through the server.

[0130] S4.2. The host computer ROS node of the unmanned vehicle parses the gesture commands and converts the parsed string commands into ROS message format, and publishes them to the ROS topic of the host computer ROS node publishing command through the ROS node publisher;

[0131] S4.3. After receiving the ROS topic message issued by the host computer ROS node, the chassis ROS node of the autonomous driving device converts it into a bus command of the underlying controller of the autonomous driving device and sends it to the chassis through the bus to start the autonomous driving device.

[0132] S5. Use a gesture switching strategy to determine the autonomous driving device to be controlled among multiple autonomous driving devices that have been started, and then use a gesture control strategy to control the driving status of the autonomous driving device.

[0133] Furthermore, the specific implementation method of step S5 is as follows: after the gesture switching strategy is matched to the autonomous driving device to be controlled, the wrist is rotated through the gesture control strategy. The glove processor calculates the pitch angle and roll angle according to the gesture control strategy and sends the gesture control command to the autonomous driving device to realize real-time control of multiple autonomous driving device clusters based on the data glove.

[0134] The specific implementation process of using smart gloves to control three autonomous driving devices is as follows: After the user connects to the three autonomous driving devices using smart gloves, the gesture switching strategy is started.

[0135] When the user makes a gesture 0, G0 = 0 (binary 000), then the state lock S1 = G0 = 0 is mapped to the binary (000) global lock, which can control 3 unmanned vehicles at the same time.

[0136] When the user makes gesture 1, G1 = 1 (binary 001), superimposed with the previous state S1, then state lock S2 = S1(000) | G1(001) = 1 (001), and state lock S2 controls device 1. When the user makes gesture 2, G2 = 1 (binary 010), superimposed with the previous state S2, then state lock S3 = S2(010) | G2(010) = 3 (011), and state lock S3 controls devices 1 and 2. When the user makes gesture 3, G3 = 1 (binary 100), superimposed with the previous state S3, then state lock S4 = S3(011) | G3 = 3 (100) = 7 (111), and state lock S4 controls devices 1, 2, and 3. When the user makes gesture 0, the previously superimposed state is canceled, G0 = 0 (binary 000), then state lock S1 reverts to S1 = 0 (000), and state lock S1 controls all devices.

[0137] When the user makes gesture 2, G2 = 2 (binary 010), superimposed with the previous state S1, then state lock S2 = S1(000) | G2(010) = 2(010), and state lock S2 controls device 2. When the user makes gesture 3, G3 = 3 (binary 010), superimposed with the previous state S2, then state lock S3 = S2(010) | G3(100) = 6(110), and state lock S3 controls both device 2 and device 3.

[0138] After the user selects the desired unmanned device through the gesture switching strategy, they can rotate their wrist using the gesture control strategy. If the user wants the device to move forward, they can raise their wrist. The glove processor will then calculate the pitch and roll angles based on the gesture control strategy and send the forward command to the unmanned device.

[0139] The glove terminal system is the core control unit of the entire interaction solution. Upon startup, the glove terminal first establishes a connection with the wireless network, obtains a local area network IP (Internet Protocol address), and initializes the built-in inertial measurement unit for gesture recognition.

[0140] Subsequently, the glove terminal creates a server socket, binds it to a preset fixed IP and port, and enables the server listening function to wait for connection requests from the autonomous driving device.

[0141] In the main thread, the glove client continuously runs the gesture recognition function, maintaining system activity through an infinite loop. When no new connection requests are received, the system blocks this thread, releasing processor resources to optimize system performance. Once the operating system detects a new connection request via an interrupt signal, the glove client responds immediately, establishing a connection with the autonomous vehicle and receiving the registration information of the newly connected device, typically its unique name, via the client socket. To achieve efficient command sending, the glove client pre-creates a thread pool containing three threads. These threads are blocked when the task queue is empty, waiting for task allocation. When gesture recognition is successful and the sending condition is triggered, the system adds the sending task to the task queue. After being awakened, the threads in the thread pool store the device name and client socket in a file table as key-value pairs, and then send specific string commands to the autonomous vehicle.

[0142] The host computer program for the autonomous vehicle acts as the controlled unit. Each device has a unique device name for registration and identification on the glove terminal. The autonomous vehicle first initializes the ROS (Robot Operating System) node publisher and creates a client socket, while also initializing the address structure of the server to be connected. The device continuously attempts to connect to the glove terminal server in a loop. Once a connection is successful, it sends its device name to the server, completing the registration process. After registration, the autonomous vehicle enters a loop receiving state, receiving messages from the glove terminal server through the client socket. Upon receiving a message, the device immediately parses the message content, converts it into corresponding ROS commands, and then publishes the commands through the ROS node publisher, driving the autonomous vehicle to complete the corresponding actions.

[0143] The ROS communication program between the autonomous vehicle and the chassis, as the program that receives ROS commands, also needs to convert the ROS commands into CAN format commands for the autonomous vehicle. These commands are then sent to the chassis via the CAN bus, thereby enabling chassis control. This design allows the autonomous vehicle to respond to hand gesture commands from the glove in real time, achieving precise control operations.

[0144] It should be noted that relational terms such as "first" and "second" are used merely to distinguish one entity or operation from another, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Furthermore, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitations, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes said element.

[0145] Although this application has been described above with reference to specific embodiments, various modifications can be made and components can be replaced with equivalents without departing from the scope of this application. In particular, as long as there is no structural conflict, the features in the specific embodiments disclosed in this application can be combined with each other in any way. The lack of an exhaustive description of these combinations in this specification is merely for the sake of brevity and resource conservation. Therefore, this application is not limited to the specific embodiments disclosed herein, but includes all technical solutions falling within the scope of the claims.

Claims

1. A real-time control system for a cluster of multiple unmanned driving devices based on data gloves, characterized in that, This includes smart gloves, wireless networks, and multiple autonomous driving devices; The inertial measurement unit and tensile sensor in the smart glove are connected to a wireless network via a processor. The wireless network is connected to the host computer of multiple unmanned driving devices, and the host computer of multiple unmanned driving devices is connected to the chassis development board of multiple unmanned driving devices. The inertial measurement unit in the smart glove is used to capture the dynamic trajectory of the hand through an accelerometer and a gyroscope, and the stretch sensor is used to collect the finger bending degree in real time and recognize static gestures. A server is created on the processor to create server sockets and bind them to a preset fixed IP address and port number, and to enable the server's listening function to enable the connection between the smart glove and the autonomous driving device. The host computer of the unmanned driving equipment is used to send real-time control commands of the smart glove to the chassis development board, so as to realize real-time control of multiple unmanned driving equipment clusters of data gloves; The control method for a real-time control system for a cluster of multiple unmanned driving devices based on a data glove includes the following steps: S1. Develop a gesture switching strategy for controlling multiple autonomous driving devices with smart gloves; Step S1 maps gestures to state locks and sets the state locks to binary bitmasks, which correspond to the currently controlled autonomous driving devices. For n autonomous driving devices to be controlled, 1≤n≤10, the state lock corresponds to an n-bit binary number. The state locks include global locks and device locks. Set gesture 0 as a global lock to control all autonomous driving devices. The corresponding on / off value of gesture 0 is G0=0, and the binary number is all 0. Set gesture k i This is a device lock used to control the k-th autonomous vehicle, where k is any one of n, corresponding to the gesture k. i switch value In a binary number, the (k-1)th bit is 1. If the user makes m gestures in sequence, and the index of each gesture is k1, k2, ..., k i ...,k m Then, the state lock Sm after m gesture transitions is the bitwise OR operation of the corresponding gesture's on / off value, with the formula: ； For calculating the state lock in real-time control, the state lock after the previous switch of the m-th gesture is superimposed with the current gesture, using the following formula: in, This is the state lock after the previous toggle of m gestures. The current on / off value for the gesture switching; The obtained current state lock is then mapped to different autonomous driving devices to complete cluster control. The mapping logic is as follows: right Calculate n times, where j is an integer. The j-th bit of the binary representation, 0≤j≤n-1, result= ( j); in, This is the state lock after the current gesture switch, and result is the result; when the obtained result is 1, the j-th unmanned vehicle is added to the control list; The gesture switching strategy is set to a mutual exclusion priority mechanism between global lock and device lock; S2. Construct a gesture control strategy for intelligent gloves to control multiple autonomous driving devices; S3. Establish initial connections between the smart glove and multiple autonomous driving devices via wireless network; S4. Initiate a start gesture command. The smart glove recognizes the gesture and sends the recognized start gesture command to multiple autonomous driving devices via wireless network, thus starting the multiple autonomous driving devices. S5. Use a gesture switching strategy to determine the autonomous driving device to be controlled among multiple started autonomous driving devices, and then use a gesture control strategy to control the driving state of the autonomous driving device; 2. The real-time control system for a cluster of multiple unmanned driving devices based on a data glove as described in claim 1, characterized in that, The host computer of the unmanned vehicle creates a ROS node publisher for communicating with the chassis ROS node on the chassis development board; the ROS node of the host computer of the unmanned vehicle creates a client socket for connecting to the server of the smart glove via a wireless network and sending the device name of the unmanned vehicle to the server of the smart glove to complete the registration on the server.

3. The real-time control system for a cluster of multiple unmanned driving devices based on a data glove according to claim 2, characterized in that, The tensile sensor is a 5-dimensional tensile sensor, and the inertial measurement unit is a 6-dimensional inertial measurement unit.

4. The real-time control system for a cluster of multiple unmanned driving devices based on a data glove as described in claim 3, characterized in that, In step S2, the gesture control strategy for the smart glove to control multiple autonomous driving devices is based on wrist rotation to control the autonomous driving devices. Raising the wrist corresponds to the autonomous driving device moving forward, lowering the wrist corresponds to the autonomous driving device moving backward, rotating the wrist to the left corresponds to the autonomous driving device turning left, and rotating the wrist to the right corresponds to the autonomous driving device turning right, thus realizing natural and convenient gesture control commands.

5. A real-time control system for a cluster of multiple unmanned driving devices based on a data glove, as described in claim 4, is characterized in that... The specific implementation method of step S3 includes the following steps: S3.

1. The smart glove establishes a network connection. The smart glove first automatically connects to the preset wireless network, obtains a local area network IP address, and performs hardware initialization to start the hardware environment for gesture recognition function. S3.

2. Create a server on the processor of the smart glove, create a server socket, bind it to a preset fixed IP address and port number, and enable the server's listening function to wait for the autonomous driving device to initiate a connection request; S3.

3. The unmanned vehicle establishes a network connection, initializes the host computer ROS node, creates a ROS node publisher and a ROS topic for publishing commands to the host computer ROS node, and uses it to communicate with the chassis ROS node of the unmanned vehicle. S3.

4. The host ROS node of the autonomous driving device creates a client socket, connects to the server of the smart glove, and sends the device name of the autonomous driving device to the server of the smart glove to complete the registration process on the server side. S3.

5. The unmanned vehicle starts the chassis ROS node, subscribes to the ROS topic that issues commands to the host computer ROS node, and establishes a connection with the chassis through the bus.

6. A real-time control system for a cluster of multiple unmanned driving devices based on a data glove, as described in claim 5, is characterized in that... The specific implementation method of step S4 includes the following steps: S4.

1. The operator raises their wrist to indicate that the unmanned vehicle is moving forward as a start gesture command. The smart glove recognizes the gesture and, upon recognizing the start gesture command, sends the command to the ROS node of the unmanned vehicle through the server. S4.

2. The host computer ROS node of the unmanned vehicle parses the gesture commands and converts the parsed string commands into ROS message format, and publishes them to the ROS topic of the host computer ROS node publishing command through the ROS node publisher; S4.

3. After receiving the ROS topic message issued by the host computer ROS node, the chassis ROS node of the autonomous driving device converts it into a bus command of the underlying controller of the autonomous driving device and sends it to the chassis through the bus to start the autonomous driving device.

7. A real-time control system for a cluster of multiple unmanned driving devices based on a data glove, as described in claim 6, is characterized in that... The specific implementation method of step S5 is as follows: after the gesture switching strategy is matched to the autonomous driving device to be controlled, the wrist is rotated through the gesture control strategy, the glove processor calculates the pitch angle and roll angle according to the gesture control strategy, and sends the gesture control command to the autonomous driving device.

Images