Module for unified voice control and interfacing in remote control systems for unmanned vehicles

Abstract

FIELD: computer technology.SUBSTANCE: module for unified voice control and interfacing in remote control systems for unmanned vehicles, comprising an unmanned vehicle control unit (UVCU), which consists of a communication module with the remote control (CPCM), a digital processing and encoding / decoding unit for messages (DPU), and a module for transmitting information to the data processing center (DPC); the remote control consists of a series-connected receiver module (RM), a biometric data processing unit (BDPU), a voice command recognition and identification module (VCRIM), a digital signal processing and encryption unit (DSPEU), a communication module with the unmanned vehicle (CMUV), and a display.EFFECT: increasing the security of remote control of unmanned vehicles of various classes using voice commands.3 cl, 3 dwg

Description

[0001] The invention relates to the field of radio engineering and can be used to provide remote control of unmanned vehicles of various types (ground, air and water), including in conditions requiring high mobility and operational control.

[0002] A device (patent CN 209674871, G10L 15 / 22) is known for remotely controlling an unmanned aerial vehicle. The device comprises a voice receiver, a device for assigning / saving user voice commands, a speech recognition device, a remote control unit, and a wireless radio transmitter. The speech recognition device stores and recognizes incoming voice information and sends it to the control unit; the manual control unit transmits corresponding commands to the control unit; the control unit processes the voice commands received from the remote control and sends commands to the UAV via a wireless transmitter for flight control. An indicator for transmitting information about the UAV's task status serves as feedback to the operator. To protect against external noise, the device's operation is organized so that voice commands are only activated when the button is pressed.

[0003] The main disadvantages of the device are the high operator involvement in the control process, the need for manual activation of the voice mode, the limited time window for entering commands, and the difficulty of operation when performing parallel actions.

[0004] The closest analogue in technical essence to the proposed one is the device of the voice control circuit for an unmanned aerial vehicle, presented in the application CN 105957529, G10L 15 / 22, G08C 17 / 00, adopted as a prototype.

[0005] Fig. 1 shows a diagram of the prototype device, where the following is indicated:

[0006] 1.2 – wireless transmitter (wireless transmitter) (transmitter module);

[0007] 2 – remote control device for controller (remote control);

[0008] 2.3 – speech recognition device (speech command recognition and identification module);

[0009] 2.5 – manual control device (control unit for inputting information in manual mode;

[0010] 5 – unmanned aerial vehicle (UAV);

[0011] 6 – voice receiver.

[0012] The prototype device comprises a wireless transmitter 1.2, the first input-output of which has a wireless connection with the UAV 5. The second output-input of the wireless transmitter 1.2 is connected to the input-output of the remote control device controller 2, the first output-input of which is connected to the input-output of the manual control device 2.5. The second output-input of the remote control device controller 2 is connected to the first input-output of the speech recognition device 2.3, the output-input of which is connected to the input of the voice receiver 6.

[0013] The prototype device operates as follows.

[0014] The speech recognition device 2.3 stores and identifies the voice information received by the voice receiver 6 and sends the voice information to the remote control device of the controller 2.

[0015] The remote control device 2 sends the voice control information from the speech recognition device 2.3 or the manual control information set from the manual control device 2.5 to the UAV 5 via the wireless RC 1.2 and controls the flight state of the UAV 5.

[0016] The main disadvantages of the prototype device are:

[0017] – inability to switch to manual control mode (the description does not include physical elements in the form of buttons or a touch panel for transmitting commands);

[0018] – low reliability of speech recognition in high-noise environments, including the presence of wind, the presence of sound components resulting from the operation of man-made devices, etc.;

[0019] – the inability to distinguish operator voice commands from the background of dialogues / speech of strangers and the operation of media devices.

[0020] The listed shortcomings of the prototype device limit the conditions for accurate command recognition and stable control of the UAV.

[0021] In addition to the listed shortcomings of the devices selected as an analogue and prototype, they have a limited scope of application, since they relate to the field of control of unmanned aerial vehicles (UAVs) and obviously do not have the ability to interface with unmanned mechanisms (UPM) of different types, depending on their purpose.

[0022] The task is to expand the functionality of the device.

[0023] The functional capabilities of the proposed device are:

[0024] – organization of variable speech control of unmanned vehicles of various types, including ground, air and water platforms with the possibility of upgrading the control subsystem without the need for structural changes;

[0025] – ensuring the ability to synchronize control means with the required set of unmanned mechanisms;

[0026] – increasing the reliability of the operator’s voice command in conditions of high ambient noise levels and protection against accidental or intentional transmission of commands from persons who are not supposed to have access to the mechanism’s controls;

[0027] – simplification of the operator’s interface with the unmanned vehicle, increasing its mobility.

[0028] To solve the stated problem, a voice control module for remote control systems of unmanned mechanisms, comprising a transmitter unit, the output-input of which is connected to the first input-output of the remote control (RC), a voice command recognition and identification module, a manual information input control unit (MICU), as well as an unmanned mechanism (UPM), according to the invention, are introduced

[0029] unmanned mechanism control unit (UPMU), the first output-input of which is connected to the input-output of the module for interaction with on-board functional elements of unmanned mechanisms (MOFEM) located on the UMU;

[0030] the UBPM unit consists of a module for communication with the remote control (MCPDU), the first output-input of which is connected to the first input-output of the digital processing and encoding / decoding unit of messages (DPU), the second output-input of which is connected to the input-output of the decision device, the first output of which is connected to the first input of the module for transmitting information to the data processing center (MPI), the output of which is the first output of the UBPM, wherein the first output of the unit for monitoring the state of the environment and on-board subsystems of the UBPM (BK) is connected to the input of the DPU, the second output of the BC is connected to the first input of the decision device, the third output of the BC is connected to the second input of the MPI, the output-input of the BC is the first output-input of the UBPM and is connected to the input-output of the MVBPM; the second output of the decision device 3.5 is the second output of the UBPM and is connected to the input of the MVBPM, in addition, the first output of the module for receiving and processing navigation information (MPONI) is connected to the input of the MSPDU, the second output of MPONI is connected to the second input of the decision device, the third output of MPONI is connected to the third input of the MPI;

[0031] the second output-input of the MSPDU is the second output-input of the UBPM;

[0032] The MPONI input is the input of the UBPM block for global navigation satellite system (GNSS) signals;

[0033] the remote control consists of a series-connected receiver module (RM), a biometric data processing unit (BDPU), a voice command recognition and identification module (SCRIM), a digital signal processing and encryption unit (DSPEU), an unmanned vehicle communication module (UVM) and a display, the output of which is the RC output, while the output of the keyboard panel (KPL) is connected to the input of the manual mode information input control unit (MIICU), the output of which is connected to the second input of the DSPE; the output of the condition monitoring subsystem (CMS) is connected to the second input of the display, in addition, the RM input-output is the first RC input-output, connected to the microphone output-input placed in a shock-resistant housing with fastening elements and consisting of a series-connected audio signal-to-radio signal conversion unit (ASRC) and the PRD unit, the output-input of which is the microphone input-output;

[0034] MSBM output-input is the second output-input of the remote control and is connected to the second input-output of the UBPM unit.

[0035] The following designations are introduced in Figures 2 and 3:

[0036] 1 – microphone (M);

[0037] 1.1 – audio-to-radio signal conversion unit (ASRCU);

[0038] 1.2 – transmitter unit (TU);

[0039] 2 – remote control;

[0040] 2.1 – receiver module (MPrm);

[0041] 2.2 – biometric data processing unit (BDNU);

[0042] 2.3 – voice command recognition and identification module (SCIM);

[0043] 2.4 – keyboard panel (PKL);

[0044] 2.5 – manual input control unit (MICU);

[0045] 2.6 – digital signal processing and encryption unit (DSPEU);

[0046] 2.7 – unmanned vehicle communication module (UMCM);

[0047] 2.8 – condition control subsystem (CCS);

[0048] 2.9 – display;

[0049] 3 – unmanned mechanism control unit (UPMU);

[0050] 3.1 – communication module with the remote control (MSPDU);

[0051] 3.2 – digital processing and encoding / decoding unit (DPU);

[0052] 3.3 – module for receiving and processing navigation information (MPONI);

[0053] 3.4 – environmental control unit and on-board subsystems of the BPM (BC);

[0054] 3.5 – decision device (DD);

[0055] 3.6 – module for transmitting information to the data processing center (MPI);

[0056] 4 – module for interaction with on-board functional elements of unmanned vehicles (MVBPM);

[0057] 5 – unmanned vehicle (including UAV).

[0058] The claimed device comprises a microphone 1, the input / output of which is connected to the input / output of a remote control 2, the input / output of which is connected to a data processing unit 5, the output of which is the output of signals sent to the data processing center. The output of the remote control 2 is the output of signals to the data processing unit operator.

[0059] In this case, the microphone 1 of the “lavatory” type with a built-in power source (for example, a compact Li-ion battery), is placed in a shock-resistant housing with fastening elements on the operator’s clothing (not shown in Fig. 2 and 3) and includes a series-connected unit for converting an audio signal into a radio signal 1.1 and a transmitter 1.2, the output-input of which is connected to the input-output of the remote control 2.

[0060] The remote control 2 is placed in a housing made of impact-resistant material with elements for fastening on the operator's forearm (not shown in Figs. 2 and 3) and contains a series-connected remote control receiver module 2.1, a biometric data processing unit 2.2, a voice command recognition and identification module 2.3, a digital signal processing and encryption unit 2.6, a module for communicating with an unmanned mechanism 2.7, the output of which is connected to the input of the display 2.9, the output of which is the output of the remote control 2. In this case, the keyboard panel 2.4 is connected to the input of the manual mode information input control unit 2.5, the output of which is connected to the second input of the digital signal processing and encryption unit 2.6. The output of the state control subsystem 2.8 is connected to the second input of the display 2.9. The input of MPrd 2.1 is the input of PDU 2. In addition, the output-input of MSBM 2.7 is connected to the input-output of the control unit of the unmanned mechanism 3.

[0061] A Li-Po battery or lithium primary batteries (not shown in Fig. 3) can be used as a power source for remote control 2.

[0062] The UBPM 3 unit performs all the necessary functions for controlling the on-board subsystems and drives of the BPM 5 in accordance with the information received from the module for interaction with the on-board functional elements 4. The UBPM 3 unit and module 4 are integral parts of the unmanned mechanism of the BPM 5 itself (integrated into its body).

[0063] The UBPM 3 unit contains a module for communication with the remote control unit 3.1, the first input / output of which is the input / output of the UBPM 3 unit. The second output / input of the MSPDU 3.1 is connected to the first input / output of the digital processing and encoding / decoding unit of messages 3.2, the second output / input of which is connected to the input / output of the decision device 3.5, the first output of which is connected to the first input of the module for transmitting information to the data processing center 3.6, the output of which is the output of the UBPM 3. The second output of the decision device 3.5 is connected to the input of the module 4, the output / input of which is connected to the input / output of the BC 3.4, the first output of which is connected to the input of the DCO 3.2. The second output of the BC 3.4 is connected to the first input of the RU 3.5. The third output of the BC 3.4 is connected to the second input of the MPI 3.6. In addition, the first output of MPONI 3.3 is connected to the input of MSPDU 3.1. The second output of MPONI 3.3 is connected to the second input of RU 3.5. The third output of MPONI 3.3 is connected to the third input of MPI 3.6.

[0064] To ensure the smooth operation of all subsystems, regardless of the conditions of use, microphone 1, remote control 2, and BPM 5 are waterproof (with a protection level of at least IP54 according to IEC 60529). To ensure a minimum level of noise components in the received signal, microphone 1 is housed in a windproof capsule (e.g., in the form of an acoustic membrane or a protective fleece cover) and is equipped with the necessary set of acoustic elements and signal processing tools to implement active noise reduction algorithms (e.g., differential signal processing).

[0065] Data transfer between microphone 1 and remote control 2, as well as data exchange between remote control 2 and UBPM control unit 3, is accomplished via wireless communication channels. For this purpose, communication between operator microphone 1, remote control 2, and UBPM control unit 3 is organized via Bluetooth channels, local WiFi networks, or other radio channels. The selection of these data transfer technologies, taking into account the expected range, can ensure separation of the required number of subscribers during simultaneous operation of their devices in close proximity. To ensure the specified level of transmission reliability, noise immunity, and high-speed communication, signal processing is implemented based on a combination of digital algorithms (block codes, convolutional codes, cascaded and turbo coding, etc.) with various types of modulation schemes (angle keying, including Gold and Kasami sequences, signals with modulation on subcarrier frequencies, etc.) or the use of noise-like signals.Thus, the choice of specific algorithms is determined by operating conditions.

[0066] Communication between UPPM 3 and Module 4 is carried out via a wired channel. Information exchange between them is implemented according to the following algorithm: Module 4 collects information from node elements (e.g., various types of sensors, indicators, detectors, etc.), generates a service data packet, and transmits it to UPPM 3 via a specified interface type, such as SPI, I2C, or another, in accordance with the applicable exchange protocol.

[0067] This system structure allows for the functional separation of algorithms and operations required to ensure the stable operation of the control radio channel and the functioning of the on-board subsystems of the unmanned mechanism 5, which makes it possible to implement the UBPM block 3 and module 4 as independent subsystems.

[0068] Separate implementation of the specified subsystems, firstly, provides the ability to modify the hardware of the BPM 5 without making changes to the hardware of the PDU 2, while the adaptation of the system is carried out through the optimization of software algorithms, including the expansion of the command base and the replacement of control scenarios.

[0069] Secondly, it provides the ability to pair the remote control 2 with several unmanned vehicles of various types, since the set of supported voice commands is implemented at the software level and does not require the introduction of additional hardware for manual control.

[0070] The proposed device operates as follows.

[0071] The information received from microphone 1, after preliminary processing in the audio-to-radio signal conversion unit 1.1 and transmitter unit 1.2, is fed to the input of the remote control receiver module 2.1. The subsequent biometric data processing unit 2.2 is implemented, firstly, to reliably transmit operator commands to the UBPM 3 in conditions of high ambient noise, including speech fragments and voice interference, and, secondly, to prohibit the transmission of commands from persons who do not have access to the control system. The working algorithms of 2.2 can be based on, for example, a correlation analysis scheme. To account for natural changes in the operator's voice, UBPM 2.2 constantly monitors the presence and degree of possible voice changes (the data is not constant, but is updated during the operator's work). The result of the operation of biometric data processing unit 2.2 is the formation of the operator's clearance or prohibition for further processing.Voice command recognition and identification module 2.3 is capable of processing the voice signal after operator access confirmation (UDB 2.2) and is designed to determine the content of the voice command. Module 2.3 consists of an element storing a set of executable commands and devices that compare the received command with the set of available commands. As a result of this processing (implemented, for example, based on the maximum likelihood method, correlation analysis, or other statistical analysis methods with an adaptive decision threshold), a decision is made on the type of the transmitted command. The presence of a manual input command via the physical keyboard panel 2.4 (can be represented as physical buttons or a callable touchpad) is monitored by the manual input control unit 2.5; if present, this operation is performed. The signal is then transmitted to the digital signal processing and encryption unit 2.6.The communication module with the unmanned mechanism 2.7 exchanges information with the UBPM unit 3.

[0072] The 2.8 remote control status monitoring subsystem is provided for monitoring the status of the control subsystem parameters. Display 2.9 is used to display messages and video signals to the control operator about the status of the current operation and to provide a keyboard if its touch mode is selected.

[0073] The UBPM unit 3 receives commands from the remote control 2 using the communication module 3.1 and decrypts them using the message processing and encoding / decoding unit 3.2. The decision maker 3.5, based on the data received from the message processing and encoding / decoding unit 3.2, the navigation information reception and processing module 3.3 and the environmental and on-board subsystems monitoring unit 3.4, makes a decision on the possibility of its execution and transmits it to module 4. The signal from the outputs of the decision maker 3.5, the navigation information reception and processing module 3.3 and the environmental and on-board subsystems monitoring unit 3.4 is transmitted by the information transmission module to the data processing center 3.6 via a parallel communication channel (via cellular networks). The collection of such information makes it possible to determine the location of the UAV 5 and the operator and obtain up-to-date data on its condition, which is important for drones used for rescue or industrial purposes.The return channel of the communication module 3.1 simultaneously transmits data from the module for receiving and processing navigation information 3.3 with the encoded data from the unit for processing and encoding / decoding messages 3.2 (received in turn from the environmental control unit and the on-board subsystems of the BPM 3.4 and RU 3.5).

[0074] Let's give an example of the operation of the proposed device.

[0075] The initial system startup is performed through a sequential sequence of actions, which are embedded in software algorithms. After remote control 2 is turned on, the system requests authorization data (login and password pairs), which are entered through the physical panel of keypad 2.4. A request is made to turn on microphone 1 and pair it with remote control 2. Using the signal received from microphone 1, the system determines the level of background noise components to determine whether voice scanning is possible under the given conditions.

[0076] Using display 2.9, a dialogue with the user is initiated in the form of questions. The purpose of this procedure is to obtain the necessary statistics for processing the user's voice data. If the system lacks data for a certain group of phonemes, the algorithm may request the reading of certain reserved phrases. This process processes the speech signal (analyzing the spectral composition of the voice when pronouncing various phonemes and their combinations, determining the characteristic level of amplitude characteristics, etc.). The data obtained as a result of this processing is transmitted to biometric data processing unit 2.2. At the final stage, the system reports successful user registration and readiness to connect to the UBPM unit 3.

[0077] When BPM 5 starts, a protocol check is initiated between its UBPM 3 unit and module 4. If communication is successful, UBPM 3 unit, via communication module 3.1, transmits information to remote control 2 regarding successful pairing and readiness to execute user commands. The corresponding information is also displayed on display 2.9.

[0078] After initial authorization, the connection between Remote Control 2 and Microphone 1 is established automatically when they are turned on. The communication protocols between the UBPM 3 unit and Module 4 are checked automatically when the BPM 5 is turned on. The operator dons Microphone 1, which is attached to clothing, such as a collar, while Remote Control 2 is secured to the forearm. The system then checks the battery level. Display 2.9 of Remote Control 2 displays basic system status information: battery level, coordinates, BPM 5 speed, and connection status with navigation services. The operator gives a voice command, activating recognition using a marker phrase, such as "Start." Microphone 1 captures the voice and, after preliminary noise processing, transmits it to Remote Control 2.

[0079] Remote control unit 2 identifies the operator using biometric data processing unit 2.2 and recognizes the command using voice command recognition and identification module 2.3. After processing the command, the signal is transmitted to the unmanned vehicle to complete the task.

[0080] The UBPM 3 unit receives a command via a wireless communication channel. The system analyzes the surrounding conditions using the environmental control unit and the onboard subsystems of the BPM 3.4, such as weather conditions, interference, and the presence of foreign objects, to verify the ability to execute commands. If the check is successful, the command is executed based on the current communication protocol from module 4. The control panel analyzes the presence of a manual input command using the manual input control unit 2.5 and, if necessary, cancels or modifies previous commands.

[0081] Remote Control Panel 2 displays the current operating parameters of the BPM 5 on display 2.9: coordinates, speed, battery charge data, and other readings, including environmental parameters. Feedback to the operator is provided via indicators on Remote Control Panel 2, allowing the operator to monitor the progress of tasks.

[0082] After completing the current task with the first BPM 5, the operator initiates the control session termination procedure via the remote control interface 2. Information about the completion of the current operation is recorded by the remote control status control subsystem 2.8 and displayed on display 2.9.

[0083] Then the operator selects another BPM 5. The selection can be made either via the keyboard panel 2.4 or via a voice command recognized by the voice command recognition and identification module 2.3 after confirmation of the operator’s access by the biometric data processing unit 2.2.

[0084] After selecting a new UAV 5, the remote control unit 2 pairs with its UBPM unit 3 via the unmanned vehicle communication module 2.7 and the remote control communication module 3.1. In this case, repeated biometric registration of the operator is not required, since previously generated biometric data is used by the biometric data processing unit 2.2 to confirm the operator's identity during subsequent control sessions.

[0085] After establishing a connection, the operator generates control commands for the second BPM 5 in the same manner as described above, while the processing, protection and transmission of commands is carried out using the digital signal processing and encryption unit 2.6 and the corresponding modules of the BPM control unit 3.

[0086] Note that the described example of controlling various types of UMP 5 is achieved through the unified architecture of the remote control panel 2 and the ability to configure the parameters for interaction with a specific UMP 5. When pairing the remote control panel 2 with a specific UMP 5, a corresponding control profile is generated in communication module 2.7, the UBPM unit 3, and module 4, taking into account the type of UMP 5, its functionality, the composition of its onboard subsystems, and the permissible operating modes. This ensures the versatility and expandability of the system without changing the remote control hardware.

[0087] Depending on the selected control profile, command processing in the corresponding nodes and modules is carried out taking into account the characteristics of the specific controlled unmanned vehicle, including differences in movement dynamics, navigation parameters and available actuators.

[0088] Thus, one remote control provides sequential pairing and control of several unmanned vehicles of different types and purposes without the need to make changes to the hardware of the remote control and without re-configuring the operator's biometric parameters.

[0089] The proposed technical solution for implementing voice control enables the transmission of recognized commands to the unmanned vehicle without the need for manual input. The use of separate wireless channels, autonomous power sources, and a remote microphone module reduces command transmission time and the likelihood of erroneous command transmission by the operator, eliminating the need for constant control monitoring and expanding the system's functionality. Combining the voice interface with manual control ensures increased command transmission reliability, ease of use, and system configuration flexibility.

[0090] This solution is particularly suitable for applications in low visibility conditions, industrial monitoring, search and rescue operations, and robotic systems where mobility and efficiency are required in human-machine interaction.

[0091] The technical result consists in providing secure remote control of unmanned vehicles of various classes, including ground, air and water platforms using voice commands, increasing the reliability and ease of control due to biometric identification of the operator, as well as the ability to pair one remote control with several UAVs without additional physical replacement of control elements.

Claims

1. A module for unified voice control and interfacing in remote control systems for unmanned vehicles, comprising a transmitter unit, the input / output of which is connected to the first input / output of a remote control (RC), a voice command recognition and identification module, a manual input control unit (MICU), and also an unmanned vehicle (UVM), characterized in that the input an unmanned mechanism control unit (UPMU), the first output-input of which is connected to the input-output of the module for interaction with the on-board functional elements of unmanned mechanisms (MOFEM) located on the UMU; the UBPM block consists of a module for communication with the remote control (MSPDU), the first output-input of which is connected to the first input-output of the digital processing and encoding / decoding unit of messages (DPU), the second output-input of which is connected to the input-output of the decision device, the first output of which is connected to the first input of the module for transmitting information to the data processing center (MPI), the output of which is the first output of the UBPM, wherein the first output of the environmental control unit and on-board subsystems of the BPM (BC) is connected to the input of the UCO, the second output of the BC is connected to the first input of the decision device, the third output of the BC is connected to the second input of the MPI, the output-input of the BC is the first output-input of the UBPM and is connected to the input-output of the MVBPM; the second output of the decision device 3.5 is the second output of the UBPM and is connected to the input of the MVBPM, in addition, the first output of the module for receiving and processing navigation information (MRIPI) is connected to the input of the MSPDU, the second output of MRIPI is connected to the second input of the decision device, the third output of MRIPI is connected to the third input of the MPI; the second output-input of the MSPDU is the second output-input of the UBPM; The MPONI input is the input of the UBPM block for global navigation satellite system (GNSS) signals; the remote control consists of a series-connected receiver module (RM), a biometric data processing unit (BDPU), a voice command recognition and identification module (SCRIM), a digital signal processing and encryption unit (DSEU), a module for communication with the unmanned mechanism (MSBM) and a display, the output of which is the RC output, wherein the output of the keyboard panel (KPL) is connected to the input of the manual mode information input control unit (MIICU), the output of which is connected to the second input of the DSEU; the output of the condition control subsystem (CCS) is connected to the second input of the display, in addition, the RM input-output is the first RC input-output, connected to the microphone output-input placed in a shock-resistant housing with fastening elements and consisting of a series-connected audio signal-to-radio signal conversion unit (ASRC) and a Transmitter unit, the output-input of which is the microphone input-output; The output-input of the MSBM is the second output-input of the remote control and is connected to the second input-output of the UBPM block.

2. The module according to paragraph 1, characterized in that the microphone is made in the form of a lavalier, placed in a windproof capsule and equipped with active noise reduction means.

3. The module according to paragraph 1, characterized in that the housings of the microphone, remote control and unmanned mechanism are made moisture-proof, with a protection level of at least IP54.

Images