Limo intelligent equipment off-line voice control system and method based on edge equipment

Abstract

The invention belongs to the technical field of Internet of Things, and particularly provides a limo intelligent equipment off-line voice control system and method based on edge equipment, the system comprises intelligent home gateway equipment and microphone equipment, and the intelligent home gateway equipment serves as the edge equipment; the microphone device is installed in the smart home gateway device and is used for obtaining a user voice instruction. A voice recognition engine is also deployed on the smart home gateway equipment; the smart home gateway is also provided with an edge computing platform, the edge computing platform pre-processes the voice instruction, calls a locally installed voice recognition engine, converts the pre-processed voice into a text and converts the text into an equipment control command, and the edge computing platform sends the equipment control command to the motor home intelligent equipment. In an environment without network connection, a user can control intelligent equipment in the motor home in real time through a voice instruction, so that the response speed and autonomy of the system are greatly improved, and the system is not limited by network conditions any more.

Description

Technical Field

[0001] The present invention relates to the technical field of Internet of Things, and in particular to an offline voice control system and method for RV intelligent equipment based on edge devices. Background Art

[0002] With the development of Internet of Things technology, its related fields have also made rapid progress and development in the application of RV compartments, especially in the intelligent aspect. In the RV compartment, intelligent technology mainly relies on the terminal equipment therein. We can use the mobile phone or voice to remotely control the smart home in the RV, such as lights, curtains, etc. Among them, voice control has become one of the most frequent means for users to operate smart homes. Users can use voice to directly control the home in the RV compartment.

[0003] However, this method mostly relies on the Internet, transmitting voice data to the cloud server through the network, processing the data in the server and converting it into specific commands to return to the terminal device. Due to its special properties, RVs may go to some remote areas or mountainous areas with poor network signals. Once the network is interrupted, the voice control function will not work properly. Therefore, there is an urgent need for an intelligent terminal device control system that can realize voice control even in an offline state.

[0004] Most offline voice technologies currently rely on local devices, which need to handle all tasks on their own. Voice recognition and processing tasks usually consume a lot of resources, resulting in a shortened device battery life. In addition, offline devices rely on the limited computing power of the local device, and the efficiency of processing more complex tasks will be limited by device performance, resulting in unsatisfactory recognition accuracy and results. Summary of the invention

[0005] In order to achieve efficient voice control of intelligent terminal devices in an offline state, the present invention provides an offline voice control system and method for RV intelligent equipment based on edge devices.

[0006] In a first aspect, the technical solution of the present invention provides an offline voice control system for RV intelligent devices based on edge devices, including a smart home gateway device and a microphone device, wherein the smart home gateway device serves as an edge device; the microphone device is installed in the smart home gateway device to obtain user voice commands; A speech recognition engine is also deployed on the smart home gateway device; The smart home gateway is also equipped with an edge computing platform. After preprocessing the voice commands, the edge computing platform calls the locally installed voice recognition engine to convert the preprocessed voice into text, and converts the text into device control commands. The edge computing platform sends the device control commands to the RV's intelligent devices.

[0007] As a preferred embodiment of the technical solution of the present invention, the edge computing platform is embedded with a device service module, which sends device control instructions to the RV intelligent device through a set protocol.

[0008] As a preferred embodiment of the technical solution of the present invention, the speech recognition engine converts speech into text and converts text into device control commands, and at the same time, returns the feedback of the RV intelligent device based on the device control commands to the user in the form of natural language.

[0009] As a preferred embodiment of the technical solution of the present invention, the microphone device includes a microphone array.

[0010] As a preferred embodiment of the technical solution of the present invention, the speech recognition engine includes a DeepSpeech model trained based on a deep learning algorithm.

[0011] As a preferred embodiment of the technical solution of the present invention, the user's voice commands are obtained through the built-in microphone device of the smart home gateway device, the obtained voice commands are encoded through base64, and transmitted to the device service module of the edge computing platform in real time through the Mqtt protocol.

[0012] As a preferred embodiment of the technical solution of the present invention, the device service module of the edge computing platform receives the voice information encoding obtained from the microphone device through the Mqtt service, and sends the voice data to the core data service of the edge computing platform for preprocessing of the voice data; the preprocessed voice data is distributed to the application service, and the application service calls the installed voice recognition engine to recognize and process the voice to generate device control commands.

[0013] As a preferred embodiment of the technical solution of the present invention, the device service module receives the execution result returned by the RV intelligent device when executing the device control command, and sends the execution result to the core data service of the edge computing platform for storage.

[0014] In a second aspect, the technical solution of the present invention also provides an offline voice control method for RV intelligent equipment based on edge devices, comprising the following steps: The user's voice commands are obtained through the built-in microphone device of the smart home gateway device, the obtained voice commands are encoded through base64, and transmitted to the device service module of the edge computing platform in real time through the MQTT protocol; The device service module receives the voice information encoding obtained from the microphone device through the Mqtt service, sends the voice data to the core data service of the edge computing platform for preprocessing of the voice data; and distributes the preprocessed voice data to the application service; The application service calls the locally installed speech recognition engine, converts the pre-processed speech into text, and converts the text into device control commands and sends them to the RV's intelligent devices.

[0015] As a preferred embodiment of the technical solution of the present invention, the method further comprises: The device service module receives the execution results returned by the RV intelligent equipment when executing the device control command, and sends the execution results to the core data service of the edge computing platform for storage.

[0016] It can be seen from the above technical solutions that the present invention has the following advantages: voice processing tasks based on edge computing can be performed on edge nodes, which usually have powerful computing capabilities and can handle complex voice recognition and natural language processing tasks. Since the computing tasks are offloaded to the edge nodes, the local devices only need to handle data collection and lightweight tasks, which significantly reduces energy consumption and prolongs the battery life of the device, making it particularly suitable for low-power scenarios such as IoT devices. BRIEF DESCRIPTION OF THE DRAWINGS

[0017] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, for ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.

[0018] Figure 1 is a schematic block diagram of a system according to an embodiment of the present invention.

[0019] Figure 2 is a schematic flow chart of a method according to an embodiment of the present invention. DETAILED DESCRIPTION

[0020] In order to enable those skilled in the art to better understand the technical solutions in the present invention, the technical solutions in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work should fall within the scope of protection of the present invention.

[0021] like Figure 1 As shown, an embodiment of the present invention provides an offline voice control system for RV intelligent devices based on edge devices, including a smart home gateway device and a microphone device, wherein the smart home gateway device serves as an edge device; the microphone device is installed in the smart home gateway device to obtain user voice commands; A speech recognition engine is also deployed on the smart home gateway device; The smart home gateway is also equipped with an edge computing platform. After preprocessing the voice commands, the edge computing platform calls the locally installed voice recognition engine to convert the preprocessed voice into text, and converts the text into device control commands. The edge computing platform sends the device control commands to the RV's intelligent devices.

[0022] In the embodiment of the present invention, offline voice control is achieved by deploying the voice recognition engine on the edge device of the smart home gateway. This means that even in an environment without a network connection, users can control the intelligent devices in the RV in real time through voice commands, which greatly improves the response speed and autonomy of the system and is no longer limited by network conditions. In this application, all voice processing and recognition processes are performed on the local edge device, and the user's voice data does not need to be uploaded to the cloud server for processing, thereby effectively avoiding the risk of privacy leakage that may occur during data transmission, and enhancing the security and privacy protection of user data. The offline voice control system of the present invention does not rely on external networks or cloud services, reduces dependence on external resources, and reduces the risk of system unavailability due to network failures or cloud service interruptions. At the same time, it also avoids the additional costs that may be incurred by long-term use of cloud services, reducing overall operating costs.

[0023] In addition, the introduction of the edge computing platform enables the system to process data and make decisions locally, reducing the delay and uncertainty of data transmission and improving the stability and reliability of the system. Even in complex or unstable network environments, the normal operation of the voice control system can be ensured. It allows users to interact with intelligent devices in the RV more conveniently and naturally without manual operation or waiting for network response, which improves the convenience and comfort of use. In addition, due to the reduction of network delays and data processing time, the response to user commands is faster, further improving the user experience.

[0024] In some embodiments, the edge computing platform is embedded with a device service module, which sends device control instructions to the RV intelligent devices through a set protocol.

[0025] First, the edge computing platform needs to select or define a suitable communication protocol for data exchange with the intelligent devices in the RV. Common protocols include MQTT, Modbus, OPC UA, etc. These protocols are widely used in the field of Internet of Things (IoT). According to the specific needs and characteristics of the RV intelligent devices, the most suitable protocol is selected for communication.

[0026] In the edge computing platform, the device service module needs to be correctly configured to support the selected communication protocol. This includes setting the communication port, protocol parameters, data encryption method, etc. to ensure the security and reliability of data during transmission. At the same time, the device service module also needs to be integrated with other components in the smart home gateway device (such as the voice recognition engine) to realize the conversion and sending of voice commands to device control commands. When the edge computing platform receives text commands processed by the voice recognition engine, it converts these text commands into corresponding device control commands according to preset rules and logic. These commands need to be encapsulated into data packets that conform to the selected protocol format so that they can be correctly parsed and executed during the communication process. The device service module sends the encapsulated device control commands to the intelligent devices in the RV through the selected communication protocol. After receiving the command, the intelligent device will parse and perform the corresponding operations, such as adjusting the brightness of the light, controlling the temperature of the air conditioner, etc. At the same time, the intelligent device can also send status information or feedback data to the edge computing platform so that the user can understand the current status of the device or conduct further interaction. It should be noted that during the process of sending and receiving commands, the edge computing platform also needs to monitor the communication process to ensure the integrity and accuracy of the data. If communication anomalies or equipment failures occur, the edge computing platform needs to handle the exceptions in a timely manner, such as resending instructions and recording error logs, to ensure the stability and reliability of the system.

[0027] The device service module embedded in the edge computing platform sends device control instructions to the RV intelligent equipment through the set protocol. The specific implementation process involves multiple links such as protocol selection, module configuration, instruction generation and packaging, instruction sending and receiving, monitoring and exception handling, and security assurance. These links together constitute an efficient, reliable, and secure offline voice control system.

[0028] In some embodiments, the speech recognition engine converts speech into text and converts text into device control commands, and at the same time, returns feedback from the RV intelligent device based on the device control commands to the user in the form of natural language.

[0029] In an embodiment of the present invention, the microphone device includes a microphone array.

[0030] The speech recognition engine includes the DeepSpeech model trained based on a deep learning algorithm.

[0031] The user issues a voice command through the microphone array installed in the smart home gateway device. The user's voice signal is captured and converted into a digital signal. The collected voice signal is subjected to noise suppression to improve the signal-to-noise ratio of the voice signal. The starting and ending points of the voice signal are identified and the silent part is removed. Acoustic features such as Mel-frequency cepstral coefficients (MFCC) are extracted from the preprocessed voice signal for subsequent voice recognition. The pre-trained DeepSpeech model is loaded on the edge computing platform. The DeepSpeech model is used to perform acoustic modeling on the extracted acoustic features and predict the character probability at each time step. Combined with the language model, the output of the acoustic model is converted into a text sequence, that is, speech-to-text conversion is realized. The recognized text is semantically parsed to understand the user's intention and command content. According to the parsing results, the corresponding device control command is generated.

[0032] The device control command is encapsulated into a format that conforms to the communication protocol of the RV intelligent device. The device control command is sent to the RV intelligent device through the set communication protocol. After receiving the control command, the RV intelligent device performs the corresponding operation. After the RV intelligent device performs the operation here, it will feedback the execution result or status information to the smart home gateway device. The feedback information is converted into natural language text for the user to understand. Using speech synthesis technology, the natural language text is converted into a voice signal and played to the user through the speaker. The system enters the standby state and waits for the user's next voice command, thus forming a complete interactive cycle. Here, it should be noted that the DeepSpeech model needs to be optimized and updated regularly to improve the accuracy and efficiency of speech recognition. The edge computing platform needs to have sufficient computing power to support the real-time reasoning of the DeepSpeech model.

[0033] In some embodiments, the user's voice commands are obtained through a built-in microphone device of the smart home gateway device, the obtained voice commands are encoded through base64, and transmitted to the device service module of the edge computing platform in real time through the Mqtt protocol.

[0034] The device service module of the edge computing platform receives the voice information encoding obtained from the microphone device through the Mqtt service, and sends the voice data to the core data service of the edge computing platform for preprocessing of the voice data; the preprocessed voice data is distributed to the application service, and the application service calls the installed voice recognition engine to recognize and process the voice and generate device control commands.

[0035] The device service module receives the execution results returned by the RV intelligent equipment when executing the device control command, and sends the execution results to the core data service of the edge computing platform for storage.

[0036] The specific usage process of the system is as follows: 1) Capture voice input. Through the built-in microphone array of the gateway, we can obtain the user's voice, encode the obtained voice signal through base64, and transmit it in real time to the device service module of the edge computing platform EdgeX Foundry through the MQTT protocol. This module is the entry point for processing and managing device data.

[0037] 2) On the EdgeX Foundry platform, we receive the voice information encoding obtained from the microphone array through the Mqtt service. After the voice data reaches the EdgeX Foundry platform device service module, it will send the data to the platform's core data service. The core data service is the storage and distribution center for all device data. It receives data from device services, stores it in the database, and distributes the data to other services or applications as needed. At the core data service layer, we perform pre-processing on voice data such as noise reduction and feature extraction to improve the voice recognition rate.

[0038] 3) The edge computing platform distributes the processed voice data to the application service, which calls the installed DeepSpeech speech recognition engine service to recognize the voice and convert the voice data into text data. 4) Next, the text data needs to be processed initially, including noise removal, format unification, word segmentation, and other operations. At the same time, intent recognition is performed, intent is matched according to a predefined keyword list, and then key parameters are extracted through regular expressions, rule matching is performed, and then converted into specific commands.

[0039] 5) Before this step, the processed commands have been obtained. In the application service layer of the edge computing platform, we can convert the commands into the format specified by the protocol and send them to the specified device according to the specified protocol.

[0040] 6) Device response processing. When the intelligent device receives the corresponding control command, it will perform the corresponding operation. After the execution is completed, it will return the execution result to the device service module of the platform. After the device service module receives the execution result of the device, it will send the result to the platform core data service again. In the core data service, the result can be saved.

[0041] like Figure 2 As shown, the embodiment of the present invention also provides an offline voice control method for RV intelligent equipment based on edge devices, comprising the following steps: Step 1: Obtain the user's voice command through the built-in microphone device of the smart home gateway device, encode the obtained voice command through base64, and transmit it to the device service module of the edge computing platform in real time through the MQTT protocol; Step 2: The device service module receives the voice information encoding obtained from the microphone device through the MQTT service, sends the voice data to the core data service of the edge computing platform for preprocessing of the voice data; and distributes the preprocessed voice data to the application service; Step 3: The application service calls the locally installed speech recognition engine, converts the pre-processed speech into text, and converts the text into device control commands and sends them to the RV intelligent devices.

[0042] In some embodiments, the method further includes: a device service module receives an execution result returned by the RV intelligent device executing a device control command, and sends the execution result to a core data service of the edge computing platform for storage.

[0043] It should be noted that before executing the above method steps disclosed in this application, the following steps are also included: 1) Select edge devices: Smart home gateways are a common choice as edge devices because they are usually at the center of the home network and can communicate quickly with all smart home devices. Many modern smart gateways are already equipped with sufficient processing power, memory, and storage space to support complex edge computing tasks.

[0044] 2) Install the voice module: Install the microphone device into the gateway device. Here I can use a microphone array. The microphone array can reduce environmental noise and echo by using multiple microphones, thereby improving the clarity of the audio signal.

[0045] 3) Deployment of offline speech recognition engine: In this invention, we select DeepSpeech as the offline speech recognition engine and deploy it on the edge device (smart gateway). DeepSpeech uses a deep learning-based model to achieve high-accuracy speech recognition, supports offline speech recognition, does not rely on Internet connection, and is very suitable for use on edge devices or mobile devices. In order to better adapt to specific application scenarios or languages, you can train your own DeepSpeech model.

[0046] 4) Install the edge computing platform: In this invention, we select EdgeX Foundry as the edge computing platform. The edge computing platform can not only process and store data, but also perform complex computing tasks, thereby achieving efficient and fast voice recognition and device control in an offline environment. In the device service module of this platform, we need to subscribe to the voice data from the microphone device through the Mqtt protocol. At the same time, we also need to create an application service layer, which has two main functions. The first function is to call the locally installed voice recognition engine to convert voice into text. The second function is to convert text into device control commands and send them to smart devices through relevant protocols.

[0047] 5) Configure the local device control module: The local device control module can be embedded in the edge computing platform. This module is responsible for converting the recognized text commands into specific device control instructions and sending the commands to the specified device through the specified protocol.

[0048] It should be understood that the order of execution of the steps in the above embodiment does not necessarily mean the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiment of the present invention.

[0049] Although the present invention has been described in detail with reference to the accompanying drawings and in combination with the preferred embodiments, the present invention is not limited thereto. Without departing from the spirit and essence of the present invention, a person of ordinary skill in the art may make various equivalent modifications or substitutions to the embodiments of the present invention, and these modifications or substitutions shall be within the scope of the present invention. Any person of ordinary skill in the art may easily think of changes or substitutions within the technical scope disclosed by the present invention, and these shall be within the scope of protection of the present invention.

Claims

1. An offline voice control system for RV intelligent equipment based on edge devices, characterized in that: It includes a smart home gateway device and a microphone device, wherein the smart home gateway device serves as an edge device; the microphone device is installed in the smart home gateway device to obtain user voice commands; A speech recognition engine is also deployed on the smart home gateway device; The smart home gateway is also equipped with an edge computing platform. After preprocessing the voice commands, the edge computing platform calls the locally installed voice recognition engine to convert the preprocessed voice into text, and converts the text into device control commands. The edge computing platform sends the device control commands to the RV's intelligent devices.

2. The offline voice control system for RV intelligent equipment based on edge devices according to claim 1 is characterized in that: The edge computing platform is embedded with a device service module, which sends device control instructions to the RV intelligent devices through a set protocol.

3. The offline voice control system for RV intelligent equipment based on edge devices according to claim 2 is characterized in that: The speech recognition engine converts speech into text and converts text into device control commands. At the same time, it returns the feedback of the RV intelligent equipment based on the device control commands to the user in the form of natural language.

4. The offline voice control system for RV intelligent equipment based on edge devices according to claim 3 is characterized in that: The microphone device includes a microphone array.

5. The offline voice control system for RV intelligent equipment based on edge devices according to claim 4 is characterized in that: The speech recognition engine includes the DeepSpeech model trained based on a deep learning algorithm.

6. The offline voice control system for RV intelligent equipment based on edge devices according to claim 5 is characterized in that: The user's voice commands are obtained through the built-in microphone device of the smart home gateway device, the obtained voice commands are encoded through base64, and transmitted to the device service module of the edge computing platform in real time through the MQTT protocol.

7. The offline voice control system for RV intelligent equipment based on edge devices according to claim 6 is characterized in that: The device service module of the edge computing platform receives the voice information encoding obtained from the microphone device through the Mqtt service, and sends the voice data to the core data service of the edge computing platform for voice data preprocessing; The pre-processed voice data is distributed to the application service, and the application service calls the installed voice recognition engine to recognize and process the voice to generate device control commands.

8. The offline voice control system for RV intelligent equipment based on edge devices according to claim 7 is characterized in that: The device service module receives the execution results returned by the RV intelligent equipment when executing the device control command, and sends the execution results to the core data service of the edge computing platform for storage.

9. An offline voice control method for RV intelligent equipment based on edge devices, characterized in that: The steps include: The user's voice commands are obtained through the built-in microphone device of the smart home gateway device, the obtained voice commands are encoded through base64, and transmitted to the device service module of the edge computing platform in real time through the MQTT protocol; The device service module receives the voice information encoding obtained from the microphone device through the Mqtt service, and sends the voice data to the core data service of the edge computing platform for voice data preprocessing; Distribute the pre-processed voice data to the application service; The application service calls the locally installed speech recognition engine, converts the pre-processed speech into text, and converts the text into device control commands and sends them to the RV's intelligent devices.

10. The offline voice control method for RV intelligent equipment based on edge devices according to claim 9 is characterized in that: The method further includes: The device service module receives the execution results returned by the RV intelligent equipment when executing the device control command, and sends the execution results to the core data service of the edge computing platform for storage.

Images