Bluetooth mesh-based offline voice recognition control method and system

Abstract

The embodiment of the invention discloses an off-line voice recognition control method and system based on Bluetooth mesh. The method comprises the steps that voice signals are collected and preprocessed; recognizing the preprocessed signal by using a voice recognition model which is locally deployed in advance, and converting the audio file into a text instruction; performing analysis and semantic understanding on the text instruction, and extracting control information to generate a corresponding control instruction; and establishing a Bluetooth mesh network, sending the control instruction to a Bluetooth mesh receiver of a target device, and after receiving the control instruction, calling a corresponding control interface and a driving program by the target device to perform control operation on a corresponding intelligent device. According to the embodiment of the invention, accurate voice control of a plurality of intelligent devices can be realized without network connection, large-scale networking and flexible expansion of the devices are realized by using the advantages of the Bluetooth mesh technology, the reliability and stability of the system are improved, and the privacy security of users is guaranteed.

Description

Technical Field

[0001] The embodiments of the present invention relate to the technical field of speech recognition and intelligent control, and in particular to an offline speech recognition control method and system based on Bluetooth mesh. Background Art

[0002] With the rapid development of Internet of Things technology, smart devices have been widely used in people's lives and work. As an important way of human-computer interaction, voice recognition provides great convenience for users to operate smart devices. However, most voice recognition control systems currently rely on network connections, uploading voice data to the cloud for processing, and then receiving control instructions returned from the cloud to achieve device control. This online voice recognition method has many disadvantages:

[0003] Network dependency: When the network signal is unstable, interrupted, or in a networkless environment, the voice recognition function cannot be used normally, resulting in device control failure. For example, in remote areas, underground parking lots, or indoor places with weak signals, users may not be able to control smart devices smoothly through voice commands.

[0004] Privacy and security risks: When voice data is transmitted to the cloud, there is a risk of being stolen, tampered with, or leaked, and the privacy and security of users cannot be effectively protected. In particular, for some voice commands involving personal sensitive information, such as control commands for home security systems, once privacy is leaked, it may bring serious security risks to users.

[0005] Response delay: Since voice data needs to go through multiple links such as uploading, cloud processing, and downloading control instructions, there will be a certain response delay. This will affect the user experience in control scenarios with high real-time requirements, such as the rapid switching of smart lights and the instant opening and closing of smart curtains.

[0006] Offline speech recognition is to complete speech recognition and command processing on the local device without relying on network connection. It has the advantages of fast response speed and privacy security. However, the existing offline speech recognition system has shortcomings in interoperability and scalability between devices. Traditional Bluetooth technology is limited in the number of connected devices and has a small communication range, which makes it difficult to meet the needs of large-scale smart device networking. Summary of the invention

[0007] To this end, an embodiment of the present invention provides an offline voice recognition control method and system based on Bluetooth mesh to solve the problems that most current voice recognition control systems rely on network connections, offline voice recognition systems are insufficient in interoperability and scalability between devices, the number of connected devices is limited, and the communication range is small.

[0008] In order to achieve the above purpose, the embodiment of the present invention provides the following technical solutions:

[0009] According to a first aspect of an embodiment of the present invention, there is provided an offline speech recognition control method based on Bluetooth mesh, the method comprising:

[0010] S1, collecting a first voice signal from a user, and preprocessing the collected first voice signal to form a second voice signal;

[0011] S2. Recognize the second voice signal using a pre-deployed local voice recognition model, and convert the audio file into a text instruction;

[0012] S3, obtaining the text instruction and parsing and semantically understanding the text instruction, extracting control information and generating corresponding control instructions according to preset control logic;

[0013] S4. Establish a Bluetooth mesh network between various Bluetooth mesh devices, send the control instruction to the Bluetooth mesh receiver of the target device, and after receiving the control instruction, the target device calls the corresponding control interface and driver to control the corresponding smart device.

[0014] Further, the preprocessing includes noise reduction, echo removal and speech enhancement;

[0015] The noise reduction is achieved by adjusting the filter parameters in real time according to the changes in the ambient noise through an adaptive filtering algorithm.

[0016] Further, the second voice signal is recognized by using a pre-locally deployed voice recognition model to convert the audio file into a text instruction, including:

[0017] Deploy the speech recognition model locally and obtain a large amount of audio training data;

[0018] Using a deep learning algorithm and audio training data to train the speech recognition model, adjusting the model structure and parameters, and obtaining a trained speech recognition model;

[0019] The second voice signal is recognized using the trained voice recognition model, and the audio file is converted into text instructions.

[0020] Furthermore, the audio training data includes audio data of different accents, speaking speeds, and language habits, as well as various common control instruction data.

[0021] Furthermore, a Bluetooth mesh network is established between each Bluetooth mesh device, including:

[0022] The Bluetooth mesh protocol stack is used to realize multi-hop communication and network self-organization between various devices. The Bluetooth mesh networking is a distributed networking.

[0023] Furthermore, the method further comprises:

[0024] When a new device needs to be added, the Bluetooth mesh unit of the new device is added to the Bluetooth mesh network, and other devices in the Bluetooth mesh network bring the new device into the control range.

[0025] Furthermore, the method further comprises:

[0026] Obtain the control status and operation information of the target device in real time;

[0027] The control status and operation information are fed back to the user terminal in real time.

[0028] Furthermore, the method further comprises:

[0029] Acquire usage data from users and regularly collect new audio training data to update and optimize the speech recognition model, and regularly inspect and maintain each device.

[0030] Furthermore, the method also includes: when a device fails or a communication connection is abnormal, an alarm is sent to a user terminal to remind the user to handle it.

[0031] According to a second aspect of an embodiment of the present invention, there is provided an offline speech recognition control system based on Bluetooth mesh, the system comprising:

[0032] A voice collection and preprocessing module, used for collecting a first voice signal from a user, and preprocessing the collected first voice signal to form a second voice signal;

[0033] An offline speech recognition engine, configured to recognize the second speech signal using a pre-locally deployed speech recognition model, and convert the audio file into a text instruction;

[0034] An instruction parsing and processing module, used to obtain the text instruction and parse and semantically understand the text instruction, extract control information and generate corresponding control instructions according to preset control logic;

[0035] A Bluetooth mesh communication module, used to establish a Bluetooth mesh network between various Bluetooth mesh devices and send the control command to the Bluetooth mesh receiver of the target device;

[0036] The device control module is used for the target device to call the corresponding control interface and driver to control the corresponding intelligent device after receiving the control instruction.

[0037] The embodiments of the present invention have the following advantages:

[0038] The embodiment of the present invention includes: S1, collecting a first voice signal from a user, and preprocessing the collected first voice signal to form a second voice signal; S2, using a pre-locally deployed voice recognition model to recognize the second voice signal and convert the audio file into a text instruction; S3, obtaining the text instruction and parsing and semantically understanding the text instruction, extracting control information and generating corresponding control instructions according to a preset control logic; S4, establishing a Bluetooth mesh network between various Bluetooth mesh devices, sending the control instruction to the Bluetooth mesh receiver of the target device, and the target device calls the corresponding control interface and driver to control the corresponding smart device after receiving the control instruction. The embodiment of the present invention can realize accurate voice control of multiple smart devices without network connection, and at the same time, utilize the advantages of Bluetooth mesh technology to realize large-scale networking and flexible expansion of devices, improve the reliability and stability of the system, and protect the privacy and security of users. BRIEF DESCRIPTION OF THE DRAWINGS

[0039] In order to more clearly illustrate the implementation methods of the present invention or the technical solutions in the prior art, the following briefly introduces the drawings required for the implementation methods or the description of the prior art. Obviously, the drawings in the following description are only exemplary, and for ordinary technicians in this field, other implementation drawings can be derived from the provided drawings without creative work.

[0040] The structures, proportions, sizes, etc. illustrated in this specification are only used to match the contents disclosed in the specification so as to facilitate understanding and reading by persons familiar with the technology. They are not used to limit the conditions under which the present invention can be implemented, and therefore have no substantial technical significance. Any structural modification, change in proportion or adjustment of size shall still fall within the scope of the technical contents disclosed in the present invention without affecting the effects and purposes that can be achieved by the present invention.

[0041] Figure 1 A schematic diagram of the logical structure of an offline speech recognition control system based on Bluetooth mesh provided by an embodiment of the present invention;

[0042] Figure 2 A flowchart of an offline speech recognition control method based on Bluetooth mesh is provided in an embodiment of the present invention. DETAILED DESCRIPTION

[0043] The following is a description of the implementation of the present invention by specific embodiments. People familiar with the art can easily understand other advantages and effects of the present invention from the contents disclosed in this specification. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0044] Offline speech recognition is to complete speech recognition and command processing on the local device without relying on network connection. It has the advantages of fast response speed and privacy security. However, the existing offline speech recognition system has shortcomings in interoperability and scalability between devices. Traditional Bluetooth technology is limited in the number of connected devices and has a small communication range, which makes it difficult to meet the needs of large-scale smart device networking.

[0045] How to combine Bluetooth mesh technology with offline speech recognition technology to develop an offline speech recognition control system based on Bluetooth mesh to achieve efficient voice control of devices in a network-free environment, large-scale networking and good scalability has become an urgent problem to be solved in the current intelligent control field.

[0046] In order to solve the problem that the above-mentioned online speech recognition control system relies on network connection, the offline speech recognition system has deficiencies in interoperability and scalability between devices, the number of connected devices is limited, and the communication range is small.

[0047] refer to Figure 1 The embodiment of the present invention discloses an offline speech recognition control system based on Bluetooth mesh, which includes: a speech recognition unit 101; a speech collection and preprocessing module 102; an offline speech recognition engine 103; an instruction parsing and processing module 104; a Bluetooth mesh communication module 105; and a device control module.

[0048] The voice collection and preprocessing module 102 is responsible for collecting the user's voice signal and preprocessing the collected voice signal, including noise reduction, echo removal, voice enhancement and other operations, to improve the quality of the voice signal and provide more accurate data for subsequent voice recognition. For example, it adopts advanced adaptive filtering algorithms to adjust the filter parameters in real time according to the changes in environmental noise to effectively remove the interference of background noise; it uses echo cancellation technology to eliminate echoes caused by equipment reflection and other reasons to ensure the clarity of the voice signal.

[0049] The offline speech recognition engine 103 uses a locally deployed speech recognition model to recognize the preprocessed speech signal and convert the speech into text instructions. The speech recognition model is trained based on a deep learning algorithm, has a high recognition accuracy and generalization ability, and can recognize a variety of natural language instructions. By continuously optimizing the model structure and parameters, the model's adaptability to different accents, speaking speeds, and language habits is improved, ensuring that the user's voice instructions can be accurately recognized in various complex environments.

[0050] The instruction parsing and processing module 104 receives the text instruction output by the offline speech recognition engine, parses and understands the text, extracts key control information, and generates corresponding control instructions according to the preset control logic. For example, when the user says the voice instruction "turn on the living room light", the module can accurately parse key information such as "turn on" and "living room light" and generate corresponding light control instructions.

[0051] The Bluetooth mesh communication module 105 is responsible for communicating with other Bluetooth mesh devices and sending the control instructions generated by the instruction parsing and processing module to the target device. This module uses the Bluetooth mesh protocol stack to achieve multi-hop communication and network self-organization between devices. Through the Bluetooth mesh network, control instructions can be quickly and reliably transmitted to each device in the network, achieving unified control of different areas and different types of devices.

[0052] The device control module includes: smart switch 401, smart curtain 301, smart air conditioner 201 and other control devices. Each control device is equipped with a corresponding Bluetooth mesh communication module. The smart switch 401 corresponds to the switch Bluetooth mesh communication module 403; the smart curtain 301 corresponds to the curtain Bluetooth mesh communication module 303; the smart air conditioner 201 corresponds to the air conditioner Bluetooth mesh communication module 203.

[0053] The device control module receives control commands from the Bluetooth mesh communication module and controls the corresponding smart devices to realize functions such as turning the devices on and off and adjusting parameters. For different types of devices, such as smart lights, smart curtains, and smart home appliances, the module has corresponding control interfaces and drivers to ensure that control commands can be executed accurately.

[0054] The smart switch 401 also includes a light control switch 402 , the smart curtain 301 also includes a curtain module 302 , and the smart air conditioner 201 also includes an air conditioner module 202 .

[0055] Furthermore, the system proposed in the embodiment of the present invention also includes a mobile phone configuration unit 501, which specifically includes a voice command editing 502, a voice command generation 503, and a mobile phone Bluetooth mesh communication module 504.

[0056] Each device in the system forms a distributed network through Bluetooth mesh technology. Each device can act as a control node to receive and process the user's voice commands, or as a transfer node to help other devices transmit data. When a new device needs to be added, just add the new device to the Bluetooth mesh network, and the system will automatically identify it and bring it into the control range, realizing seamless expansion of the system.

[0057] For example, in a smart home scenario, users can add new smart lamps, smart sockets and other devices at any time, and control the new devices by voice without rewiring or performing complex configurations.

[0058] Optional user interaction interface, such as mobile phone APP or smart control panel. Users can view device status, set voice recognition parameters, manage Bluetooth mesh network, etc. through the interactive interface. The interactive interface adopts a simple and intuitive design to facilitate user operation and improve user experience. For example, on the mobile phone APP, users can see the switch status and operating parameters of each smart device in the home at a glance, and can also remotely control the device by sliding, clicking, etc. At the same time, users can set voice wake-up words and adjust voice recognition sensitivity on the APP to meet personalized needs.

[0059] Corresponding to the above-disclosed offline voice recognition control system based on Bluetooth mesh, an embodiment of the present invention further discloses an offline voice recognition control method based on Bluetooth mesh. The following describes in detail an offline voice recognition control method based on Bluetooth mesh disclosed in an embodiment of the present invention in combination with the above-described offline voice recognition control system based on Bluetooth mesh.

[0060] Install hardware devices that integrate Bluetooth mesh communication modules and device control modules on each smart device to ensure that the device can work properly and communicate with other devices. For voice collection devices, such as smart speakers or voice remote controls, install voice collection and preprocessing modules and perform corresponding initialization settings, including setting voice wake-up words and adjusting voice collection sensitivity. Place all devices within the coverage of the Bluetooth mesh network, and the devices will automatically search and connect to the network to complete the networking process.

[0061] refer to Figure 2The present invention discloses an offline voice recognition control method based on Bluetooth mesh, which is used to realize efficient, convenient and stable voice control of various devices. The method comprises: S1, collecting a first voice signal from a user, and preprocessing the collected first voice signal to form a second voice signal; S2, using a pre-locally deployed voice recognition model to recognize the second voice signal, and converting the audio file into a text instruction; S3, obtaining the text instruction and parsing and semantically understanding the text instruction, extracting control information and generating corresponding control instructions according to a preset control logic; S4, establishing a Bluetooth mesh network between various Bluetooth mesh devices, sending the control instruction to the Bluetooth mesh receiver of the target device, and the target device calls the corresponding control interface and driver after receiving the control instruction to control the corresponding smart device.

[0062] The user issues a voice command, and the microphone of the voice acquisition device collects the voice signal and transmits it to the voice acquisition and preprocessing module for preprocessing. The preprocessed voice signal is sent to the offline voice recognition engine, which recognizes the voice signal and converts it into a text command. The command parsing and processing module receives the text command, parses and understands the semantics, and generates the corresponding control command. The control command is sent to the device control module of the target device through the Bluetooth mesh communication module, and the device control module controls the device according to the command.

[0063] If the system is configured with a user interaction interface, the control status and operation information of the device will be fed back to the interaction interface in real time, making it convenient for users to view and manage.

[0064] During the system development phase, a large amount of voice data is collected, covering different accents, speaking speeds, language habits, and various common control commands. The offline voice recognition model is trained using a deep learning algorithm to optimize the model's parameters and structure and improve recognition accuracy. During system use, new voice data can be regularly collected based on actual user usage and feedback, and the model can be updated and optimized to adapt to the ever-changing voice environment and user needs. For example, when a user finds that the recognition accuracy of certain specific commands is low, the system can be used to upload the voice data to the server for analysis and training through the feedback mechanism provided by the system, and then the updated model can be downloaded to the device to improve the system's recognition performance.

[0065] Furthermore, the preprocessing includes noise reduction, echo removal and speech enhancement; the noise reduction is to adjust the filter parameters in real time according to the changes of environmental noise through an adaptive filtering algorithm.

[0066] Furthermore, the second voice signal is recognized by using a pre-locally deployed voice recognition model, and the audio file is converted into text instructions, including: deploying the voice recognition model locally and obtaining a large amount of audio training data; training the voice recognition model using a deep learning algorithm and audio training data, adjusting the model structure and parameters, and obtaining a trained voice recognition model; and recognizing the second voice signal using the trained voice recognition model, and converting the audio file into text instructions.

[0067] Furthermore, the audio training data includes audio data of different accents, speaking speeds, and language habits, as well as various common control instruction data.

[0068] Furthermore, a Bluetooth mesh network is established between various Bluetooth mesh devices, including: using a Bluetooth mesh protocol stack to implement multi-hop communication and network self-organization between various devices, and the Bluetooth mesh network is a distributed network.

[0069] Furthermore, the method further comprises: when a new device needs to be added, the Bluetooth mesh unit of the new device is added to the Bluetooth mesh network, and other devices in the Bluetooth mesh network bring the new device into the control range.

[0070] Furthermore, the method also includes: acquiring the control state and operation information of the target device in real time; and feeding back the control state and operation information to the user terminal in real time.

[0071] Furthermore, the method also includes: obtaining usage data from users and regularly collecting new audio training data to update and optimize the speech recognition model, and regularly inspecting and maintaining each device.

[0072] Furthermore, the method also includes: when a device fails or a communication connection is abnormal, an alarm is sent to a user terminal to remind the user to handle it.

[0073] Regularly check and maintain the equipment in the system to ensure the normal operation of the equipment's hardware and software. Through the user interface or management background, you can set system parameters, manage equipment, update voice recognition models, and more. At the same time, the system has fault diagnosis and alarm functions. When a device fails or the network connection is abnormal, it can issue an alarm in time to remind the user to handle it and ensure the stable operation of the system.

[0074] The embodiments of the present invention have the following beneficial effects:

[0075] 1) Work offline, not restricted by network:

[0076] Offline voice recognition technology is used to realize voice control function without network connection, which solves the problem of online voice recognition's dependence on the network and ensures that users can still control smart devices conveniently and quickly in an environment without network or unstable network;

[0077] 2) Privacy and security protection:

[0078] Voice recognition and command processing are all completed on the local device, and voice data does not need to be uploaded to the cloud, effectively avoiding the risk of privacy data leakage and ensuring the privacy security of users;

[0079] 3) Quick response to improve user experience:

[0080] Since there is no need for network transmission and cloud processing, the system has an extremely fast response speed and can respond to user voice commands in real time, providing users with a smooth and efficient control experience in scenarios with high real-time requirements such as smart lighting control and smart curtain opening and closing.

[0081] 4) Large-scale networking and flexible expansion:

[0082] Using Bluetooth mesh technology, the system can achieve large-scale device networking, with unlimited number of connections and support flexible device expansion. Whether it is a small home or a large commercial venue, a large number of smart devices can be easily deployed and managed to meet the needs of different scenarios;

[0083] 5) Low-cost and efficient deployment:

[0084] Compared with the online speech recognition system that relies on cloud services, the offline speech recognition control system of the present invention does not require payment of cloud service fees, which reduces the cost of use. At the same time, the deployment and installation of the system are simple and convenient, without the need for complex network configuration and support from professional technicians. Users can complete the installation and settings of the equipment by themselves.

[0085] Although the present invention has been described in detail above by general description and specific embodiments, it is obvious to those skilled in the art that some modifications or improvements can be made to the present invention. Therefore, these modifications or improvements made without departing from the spirit of the present invention all belong to the scope of protection claimed by the present invention.

Claims

1. An offline speech recognition control method based on Bluetooth mesh, characterized in that: The method comprises: S1, collecting a first voice signal from a user, and preprocessing the collected first voice signal to form a second voice signal; S2. Recognize the second voice signal using a pre-deployed local voice recognition model, and convert the audio file into a text instruction; S3, obtaining the text instruction and parsing and semantically understanding the text instruction, extracting control information and generating corresponding control instructions according to preset control logic; S4. Establish a Bluetooth mesh network between various Bluetooth mesh devices, send the control instruction to the Bluetooth mesh receiver of the target device, and after receiving the control instruction, the target device calls the corresponding control interface and driver to control the corresponding smart device.

2. The offline speech recognition control method based on Bluetooth mesh according to claim 1, characterized in that: The preprocessing includes noise reduction, echo removal and speech enhancement; The noise reduction is achieved by adjusting the filter parameters in real time according to the changes in the ambient noise through an adaptive filtering algorithm.

3. The offline speech recognition control method based on Bluetooth mesh as claimed in claim 2, characterized in that: Using a pre-locally deployed speech recognition model to recognize the second speech signal and convert the audio file into a text instruction includes: Deploy the speech recognition model locally and obtain a large amount of audio training data; Using a deep learning algorithm and audio training data to train the speech recognition model, adjusting the model structure and parameters, and obtaining a trained speech recognition model; The second voice signal is recognized using the trained voice recognition model, and the audio file is converted into text instructions.

4. The offline speech recognition control method based on Bluetooth mesh as claimed in claim 3, characterized in that: The audio training data includes audio data of different accents, speaking speeds, and language habits, as well as various common control instruction data.

5. The offline speech recognition control method based on Bluetooth mesh as claimed in claim 4, characterized in that: Establish a Bluetooth mesh network between various Bluetooth mesh devices, including: The Bluetooth mesh protocol stack is used to realize multi-hop communication and network self-organization between various devices. The Bluetooth mesh networking is a distributed networking.

6. The offline speech recognition control method based on Bluetooth mesh as claimed in claim 5, characterized in that: The method further comprises: When a new device needs to be added, the Bluetooth mesh unit of the new device is added to the Bluetooth mesh network, and other devices in the Bluetooth mesh network bring the new device into the control range.

7. The offline speech recognition control method based on Bluetooth mesh as claimed in claim 6, characterized in that: The method further comprises: Obtain the control status and operation information of the target device in real time; The control status and operation information are fed back to the user terminal in real time.

8. The offline speech recognition control method based on Bluetooth mesh as claimed in claim 7, characterized in that: The method further comprises: Acquire usage data from users and regularly collect new audio training data to update and optimize the speech recognition model, and regularly inspect and maintain each device.

9. The offline speech recognition control method based on Bluetooth mesh as claimed in claim 8, characterized in that: The method further includes: when a device fails or a communication connection is abnormal, sending an alarm to a user terminal to remind the user to handle the problem.

10. An offline speech recognition control system based on Bluetooth mesh, characterized in that: The system comprises: A voice collection and preprocessing module, used to collect a first voice signal from a user, and preprocess the collected first voice signal to form a second voice signal; An offline speech recognition engine, configured to recognize the second speech signal using a pre-locally deployed speech recognition model, and convert the audio file into a text instruction; An instruction parsing and processing module, used to obtain the text instruction and parse and semantically understand the text instruction, extract control information and generate corresponding control instructions according to preset control logic; A Bluetooth mesh communication module, used to establish a Bluetooth mesh network between various Bluetooth mesh devices and send the control command to the Bluetooth mesh receiver of the target device; The device control module is used for the target device to call the corresponding control interface and driver to control the corresponding intelligent device after receiving the control instruction.

Images