Modular voice recognition method and detection system

Abstract

This application relates to the technical field of intelligent voice interaction and industrial control, and particularly to a modular voice recognition method and detection system. The method includes: real-time parsing and feature extraction of raw business data to obtain a sequence of equipment operating states; semantic classification of the equipment operating state sequence to obtain state labels; constructing a state broadcast queue based on the state labels and preset dynamic environment categories; filtering target state information from the state broadcast queue and synthesizing the target state information into a voice broadcast signal for output; capturing the voiceprint features of the current voice signal to obtain a current voice feature vector; inputting the current voice feature vector into a pre-trained instruction recognition model to parse and obtain the user's control intent; encapsulating the user's control intent into a control instruction data packet conforming to the communication protocol of the controlled equipment; and injecting the control instruction data packet into the communication bus. This application improves the accuracy and reliability of instruction transmission.

Description

Technical Field

[0001] This application relates to the technical field of intelligent voice interaction and industrial control, and in particular to a modular voice recognition method and detection system. Background Technology

[0002] The rapid development of IoT technology has driven the popularization of various smart devices. Voice recognition and voice broadcasting, as core functions of intelligent interaction, are widely used in various controlled devices such as industrial control, smart homes, and smart appliances. They provide users with convenient non-contact operation and device status perception methods, greatly improving the human-computer interaction experience and intelligence level of the devices.

[0003] In existing technologies, adding voice recognition and voice broadcasting functions to controlled devices mainly adopts two methods: pre-design integration or post-modification upgrades. However, both methods have significant technical drawbacks. For pre-design integration, the hardware interface, communication protocol, and software adaptation logic of the voice module need to be planned in advance during the device development stage, increasing the complexity of product design and failing to meet the personalized selection needs of different users for voice functions. For post-modification upgrades, older devices or existing mass-produced devices did not have pre-reserved communication ports for voice recognition and broadcasting components in their initial design, meaning direct installation is not supported at the hardware level. This often requires redesigning the motherboard and adding additional communication interfaces, resulting in high costs and long cycles. Furthermore, even if the hardware modification is completed, the device's main control program still needs to be upgraded for compatibility to adapt to the voice module's command interaction logic, further increasing the development workload. Moreover, the modified program is prone to compatibility issues, affecting the normal operation of the original device.

[0004] Furthermore, the existing voice recognition and status broadcast functions of the devices have several technical shortcomings: First, the device status broadcasts lack specificity and fail to dynamically adjust the broadcast strategy based on the urgency of the device's operating status and the actual environmental scenario. Ineffective broadcasts in high-noise environments and redundant broadcasts in unmanned scenarios are prominent issues, which not only affect user experience but also waste resources. Second, the voice recognition has weak anti-interference capabilities and is prone to feature extraction deviations in complex noise environments, leading to errors in interpreting user control intentions. Third, the issuance of voice control commands does not consider the load status of the controlled device's communication bus. Blindly injecting commands can easily cause bus data conflicts, leading to command execution failures or even affecting the stability of the original device's bus communication. Fourth, the broadcast of device operating status does not filter redundant status data, and repetitive and meaningless status broadcasts reduce the efficiency of information transmission.

[0005] To address the aforementioned issues, there is an urgent need for a modular, retrofittable voice recognition solution that requires no modification to the original equipment's hardware and software. This solution should enable intelligent reporting of equipment status and stable delivery of voice control commands, allowing for rapid adaptation of voice interaction functions to various controlled devices without affecting the normal operation of the original equipment. Summary of the Invention

[0006] To address at least one of the aforementioned technical problems, this application provides a modular speech recognition method and detection system.

[0007] Firstly, this application provides a modular speech recognition method, which adopts the following technical solution: Based on the first and second communication segments connected in series on the communication bus of the controlled device, the original service data transmitted by the controlled device and the current voice signal collected by the user terminal are obtained. The original business data is parsed and features are extracted in real time to obtain the equipment operating status sequence; The device operation state sequence is semantically classified to obtain corresponding state labels. The state labels include at least emergency fault, normal operation, and silent ignore categories. The emergency fault category corresponds to a high-priority broadcasting strategy, the normal operation category corresponds to a low-priority broadcasting strategy, and the silent ignore category corresponds to a no-broadcasting strategy. Based on the status tags and preset dynamic environment categories, a status broadcast queue is constructed, target status information is filtered out from the status broadcast queue, and the target status information is synthesized into a voice broadcast signal for output. The current speech signal is subjected to voiceprint feature capture to obtain the current speech feature vector; The current speech feature vector is input into a pre-trained instruction recognition model to parse and obtain the user's control intention; Monitor the real-time load status of the communication bus, calculate the bus idle time window, and when the current time meets the bus idle time window, encapsulate the user control intention into a control instruction data packet that conforms to the communication protocol of the controlled device; The control command data packet is injected into the communication bus to control the controlled device to perform corresponding actions.

[0008] By adopting the above technical solution, based on the first and second communication segments connected in series on the communication bus of the controlled device, it is possible to simultaneously acquire the raw service data transmitted by the controlled device and the current voice signal collected by the user terminal. The raw service data contains key information about the operation of the controlled device, providing basic data support for subsequent analysis of the device status; the current voice signal is the carrier of the user's control intention. Real-time parsing and feature extraction of the raw service data can intuitively reflect the operating status of the device at different times. By semantically classifying the device operating status sequence and assigning it corresponding status labels, different labels correspond to different broadcast strategies, laying the foundation for subsequent reasonable broadcasting of device status information, making the transmission of device status information more organized and targeted. After obtaining the status labels corresponding to the device operating status sequence, a status broadcast queue is constructed in conjunction with preset dynamic environment categories. The dynamic environment categories take into account the differences in user attention to device status information in different scenarios, making the status broadcast queue more in line with actual usage scenarios. Filtering out target status information from the status broadcast queue can filter out unnecessary or user-unrelated information, reducing information interference. The system synthesizes target status information into a voice broadcast signal output, allowing users to quickly understand the key status of the device through voice without manually checking it. This voice broadcast method is convenient and fast, especially suitable for scenarios where users' hands are busy or in inconvenient situations to check the device, improving the efficiency and convenience of users obtaining device status information. Voiceprint features are captured from the current voice signal to obtain a unique voice feature vector, accurately identifying the voice characteristics of different users. This voice feature vector is then input into a pre-trained command recognition model. The pre-trained model, trained on a large amount of data, possesses powerful recognition capabilities and can accurately interpret the user's control intentions. This process realizes the conversion from user voice to specific control intentions, allowing users to interact with the device through natural language without complex operation procedures. Compared to traditional button or touch control methods, voice control is more intuitive and convenient, reducing the difficulty of operating the device and improving the user experience and comfort of interacting with the device. Monitoring the real-time load status of the communication bus and calculating the bus idle time window allows for real-time monitoring of the communication bus usage. When the current time falls within the bus idle time window, the user's control intent is encapsulated into a control command data packet conforming to the controlled device's communication protocol, ensuring that the control command data packet is sent accurately and promptly when the bus is idle. Injecting the control command data packet into the communication bus controls the controlled device to perform corresponding actions, avoiding data conflicts or loss caused by sending commands when the bus is busy. This method of sending control commands rationally based on bus load improves the accuracy and reliability of command transmission, ensuring that the controlled device can stably and accurately execute the user's control intent.

[0009] In one possible implementation, constructing the status broadcast queue based on the status label and a preset dynamic environment category includes: The current dynamic environment category is obtained in real time, and the dynamic environment category includes at least a high noise interference environment, a low noise quiet environment, and an unattended silent environment. Traverse the sequence of device operating statuses to be broadcast and read the status label corresponding to each sequence. If the status label is an emergency fault type of device operating status sequence, ignore the dynamic environment category, mark the status label as the highest priority instant broadcast item, and insert it into the status broadcast queue. If the status label is a sequence of device operating statuses in the normal operation category and the dynamic environment category is a low-noise quiet environment, then the status label is marked as a normal priority delayed broadcast item and added to the status broadcast queue in chronological order. If the status label is a device operating status sequence of the normal operation type and the dynamic environment category is a high noise interference environment, then it is marked as a low priority cache item and temporarily stored in the cache area without entering the status broadcast queue, until the dynamic environment category is detected to switch to a low noise quiet environment before being moved into the queue. If the status label is a sequence of device operating statuses in the normal operating category and the dynamic environment category is an unattended silent environment, then its status label is dynamically updated to the silent ignore category and discarded from the status broadcast queue.

[0010] In one possible implementation, the method further includes: Input any of the device operation state sequences labeled as regular operation into the silent judgment model to determine whether any of the device operation state sequences of the regular operation class belongs to redundant information. The redundant information is state data that appears repeatedly within a preset time period without numerical fluctuation. If the silent determination model determines that it belongs to redundant information, then the status label of any device operating state sequence of the regular operation class is updated to the silent ignore class; If the silent determination model determines that it does not belong to redundant information, then the status label of any device operating status sequence of the regular operating class is retained as the regular operating class, and the conditions for entering the status broadcast queue are met.

[0011] In one possible implementation, the silent determination model includes a time-series encoding layer and an attention mechanism layer. The determination of whether any sequence of device operating states in the regular operating class constitutes redundant information includes: Based on the time series encoding layer, the numerical change rate of any device operating state sequence of the regular operating class within a preset time period is encoded to obtain a time feature vector; The attention mechanism layer assigns weights to the time feature vector, focusing on the time nodes where the values ​​change abruptly. If the weight distribution output by the attention mechanism layer indicates that the rate of change of values ​​at all time points is lower than a preset small threshold, then the output is a judgment result that belongs to redundant information; otherwise, the output is a judgment result that does not belong to redundant information.

[0012] In one possible implementation, the instruction recognition model employs a multi-task learning network, which includes a shared feature extractor, a noise suppression branch, and an intent recognition branch. The parsing of the user control intent includes: Based on the shared feature extractor, deep feature mapping is performed on the current speech feature vector to obtain shared hidden layer features; The shared hidden layer features are input into the noise suppression branch, and denoised clean speech features are generated through adversarial training. The clean speech features are input into the intent recognition branch, and the probability distribution of the user control intent is output through the classification head. The category with the highest probability is selected as the final user control intent.

[0013] In one possible implementation, monitoring the real-time load status of the communication bus and calculating the bus idle time window includes: Real-time sampling of the level signals on the communication bus, and statistical analysis of the data frame density per unit time; If the data frame density is lower than a preset idle threshold, a timer is started to record the duration of no continuous data frame transmission. When the duration of the continuous absence of data frame transmission is greater than or equal to the minimum frame interval specified in the protocol, the current moment is determined as the starting point of the bus idle time window; Calculate the duration of the bus idle time window. If the duration is greater than the transmission time required to encapsulate the control instruction data packet, it is determined to be a valid injection opportunity.

[0014] Secondly, this application provides a modular speech recognition system, which adopts the following technical solution: A modular speech recognition system, comprising: The data acquisition module is used to acquire the original service data transmitted by the controlled device and the current voice signal collected by the user terminal based on the first communication segment and the second communication segment connected in series on the communication bus of the controlled device. The feature extraction module is used to perform real-time parsing and feature extraction on the original business data to obtain the device operating status sequence; The semantic classification module is used to perform semantic classification on the device operation state sequence to obtain corresponding state labels. The state labels include at least emergency fault category, normal operation category and silent ignore category. The emergency fault category corresponds to a high priority broadcast strategy, the normal operation category corresponds to a low priority broadcast strategy, and the silent ignore category corresponds to a no broadcast strategy. The broadcast output module constructs a status broadcast queue based on the status tags and preset dynamic environment categories, filters out target status information from the status broadcast queue, and synthesizes the target status information into a voice broadcast signal for output. The feature capture module is used to capture the voiceprint features of the current speech signal to obtain the current speech feature vector; The intent parsing module is used to input the current speech feature vector into a pre-trained instruction recognition model to parse and obtain the user's control intent; The intent encapsulation module is used to monitor the real-time load status of the communication bus, calculate the bus idle time window, and encapsulate the user control intent into a control instruction data packet that conforms to the communication protocol of the controlled device when the current time meets the bus idle time window. The device control module is used to inject the control command data packet into the communication bus to control the controlled device to perform corresponding actions.

[0015] Thirdly, this application provides an electronic device that adopts the following technical solution: At least one processor; Memory; At least one application, wherein the at least one application is stored in memory and configured to be executed by at least one processor, the at least one application being configured to: execute a modular speech recognition method as described in any of the first aspects.

[0016] Fourthly, this application provides a computer-readable storage medium, which adopts the following technical solution: A computer-readable storage medium having a computer program stored thereon, which, when executed in a computer, causes the computer to perform a modular speech recognition method as described in any of the first aspects.

[0017] In summary, this application includes at least one of the following beneficial technical effects: By adopting the above technical solution, based on the first and second communication segments connected in series on the communication bus of the controlled device, it is possible to simultaneously acquire the raw service data transmitted by the controlled device and the current voice signal collected by the user terminal. The raw service data contains key information about the operation of the controlled device, providing basic data support for subsequent analysis of the device status; the current voice signal is the carrier of the user's control intention. Real-time parsing and feature extraction of the raw service data can intuitively reflect the operating status of the device at different times. By semantically classifying the device operating status sequence and assigning it corresponding status labels, different labels correspond to different broadcast strategies, laying the foundation for subsequent reasonable broadcasting of device status information, making the transmission of device status information more organized and targeted. After obtaining the status labels corresponding to the device operating status sequence, a status broadcast queue is constructed in conjunction with preset dynamic environment categories. The dynamic environment categories take into account the differences in user attention to device status information in different scenarios, making the status broadcast queue more in line with actual usage scenarios. Filtering out target status information from the status broadcast queue can filter out unnecessary or user-unrelated information, reducing information interference. The system synthesizes target status information into a voice broadcast signal output, allowing users to quickly understand the key status of the device through voice without manually checking it. This voice broadcast method is convenient and fast, especially suitable for scenarios where users' hands are busy or in inconvenient situations to check the device, improving the efficiency and convenience of users obtaining device status information. Voiceprint features are captured from the current voice signal to obtain a unique voice feature vector, accurately identifying the voice characteristics of different users. This voice feature vector is then input into a pre-trained command recognition model. The pre-trained model, trained on a large amount of data, possesses powerful recognition capabilities and can accurately interpret the user's control intentions. This process realizes the conversion from user voice to specific control intentions, allowing users to interact with the device through natural language without complex operation procedures. Compared to traditional button or touch control methods, voice control is more intuitive and convenient, reducing the difficulty of operating the device and improving the user experience and comfort of interacting with the device. Monitoring the real-time load status of the communication bus and calculating the bus idle time window allows for real-time monitoring of the communication bus usage. When the current time falls within the bus idle time window, the user's control intent is encapsulated into a control command data packet conforming to the controlled device's communication protocol, ensuring that the control command data packet is sent accurately and promptly when the bus is idle. Injecting the control command data packet into the communication bus controls the controlled device to perform corresponding actions, avoiding data conflicts or loss caused by sending commands when the bus is busy. This method of sending control commands rationally based on bus load improves the accuracy and reliability of command transmission, ensuring that the controlled device can stably and accurately execute the user's control intent. Attached Figure Description

[0018] Figure 1This is a flowchart illustrating a modular speech recognition method provided in an embodiment of this application.

[0019] Figure 2 This is a schematic diagram of the structure of a modular speech recognition system provided in an embodiment of this application.

[0020] Figure 3 This is a schematic diagram of the structure of an electronic device provided in an embodiment of this application. Detailed Implementation

[0021] The following is in conjunction with the appendix Figure 1-3 This application will be described in further detail.

[0022] This specific embodiment is merely an explanation of this application and is not intended to limit it. After reading this specification, those skilled in the art can make modifications to this embodiment without contributing any inventive step, but such modifications are protected by patent law as long as they are within the scope of this application.

[0023] To make the objectives, technical solutions, and advantages of the embodiments of this application clearer, the technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0024] Furthermore, the term "and / or" in this article is merely a description of the relationship between related objects, indicating that three relationships can exist. For example, A and / or B can represent: A existing alone, A and B existing simultaneously, or B existing alone. Additionally, the character " / " in this article, unless otherwise specified, generally indicates that the preceding and following related objects have an "or" relationship.

[0025] The embodiments of this application will now be described in further detail with reference to the accompanying drawings.

[0026] This application provides a modular speech recognition method, executed by an electronic device, such as... Figure 1 As shown, the method includes: Step S10: Based on the first and second communication segments connected in series on the communication bus of the controlled device, acquire the original service data transmitted by the controlled device and the current voice signal collected by the user terminal.

[0027] In this embodiment, the modular hardware includes a processor, communication circuits A / B, a speech recognition circuit, a speech synthesis circuit, a sound equalization processing circuit, a millimeter-wave radar, a microphone system, a power amplifier and speaker, and a power module. The processor uses an STM32F103C8T6 with a maximum clock speed of 72MHz, 64KB Flash and 20KB SRAM. Its core functions are: core data processing, protocol parsing, model calculation, and instruction scheduling. The communication circuits A / B use a configurable bus interface board with integrated RS232 (MAX232), RS485 (MAX485), CAN (TJA1050), and UART (using the processor's native UART without the CH340 converter chip). The bus type is configured via the processor's GPIO pins, supporting bidirectional pass-through, level conversion, and bus monitoring. The specifications of the speech recognition circuit include: a speech recognition chip (SNR9912VR), whose core functions are: supporting offline speech recognition, a sampling rate of 16kHz, a recognition distance of 0-5m, and customizable instruction sets. The specifications of the speech synthesis circuit include: a speech synthesis chip (SNR9816TTS), whose core functions are: supporting Chinese speech synthesis, adjustable speech rate / volume, and outputting 8kHz / 16kHz audio signals. The specifications of the sound equalization processing circuit include: a speech equalization adjustment chip (CSC2313S), whose core function is: performing 5-band equalization adjustment on the synthesized speech signal to adapt to different noise environments and improve broadcast clarity. The specifications of the millimeter-wave radar include: HLK-LD1125H (24GHz), whose core functions are: detection distance of 0.1-5m, outputting a human presence level signal, and a sampling frequency of 10Hz. The microphone system uses a dual-microphone setup (single condenser microphone) with noise reduction circuitry. Its core features include a 16kHz sampling rate, a signal-to-noise ratio ≥60dB, and built-in AEC echo cancellation and ANC noise suppression. The amplifier and speaker system uses a PAM8403 amplifier with an 8Ω / 2W speaker. Its core features include a 2W output power and a frequency response of 200Hz-5kHz. The power module uses a DC-DC step-down module (LM2596), with a 9-24V DC input and a 5V / 3A output, powering the entire module.

[0028] In this embodiment of the application, based on the serially connected first and second communication segments, the original service data and user voice signals of the controlled device are acquired. Specifically, the processor configures the bus type of communication circuit A / B through GPIO (e.g., RS485) and sets the enable pin of the communication circuit to receive mode (the DE / RE pin of MAX485 is set low). The communication circuit collects the level signal on the bus in real time and performs frame parsing according to a preset baud rate (e.g., 9600bps), data bits (8 bits), stop bits (1 bit), and parity bits (even parity), converting the analog level signal into a digital original service data frame. The original service data frame format is: frame header (2 bytes) + address field (1 byte) + function code (1 byte) + data field (N bytes) + check code (1 byte) + frame tail (1 byte). The processor stores the parsed complete data frame into a FIFO buffer with a size of 1024 bytes, using a first-in-first-out rule to avoid data overflow. The processor reads the data from the FIFO buffer at a frequency of 10ms / time to ensure real-time performance.

[0029] The processor configures the microphone recording system's sampling parameters as follows: sampling rate 16kHz, quantization bit depth 16bit, mono. The microphone continuously acquires ambient voice signals. When the amplitude of the detected voice signal exceeds a preset threshold (e.g., 500mV), it triggers an interrupt in the voice recognition circuit, encapsulates the voice signals within a continuous time period into voice data packets, and transmits them to the processor's RAM. If the millimeter-wave radar detects no human presence (outputs a low level), it disables the voice signal acquisition function, retaining only bus data acquisition.

[0030] Step S11: Perform real-time analysis and feature extraction on the original business data to obtain the equipment operation status sequence.

[0031] Specifically, the bus communication protocol parsing table of the controlled device is loaded from Flash (which can be written in advance by the host computer). The parsing table includes: the mapping relationship between function codes and device status (e.g., function code 0x01 corresponds to "running status", 0x02 corresponds to "fault status"), the numerical parsing rules of data fields (e.g., data field 0x0001 corresponds to "power on", 0x0002 corresponds to "power off"), and the numerical unit (e.g., 0x0064 corresponds to "temperature 64℃").

[0032] The electronic device traverses the raw service data frames in the FIFO buffer and matches the function code according to the parsing table: if the function code is "operating status" (0x01), then the first two bytes of the parsed data field are the operating power (decimal conversion, unit W), and the third and fourth bytes are the operating temperature (decimal conversion, unit °C); if the function code is "fault status" (0x02), then the first byte of the parsed data field is the fault code (e.g., 0x01 = overtemperature, 0x02 = overload). The electronic device serializes the parsed status parameters (operating power, temperature, fault codes, etc.) according to timestamps (accurate to milliseconds) to generate a device operating status sequence, in the format: [timestamp 1: parameter 1 = value 1, parameter 2 = value 2; timestamp 2: parameter 1 = value 1, parameter 2 = value 2; ...]; 2.4 The status sequence is stored in a dedicated area of ​​RAM, with a maximum storage length of 100 records. If the length exceeds this, the oldest record will be overwritten.

[0033] Step S12: Perform semantic classification on the device operating state sequence to obtain the corresponding state labels.

[0034] The status labels include at least three categories: emergency fault, normal operation, and silent ignore.

[0035] Among them, emergency faults correspond to high-priority broadcasting strategies, normal operation faults correspond to low-priority broadcasting strategies, and silent ignore faults correspond to no broadcasting strategies.

[0036] Specifically, a preset semantic classification rule is used to semantically determine the feature parameters in the device's operating status sequence: if the feature parameters contain emergency information such as device hardware failure, operation exceeding the threshold, or emergency shutdown, they are marked as emergency failures and should be broadcast with high priority; if the feature parameters are normal operating status information (such as normal start-up and shutdown, parameter fine-tuning, and normal operating condition switching), they are marked as normal operation and broadcast with low priority as needed; if the feature parameters are meaningless empty data or invalid interactive data, they are directly marked as silent and ignored and not broadcast.

[0037] Step S13: Based on the status labels and preset dynamic environment categories, construct a status broadcast queue, filter out the target status information from the status broadcast queue, and synthesize the target status information into a voice broadcast signal for output.

[0038] Specifically, the current dynamic environment category is obtained in real time. This category includes at least high-noise interference environment, low-noise quiet environment, and unattended silent environment. The device operating status sequence to be broadcast is traversed, and the status tag corresponding to each sequence is read. If the status tag is for an emergency fault, the dynamic environment category is ignored, the status tag is marked as the highest priority immediate broadcast item, and inserted into the status broadcast queue. If the status tag is for a normal operation, and the dynamic environment category is low-noise quiet environment, the status tag is marked as a normal priority delayed broadcast item and added to the status broadcast queue in chronological order. If the status tag is for a normal operation, and the dynamic environment category is high-noise interference environment, it is marked as a low-priority cache item, temporarily stored in the cache without entering the status broadcast queue, until the dynamic environment category is detected to have switched to low-noise quiet environment, at which point it is moved into the queue. If the status tag is for a normal operation, and the dynamic environment category is unattended silent environment, its status tag is dynamically updated to the silent ignore category, discarded, and not added to the status broadcast queue.

[0039] In the embodiments of this application, the real-time acquisition methods for unmanned silent environments include, but are not limited to, using 24GHz millimeter-wave radar to detect human activity.

[0040] Furthermore, to avoid redundant broadcasting of routine operating status information, a redundant information filtering step is included: Any device operating status sequence labeled as "routine operating" is input into the silent judgment model to determine whether it belongs to redundant information. Redundant information refers to status data that repeats within a preset time period without numerical fluctuation. If the silent judgment model determines it to be redundant information, the status label of any device operating status sequence in the routine operating category is updated to the silent ignore category. If the silent judgment model determines it to be non-redundant information, the status label of any device operating status sequence in the routine operating category is retained as "routine operating," and it meets the conditions for entering the status broadcast queue.

[0041] Specifically, the silent judgment model includes a time series encoding layer and an attention mechanism layer. The time series encoding layer encodes the rate of change of values ​​in the operating state sequence of any device in a normal operating class within a preset time period, obtaining a time feature vector. The attention mechanism layer then assigns weights to the time feature vector, focusing on the time nodes where values ​​change abruptly. If the weight distribution output by the attention mechanism layer indicates that the rate of change of values ​​at all time nodes is below a preset small threshold, the output is a judgment result indicating that the information is redundant; otherwise, the output is a judgment result indicating that the information is not redundant.

[0042] Step S14: Capture the voiceprint features of the current speech signal to obtain the current speech feature vector.

[0043] Specifically, the speech recognition circuit preprocesses the current speech signal collected by the microphone recording system (denoising, filtering, and framing), captures the voiceprint features of the speech through voiceprint feature extraction algorithms (such as MFCC algorithm and Mel spectrum algorithm), and converts them into a digital current speech feature vector, which serves as the input for parsing the user's control intent.

[0044] Step S15: Input the current speech feature vector into the pre-trained instruction recognition model to parse and obtain the user's control intention.

[0045] Specifically, a deep feature mapping is performed on the current speech feature vector based on a shared feature extractor to obtain shared hidden layer features. These shared hidden layer features are then input into a noise suppression branch, where adversarial training generates denoised, clean speech features. Finally, these clean speech features are input into an intent recognition branch, which outputs a probability distribution of the user's control intent through a classification head. The category with the highest probability is selected as the final user control intent.

[0046] Step S16: Monitor the real-time load status of the communication bus, calculate the bus idle time window, and when the current time meets the bus idle time window, encapsulate the user control intent into a control instruction data packet that conforms to the communication protocol of the controlled device.

[0047] Specifically, the system samples the level signals on the communication bus in real time and counts the data frame density per unit time. If the data frame density is lower than a preset idle threshold, a timer is started to record the duration of continuous data frame transmission without transmission. When the duration of continuous data frame transmission without transmission is greater than or equal to the minimum frame interval specified in the protocol, the current moment is determined as the starting point of the bus idle time window. The duration of the bus idle time window is calculated. If the duration is greater than the transmission time required to encapsulate the control command data packet, it is determined to be a valid injection opportunity.

[0048] Step S17: Inject control command data packets into the communication bus to control the controlled device to perform corresponding actions.

[0049] Specifically, the voice box inserts the encapsulated control command data packet into the idle time window of the communication bus through communication circuit A or B to realize the transparent transmission and distribution of commands. After the main controller or each component of the controlled device receives the control command data packet that conforms to the protocol, it executes the corresponding action to complete the voice control.

[0050] This application provides a modular speech recognition method based on a first communication segment and a second communication segment connected in series on the communication bus of the controlled device. This method can simultaneously acquire the raw service data transmitted by the controlled device and the current voice signal collected by the user terminal. The raw service data contains key information about the operation of the controlled device, providing basic data support for subsequent analysis of the device status; the current voice signal is the carrier of the user's control intent. Real-time parsing and feature extraction of the raw service data can intuitively reflect the device's operating status at different times. By semantically classifying the device's operating status sequence and assigning it corresponding status labels, different labels correspond to different broadcast strategies, laying the foundation for subsequent reasonable broadcasting of device status information and making the transmission of device status information more organized and targeted. After obtaining the status labels corresponding to the device operating status sequence, a status broadcast queue is constructed by combining preset dynamic environment categories. The dynamic environment categories consider the differences in user attention to device status information in different scenarios, making the status broadcast queue more closely aligned with actual usage scenarios. Filtering out target status information from the status broadcast queue can filter out unnecessary or user-unrelated information, reducing information interference. The system synthesizes target status information into a voice broadcast signal output, allowing users to quickly understand the key status of the device through voice without manually checking it. This voice broadcast method is convenient and fast, especially suitable for scenarios where users' hands are busy or in inconvenient situations to check the device, improving the efficiency and convenience of users obtaining device status information. Voiceprint features are captured from the current voice signal to obtain a unique voice feature vector, accurately identifying the voice characteristics of different users. This voice feature vector is then input into a pre-trained command recognition model. The pre-trained model, trained on a large amount of data, possesses powerful recognition capabilities and can accurately interpret the user's control intentions. This process realizes the conversion from user voice to specific control intentions, allowing users to interact with the device through natural language without complex operation procedures. Compared to traditional button or touch control methods, voice control is more intuitive and convenient, reducing the difficulty of operating the device and improving the user experience and comfort of interacting with the device. Monitoring the real-time load status of the communication bus and calculating the bus idle time window allows for real-time monitoring of the communication bus usage. When the current time falls within the bus idle time window, the user's control intent is encapsulated into a control command data packet conforming to the controlled device's communication protocol, ensuring that the control command data packet is sent accurately and promptly when the bus is idle. Injecting the control command data packet into the communication bus controls the controlled device to perform corresponding actions, avoiding data conflicts or loss caused by sending commands when the bus is busy. This method of sending control commands rationally based on bus load improves the accuracy and reliability of command transmission, ensuring that the controlled device can stably and accurately execute the user's control intent.

[0051] The following describes a modular speech recognition system provided by an embodiment of this application. The modular speech recognition system described below can be referred to in conjunction with the modular speech recognition method described above. Figure 2 , Figure 2 This is a schematic diagram of the structure of a modular speech recognition system 20 provided in an embodiment of this application, including: The data acquisition module 21 is used to acquire the original service data transmitted by the controlled device and the current voice signal collected by the user terminal based on the first communication segment and the second communication segment connected in series on the communication bus of the controlled device. Feature extraction module 22 is used to perform real-time parsing and feature extraction on the original business data to obtain the equipment operating status sequence; The semantic classification module 23 is used to perform semantic classification on the device operation status sequence to obtain the corresponding status label. The status label includes at least emergency fault, normal operation and silent ignore. The emergency fault corresponds to the high priority broadcast strategy, the normal operation corresponds to the low priority broadcast strategy, and the silent ignore corresponds to the no broadcast strategy. The broadcast output module 24 constructs a status broadcast queue based on status tags and preset dynamic environment categories, filters out target status information from the status broadcast queue, and synthesizes the target status information into a voice broadcast signal for output. Feature capture module 25 is used to capture the voiceprint features of the current speech signal and obtain the current speech feature vector; The intent parsing module 26 is used to input the current speech feature vector into the pre-trained instruction recognition model to parse the user's control intent; The intent encapsulation module 27 is used to monitor the real-time load status of the communication bus, calculate the bus idle time window, and encapsulate the user control intent into a control instruction data packet that conforms to the communication protocol of the controlled device when the current time meets the bus idle time window. The device control module 28 is used to inject control command data packets into the communication bus to control the controlled device to perform corresponding actions.

[0052] In one possible implementation of this application embodiment, when the broadcast output module 24 constructs a status broadcast queue based on status labels and preset dynamic environment categories, it is specifically used for: Real-time acquisition of the current dynamic environment category, which includes at least high noise interference environment, low noise quiet environment, and unmanned silent environment; Iterate through the sequence of device operating statuses to be broadcast and read the status label corresponding to each sequence. If the status label is an emergency fault type device operating status sequence, ignore the dynamic environment category, mark the status label as the highest priority instant broadcast item, and insert it into the status broadcast queue. If the status label is a sequence of device operating statuses in the normal operation category and the dynamic environment category is a low-noise quiet environment, then the status label is marked as a normal priority delayed broadcast item and added to the status broadcast queue in chronological order. If the status label is a device operating status sequence of the normal operation category and the dynamic environment category is a high noise interference environment, then it is marked as a low priority cache item and temporarily stored in the cache area without entering the status broadcast queue, until the dynamic environment category is detected to switch to a low noise quiet environment before being moved into the queue. If the status label is a regular operating status sequence and the dynamic environment category is an unattended silent environment, then its status label will be dynamically updated to the silent ignore category and discarded from the status broadcast queue.

[0053] In another possible implementation of this application embodiment, the system further includes: a redundancy judgment module, a first update module, and a second update module, wherein, The redundancy judgment module is used to input any device operation status sequence with the status label of regular operation into the silent judgment model to determine whether any device operation status sequence with the status label of regular operation belongs to redundant information. Redundant information is status data that appears repeatedly within a preset time period without numerical fluctuation. The first update module is used to update the status label of any device running status sequence in the normal operation class to the silent ignore class when the silent judgment model determines that it belongs to redundant information. The second update module is used to retain the status label of any device operating status sequence of the regular operating class as the regular operating class when the silent judgment model determines that it does not belong to redundant information, and to meet the conditions for entering the status broadcast queue.

[0054] Another possible implementation in this application embodiment is that the silent determination model includes a time series encoding layer and an attention mechanism layer. The redundancy determination module, when determining whether any device operating state sequence of a normal operating class belongs to redundant information, is specifically used for: The time series coding layer encodes the rate of change of the numerical values ​​of any device operating status sequence in a regular operating class within a preset time period to obtain a time feature vector. The attention mechanism layer assigns weights to the time feature vectors, focusing on the time nodes where the values ​​change abruptly. If the weight distribution output by the attention mechanism layer indicates that the rate of change of values ​​at all time points is lower than a preset small threshold, then the output is a judgment result that belongs to redundant information; otherwise, the output is a judgment result that does not belong to redundant information.

[0055] Another possible implementation in this application embodiment is that the instruction recognition model adopts a multi-task learning network, which includes a shared feature extractor, a noise suppression branch, and an intent recognition branch. When the intent parsing module 26 parses the user's control intent, it is specifically used for: Deep feature mapping is performed on the current speech feature vector based on a shared feature extractor to obtain shared hidden layer features; The shared hidden layer features are input into the noise suppression branch, and the denoised clean speech features are generated through adversarial training. The clean speech features are input into the intent recognition branch, and the probability distribution of the user's control intent is output through the classification head. The category with the highest probability is selected as the final user control intent.

[0056] Another possible implementation in this application embodiment, whereby the encapsulation module 27 is used specifically for: monitoring the real-time load status of the communication bus and calculating the bus idle time window, Real-time sampling of the level signals on the communication bus, and statistical analysis of the data frame density per unit time; If the data frame density is lower than the preset idle threshold, a timer is started to record the duration of no continuous data frame transmission. When the duration of continuous data frame transmission is greater than or equal to the minimum frame interval specified by the protocol, the current moment is determined as the starting point of the bus idle time window; Calculate the duration of the bus idle time window. If the duration is greater than the transmission time required to encapsulate the control instruction data packet, it is determined to be a valid injection opportunity.

[0057] This application provides an electronic device, such as... Figure 3 As shown, Figure 3 This is a schematic diagram of the structure of an electronic device provided in an embodiment of this application. Figure 3 The illustrated electronic device 300 includes a processor 301 and a memory 303. The processor 301 and the memory 303 are connected, for example, via a bus 302. Optionally, the electronic device 300 may also include a transceiver 304. It should be noted that in practical applications, the transceiver 304 is not limited to one type, and the structure of this electronic device 300 does not constitute a limitation on the embodiments of this application.

[0058] Processor 301 may be a CPU (Central Processing Unit), a general-purpose processor, a DSP (Digital Signal Processor), an ASIC (Application Specific Integrated Circuit), an FPGA (Field Programmable Gate Array), or other programmable logic devices, transistor logic devices, hardware components, or any combination thereof. It can implement or execute the various exemplary logic blocks, modules, and circuits described in connection with the embodiments of this application. Processor 301 may also be a combination that implements computing functions, such as including one or more microprocessor combinations, a combination of a DSP and a microprocessor, etc.

[0059] Bus 302 may include a pathway for transmitting information between the aforementioned components. Bus 302 may be a PCI (Peripheral Component Interconnect) bus or an EISA (Extended Industry Standard Architecture) bus, etc. Bus 302 can be divided into address bus, data bus, control bus, etc. For ease of representation, Figure 3 The bus is represented by a single thick line, but this does not mean that there is only one bus or one type of bus.

[0060] The memory 303 may be a ROM (Read Only Memory) or other type of static storage device capable of storing static information and instructions, RAM (Random Access Memory) or other type of dynamic storage device capable of storing information and instructions, or an EEPROM (Electrically Erasable Programmable Read Only Memory), CD-ROM (Compact Disc Read Only Memory) or other optical disc storage, optical disc storage (including compressed optical discs, laser discs, optical discs, digital universal optical discs, Blu-ray discs, etc.), magnetic disk storage media or other magnetic storage devices, or any other medium capable of carrying or storing desired program code in the form of instructions or data structures and accessible by a computer, but not limited thereto.

[0061] The memory 303 is used to store application code that executes the scheme of the embodiments of this application, and its execution is controlled by the processor 301. The processor 301 is used to execute the application code stored in the memory 303 to implement the content shown in the foregoing method embodiments.

[0062] Among them, electronic devices include, but are not limited to: mobile terminals such as mobile phones, laptops, digital radio receivers, PDAs (personal digital assistants), PADs (tablet computers), PMPs (portable multimedia players), and in-vehicle terminals (such as in-vehicle navigation terminals), as well as fixed terminals such as digital TVs and desktop computers. Figure 3 The electronic device shown is merely an example and should not impose any limitation on the functionality and scope of use of the embodiments of this application.

[0063] The following describes a computer-readable storage medium provided by an embodiment of this application. The computer-readable storage medium described below can be referred to in correspondence with the method described above.

[0064] This application provides a computer-readable storage medium storing a computer program, which, when executed by a processor, implements the steps of the modular speech recognition system described above.

[0065] Since the embodiments of the computer-readable storage medium portion correspond to the embodiments of the method portion, please refer to the description of the embodiments of the method portion for the embodiments of the computer-readable storage medium portion.

[0066] It should be understood that although the steps in the flowcharts of the accompanying figures are shown sequentially as indicated by the arrows, these steps are not necessarily executed in the order indicated by the arrows. Unless explicitly stated herein, there is no strict order restriction on the execution of these steps, and they can be executed in other orders. Moreover, at least some steps in the flowcharts of the accompanying figures may include multiple sub-steps or multiple stages. These sub-steps or stages are not necessarily completed at the same time, but can be executed at different times, and their execution order is not necessarily sequential, but can be performed alternately or in turn with other steps or at least some of the sub-steps or stages of other steps.

[0067] The above are only some embodiments of this application. It should be noted that for those skilled in the art, several improvements and modifications can be made without departing from the principle of this application, and these improvements and modifications should also be considered within the scope of protection of this application.

Claims

1. A modular speech recognition method, characterized by, include: Based on the first and second communication segments connected in series on the communication bus of the controlled device, the original service data transmitted by the controlled device and the current voice signal collected by the user terminal are obtained. The original business data is parsed and features are extracted in real time to obtain the equipment operating status sequence; The device operation state sequence is semantically classified to obtain corresponding state labels. The state labels include at least emergency fault, normal operation, and silent ignore categories. The emergency fault category corresponds to a high-priority broadcasting strategy, the normal operation category corresponds to a low-priority broadcasting strategy, and the silent ignore category corresponds to a no-broadcasting strategy. Based on the status tags and preset dynamic environment categories, a status broadcast queue is constructed, target status information is filtered out from the status broadcast queue, and the target status information is synthesized into a voice broadcast signal for output. The current speech signal is subjected to voiceprint feature capture to obtain the current speech feature vector; The current speech feature vector is input into a pre-trained instruction recognition model to parse and obtain the user's control intention; Monitor the real-time load status of the communication bus, calculate the bus idle time window, and when the current time meets the bus idle time window, encapsulate the user control intention into a control instruction data packet that conforms to the communication protocol of the controlled device; The control command data packet is injected into the communication bus to control the controlled device to perform corresponding actions.

2. The modular speech recognition method of claim 1, wherein, The step of constructing a status broadcast queue based on the status label and a preset dynamic environment category includes: The current dynamic environment category is obtained in real time, and the dynamic environment category includes at least a high noise interference environment, a low noise quiet environment, and an unattended silent environment. Traverse the sequence of device operating statuses to be broadcast and read the status label corresponding to each sequence. If the status label is an emergency fault type of device operating status sequence, ignore the dynamic environment category, mark the status label as the highest priority instant broadcast item, and insert it into the status broadcast queue. If the status label is a sequence of device operating statuses in the normal operation category and the dynamic environment category is a low-noise quiet environment, then the status label is marked as a normal priority delayed broadcast item and added to the status broadcast queue in chronological order. If the status label is a device operating status sequence of the normal operation type and the dynamic environment category is a high noise interference environment, then it is marked as a low priority cache item and temporarily stored in the cache area without entering the status broadcast queue, until the dynamic environment category is detected to switch to a low noise quiet environment before being moved into the queue. If the status label is a sequence of device operating statuses in the normal operating category and the dynamic environment category is an unattended silent environment, then its status label is dynamically updated to the silent ignore category and discarded from the status broadcast queue.

3. The modular speech recognition method of claim 1, wherein, The method further includes: Input any of the device operation state sequences labeled as regular operation into the silent judgment model to determine whether any of the device operation state sequences of the regular operation class belongs to redundant information. The redundant information is state data that appears repeatedly within a preset time period without numerical fluctuation. If the silent determination model determines that it belongs to redundant information, then the status label of any device operating state sequence of the regular operation class is updated to the silent ignore class; If the silent determination model determines that it does not belong to redundant information, then the status label of any device operating status sequence of the regular operating class is retained as the regular operating class, and the conditions for entering the status broadcast queue are met.

4. The modular speech recognition method of claim 3, wherein, The silent determination model includes a time series encoding layer and an attention mechanism layer. The determination of whether any sequence of device operating states in the regular operating class constitutes redundant information includes: Based on the time series encoding layer, the numerical change rate of any device operating state sequence of the regular operating class within a preset time period is encoded to obtain a time feature vector; The attention mechanism layer assigns weights to the time feature vector, focusing on the time nodes where the values ​​change abruptly. If the weight distribution output by the attention mechanism layer indicates that the rate of change of values ​​at all time points is lower than a preset small threshold, then the output is a judgment result that belongs to redundant information; otherwise, the output is a judgment result that does not belong to redundant information.

5. The modular speech recognition method of claim 1, wherein, The instruction recognition model employs a multi-task learning network, which includes a shared feature extractor, a noise suppression branch, and an intent recognition branch. The parsed user control intent includes: Based on the shared feature extractor, deep feature mapping is performed on the current speech feature vector to obtain shared hidden layer features; The shared hidden layer features are input into the noise suppression branch, and denoised clean speech features are generated through adversarial training. The clean speech features are input into the intent recognition branch, and the probability distribution of the user control intent is output through the classification head. The category with the highest probability is selected as the final user control intent.

6. The modular speech recognition method of claim 1, wherein, The monitoring of the real-time load status of the communication bus and the calculation of the bus idle time window include: Real-time sampling of the level signals on the communication bus, and statistical analysis of the data frame density per unit time; If the data frame density is lower than a preset idle threshold, a timer is started to record the duration of no continuous data frame transmission. When the duration of the continuous absence of data frame transmission is greater than or equal to the minimum frame interval specified in the protocol, the current moment is determined as the starting point of the bus idle time window; Calculate the duration of the bus idle time window. If the duration is greater than the transmission time required to encapsulate the control instruction data packet, it is determined to be a valid injection opportunity.

7. A modular speech recognition system characterized by, include: The data acquisition module is used to acquire the original service data transmitted by the controlled device and the current voice signal collected by the user terminal based on the first communication segment and the second communication segment connected in series on the communication bus of the controlled device. The feature extraction module is used to perform real-time parsing and feature extraction on the original business data to obtain the device operating status sequence; The semantic classification module is used to perform semantic classification on the device operation state sequence to obtain corresponding state labels. The state labels include at least emergency fault category, normal operation category and silent ignore category. The emergency fault category corresponds to a high priority broadcast strategy, the normal operation category corresponds to a low priority broadcast strategy, and the silent ignore category corresponds to a no broadcast strategy. The broadcast output module constructs a status broadcast queue based on the status tags and preset dynamic environment categories, filters out target status information from the status broadcast queue, and synthesizes the target status information into a voice broadcast signal for output. The feature capture module is used to capture the voiceprint features of the current speech signal to obtain the current speech feature vector; The intent parsing module is used to input the current speech feature vector into a pre-trained instruction recognition model to parse and obtain the user's control intent; The intent encapsulation module is used to monitor the real-time load status of the communication bus, calculate the bus idle time window, and encapsulate the user control intent into a control instruction data packet that conforms to the communication protocol of the controlled device when the current time meets the bus idle time window. The device control module is used to inject the control command data packet into the communication bus to control the controlled device to perform corresponding actions.

8. The modular speech recognition system of claim 7, wherein, The step of constructing a status broadcast queue based on the status label and a preset dynamic environment category includes: The current dynamic environment category is obtained in real time, and the dynamic environment category includes at least a high noise interference environment, a low noise quiet environment, and an unattended silent environment. Traverse the sequence of device operating statuses to be broadcast and read the status label corresponding to each sequence. If the status label is an emergency fault type of device operating status sequence, ignore the dynamic environment category, mark the status label as the highest priority instant broadcast item, and insert it into the status broadcast queue. If the status label is a sequence of device operating statuses in the normal operation category and the dynamic environment category is a low-noise quiet environment, then the status label is marked as a normal priority delayed broadcast item and added to the status broadcast queue in chronological order. If the status label is a device operating status sequence of the normal operation type and the dynamic environment category is a high noise interference environment, then it is marked as a low priority cache item and temporarily stored in the cache area without entering the status broadcast queue, until the dynamic environment category is detected to switch to a low noise quiet environment before being moved into the queue. If the status label is a sequence of device operating statuses in the normal operating category and the dynamic environment category is an unattended silent environment, then its status label is dynamically updated to the silent ignore category and discarded from the status broadcast queue.

9. An electronic device, comprising: The electronic device includes: At least one processor; Memory; At least one application, wherein the at least one application is stored in memory and configured to be executed by at least one processor, the at least one application being configured to: perform a modular speech recognition method according to any one of claims 1-6.

10. A computer-readable storage medium, characterized in that, include: The computer program is stored and can be loaded by a processor and executed as described in any one of claims 1-6.

Images