Speech recognition method and device

Abstract

The invention discloses a voice recognition method and device, and the method comprises the steps: obtaining vehicle driving information, external environment perception information and a semantic text, and enabling the semantic text to be a text content obtained after processing the voice of a user; carrying out fusion processing on the vehicle driving information and the external environment perception information to generate a scene label which is used for describing a target scene in a current environment; and according to the semantic text and the scene label, determining a real target instruction of the user and an execution risk corresponding to the target instruction. Thus, by fusing the multi-dimensional driving state information, the recognition accuracy, safety and interaction continuity of the voice instruction can be improved, and misoperation or safety risks caused by scene influences are avoided.

Description

Technical Field

[0001] This invention relates to the field of speech recognition technology, and in particular to a speech recognition method and apparatus. Background Art

[0002] With the development of intelligent connected vehicles, voice interaction, as one of the core methods of in-vehicle human-machine interaction, is widely used in functions such as navigation control, air conditioning adjustment, and multimedia operation. However, in actual use, the complex and ever-changing driving conditions of vehicles can lead to a decrease in voice recognition accuracy. For example, at high speeds, in-vehicle noise (such as engine and wind noise) and user distraction can cause a decrease in voice recognition accuracy. Furthermore, some recognized voice commands (such as "open the window" in a high-speed scenario or "play video" while driving) may be limited by the scenario, potentially leading to safety hazards.

[0003] Since most existing in-vehicle voice systems only recognize intent based on voice content and lack the ability to comprehensively judge the current driving scenario of the vehicle, this can lead to semantic misunderstandings, interaction failures, or potential operational risks.

[0004] Therefore, how to improve the continuous dialogue control rate and safety of in-vehicle voice systems in real-world environments is a problem we need to solve. Summary of the Invention

[0005] In view of the above problems, the present invention provides a voice recognition method and device, which improves the recognition accuracy, security and interaction continuity of voice commands by integrating multi-dimensional driving status information, and avoids misoperation or safety risks caused by scene influence.

[0006] According to a first aspect of the present invention, a speech recognition method is provided, comprising: The system acquires vehicle driving information, external environment perception information, and semantic text, where semantic text refers to the text content obtained after processing the user's voice. The vehicle driving information and the external environment perception information are fused together to generate scene tags, which are used to describe the target scene in the current environment. Based on the semantic text and the scene tags, determine the user's actual target instruction and the execution risk corresponding to the target instruction.

[0007] Optionally, the vehicle driving information includes vehicle location information, navigation information, vehicle speed, and driving mode; the external environment perception information includes weather information, light intensity, ambient temperature, and ambient noise. The step of fusing the vehicle driving information with the external environment perception information to generate scene tags includes: Key features are extracted from the vehicle location information, navigation information, vehicle speed, driving mode, weather information, illumination, ambient temperature, and ambient noise. For each extracted key feature, compare that key feature with the rules in the rule base; Based on the comparison results, determine the attribute information corresponding to this key feature; The scene label is obtained based on the attribute information of each of the key features.

[0008] Optionally, determining the user's actual target instruction and the corresponding execution risk based on the semantic text and the scene tags includes: The trained intent classification model is used to process the semantic text and output user commands. Based on the scene tags, the user commands are completed and corrected to obtain the user's actual target commands; Obtain the risk assessment indicators corresponding to the scenario tags; Based on the aforementioned risk assessment indicators, the target instruction is scored, and the execution risk corresponding to the target instruction is determined.

[0009] Optionally, the step of processing the semantic text using the trained intent classification model and outputting user instructions includes: The semantic text is then subjected to structuring processing to obtain the processed structured text; The structured text is input into the trained intent classification model, which outputs user commands.

[0010] Optionally, the intent classification model is a deep learning model based on a self-attention mechanism.

[0011] Optionally, the method further includes: If the execution risk is low-level, then the target instruction is executed; If the execution risk is medium risk, a prompt message is generated, and after receiving a confirmation instruction for the prompt message within a preset time, the target instruction is executed. If the execution risk is high risk, then the execution of the target instruction is prohibited.

[0012] Optionally, before acquiring vehicle driving information, external environment perception information, and semantic text, the method further includes: Collect user voice; The user's voice is subjected to speech recognition to obtain the semantic text corresponding to the voice.

[0013] According to a second aspect of the present invention, a voice recognition device is provided, comprising: The acquisition module is used to acquire vehicle driving information, external environment perception information, and semantic text, wherein the semantic text refers to the text content obtained after processing the user's voice. The fusion processing module is used to fuse the vehicle driving information with the external environment perception information to generate scene tags, which are used to describe the target scene in the current environment. The determination module is used to determine the user's actual target instruction and the execution risk corresponding to the target instruction based on the semantic text and the scene label.

[0014] According to a third aspect of the present invention, a controller is provided, comprising: a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor performs the aforementioned speech recognition method.

[0015] According to a fourth aspect of the present invention, a vehicle is provided, the vehicle including a vehicle body and a controller installed in the vehicle body, wherein the controller performs the aforementioned voice recognition method.

[0016] The above-described one or more technical solutions in the embodiments of this specification have at least the following technical effects: This specification provides a speech recognition method and apparatus that acquires vehicle driving information, external environment perception information, and semantic text, whereby the semantic text refers to the text content obtained after processing the user's speech. The vehicle driving information and the external environment perception information are fused to generate scene tags, which describe the target scene in the current environment. Based on the semantic text and the scene tags, the user's actual target command and the corresponding execution risk are determined. Thus, by fusing multi-dimensional driving status information, the accuracy and security of voice command recognition are improved, avoiding misoperation or safety risks caused by scene influences.

[0017] The above description is only an overview of the technical solution of the present invention. In order to more clearly understand the technical means of the present invention, it can be implemented in accordance with the contents of the specification. In order to make the above and other purposes, features and advantages of the present invention more obvious and easy to understand, the specific implementation methods of the present invention are specifically listed below. Attached Figure Description

[0018] Various other advantages and benefits will become apparent to those skilled in the art upon reading the following detailed description of preferred embodiments. The accompanying drawings are for illustrative purposes only and are not intended to limit the invention. Furthermore, the same reference numerals denote the same parts throughout the drawings. In the drawings: Figure 1 A flowchart of a speech recognition method according to an embodiment of the present invention is shown.

[0019] Figure 2 A block diagram of a speech recognition device according to an embodiment of the present invention is shown. DETAILED DESCRIPTION

[0020] To make the objectives, technical solutions, and advantages of the embodiments of the present invention clearer, the technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. The components of the embodiments of the present invention described and shown in the accompanying drawings can generally be arranged and designed in various different configurations.

[0021] Therefore, the following detailed description of the embodiments of the invention provided in the accompanying drawings is not intended to limit the scope of the claimed invention, but merely to illustrate selected embodiments of the invention. All other embodiments obtained by those skilled in the art based on the embodiments of the invention without inventive effort are within the scope of protection of the invention.

[0022] It should be noted that similar reference numerals and letters denote similar items in the following drawings, and therefore, once an item is defined in one drawing, it does not need to be further defined or explained in subsequent drawings.

[0023] In the description of this invention, it should also be noted that, unless otherwise explicitly specified and limited, the terms "set," "install," "connect," and "link" should be interpreted broadly. For example, they can refer to a fixed connection, a detachable connection, or an integral connection; they can refer to a mechanical connection or an electrical connection; they can refer to a direct connection or an indirect connection through an intermediate medium; and they can refer to the internal connection of two components. Those skilled in the art can understand the specific meaning of the above terms in this invention based on the specific circumstances.

[0024] With the development of intelligent connected vehicles, voice interaction, as one of the core methods of in-vehicle human-machine interaction, is widely used in functions such as navigation control, air conditioning adjustment, and multimedia operation. However, in actual use, the complex and ever-changing driving conditions of vehicles can lead to a decrease in voice recognition accuracy. For example, at high speeds, in-vehicle noise (such as engine and wind noise) and user distraction can cause a decrease in voice recognition accuracy. Furthermore, some recognized voice commands (such as "open the window" in a high-speed scenario or "play video" while driving) may be limited by the scenario, potentially leading to safety hazards.

[0025] Since most existing in-vehicle voice systems only recognize intent based on voice content and lack the ability to comprehensively judge the current driving scenario of the vehicle, this can lead to semantic misunderstandings, interaction failures, or potential operational risks.

[0026] Based on the above, this embodiment of the invention provides a voice recognition method that improves the accuracy, security, and interactive coherence of voice commands by integrating multi-dimensional driving status information, thereby avoiding misoperation or safety risks caused by scene influences.

[0027] Specifically, embodiments of the present invention provide a speech recognition method, combined with Figure 1 The flowchart shown illustrates that the speech recognition method includes steps 101 to 103: Step 101: Obtain vehicle driving information, external environment perception information, and semantic text, wherein the semantic text refers to the text content obtained after processing the user's voice. Step 102: The vehicle driving information and the external environment perception information are fused together to generate scene tags, which are used to describe the target scene in the current environment; Step 103: Based on the semantic text and the scene tags, determine the user's actual target instruction and the execution risk corresponding to the target instruction.

[0028] Vehicle driving information includes vehicle location information, navigation information, vehicle speed, and driving mode. Location information refers to the latitude and longitude coordinates obtained through the onboard GPS module, combined with high-precision maps to determine the vehicle's specific location within the road network (e.g., highways, urban arterial roads, tunnels). Navigation information refers to data extracted from the navigation system, such as destination, remaining distance, estimated arrival time, and road conditions ahead (e.g., congestion, construction), used to determine the user's travel intentions. Vehicle speed refers to real-time data collected from vehicle wheel speed sensors or CAN bus, facilitating the differentiation between low-speed (≤30km / h), medium-speed (30-60km / h), and high-speed (≥60km / h) scenarios. Driving mode refers to identifying the current vehicle operating status, such as autonomous driving (Level 2-5), manual driving, and parking assistance.

[0029] External environmental perception information includes weather information, illumination, ambient temperature, and ambient noise. Weather information refers to the fusion of image recognition from in-vehicle cameras (e.g., identifying raindrops and snow), meteorological sensor data (humidity, air pressure changes), and cloud-based meteorological services to determine weather types such as sunny, rainy, snowy, and foggy. Illumination refers to detecting ambient light intensity through photosensors to distinguish between daytime, nighttime, and tunnel scenarios. Ambient temperature refers to collecting data from in-vehicle temperature sensors and combining it with user historical preferences (e.g., temperature adjustment habits) to optimize command understanding. Ambient noise refers to using a microphone array to collect the in-vehicle noise spectrum and using spectral subtraction or deep learning algorithms to separate human voice from background noise (e.g., engine noise, wind noise, rain noise) to improve the robustness of speech recognition.

[0030] In this embodiment, the acquisition and preliminary processing of user voice is a preliminary step in obtaining semantic text, and the specific steps may include: Collect user voice; The user's voice is subjected to speech recognition to obtain the semantic text corresponding to the voice.

[0031] User voice refers to the voice emitted by the user in the in-vehicle environment. User voice acquisition is the starting point of the entire voice interaction process, and its core lies in using appropriate hardware to convert the user's voice signals into processable electrical or digital signals.

[0032] Then, speech recognition is performed on the user's voice to obtain semantic text. Speech recognition is the process of converting the collected user speech into text content.

[0033] In an optional embodiment, the step of fusing the vehicle driving information with the external environment perception information to generate scene labels includes: Key features are extracted from the vehicle location information, navigation information, vehicle speed, driving mode, weather information, illumination, ambient temperature, and ambient noise. For each extracted key feature, compare that key feature with the rules in the rule base; Based on the comparison results, determine the attribute information corresponding to this key feature; The scene label is obtained based on the attribute information of each of the key features.

[0034] The target scenario refers to a scenario that changes dynamically in real time under the current environment. Vehicle driving information and external environment perception information encompass multi-dimensional data. To improve processing efficiency and accuracy, key feature extraction is necessary. This means extracting key features, such as vehicle speed, ambient noise, and weather type, from the collected raw data (i.e., vehicle driving information and external environment perception information). Algorithms such as moving average filtering are used to smooth continuously changing data such as vehicle speed and ambient temperature, removing abnormal fluctuations; discrete data (such as weather type and driving mode) are encoded.

[0035] The extracted key features are compared one by one with the rules in the rule base. Taking the "high-speed driving" rule as an example, the extracted vehicle speed is compared with the preset high-speed threshold of 80km / h. If the vehicle speed is greater than 80km / h, the output attribute information is determined to be high-speed driving based on the comparison result. Taking the "driving in rainy weather" rule as an example, it is checked whether the extracted weather type is rain. If the weather type is rain, the output attribute information is determined to be rainy based on the comparison result.

[0036] Then, the scene label is determined based on the attribute information of each key feature. The number of attribute information in the scene label is not fixed. It should be noted that in this embodiment, the scene labels are a preset finite number of labels, and each scene label corresponds to several attribute information.

[0037] After determining the scene labels, intent analysis is performed in conjunction with semantic text. This embodiment utilizes a trained intent classification model to process the semantic text and output user commands. Prior to this, this embodiment processes the semantic text, specifically including the following steps: The semantic text is then subjected to structuring processing to obtain the processed structured text; The structured text is input into the trained intent classification model, which outputs user commands.

[0038] Semantic text structuring involves breaking down user speech into natural language text, which is then understood by machines, into structured data. Natural language is flexible, but difficult for computers to analyze directly. Structuring extracts key information and labels semantic relationships, transforming semantic text into a clear logical framework, facilitating subsequent intent classification and command judgment. For example, the phrase "set the air conditioner temperature to 26 degrees" can be clearly structured to specify the command type (adjustment), the target (air conditioner), the parameter (temperature), and the specific value (26 degrees).

[0039] In practice, the process begins by using word segmentation technology to break down the text into individual words or phrases, while simultaneously tagging parts of speech to distinguish between nouns, verbs, adjectives, etc. Next, named entity recognition (NER) technology is used to locate key entities in the text, such as device names, locations, and numerical values. Finally, syntactic analysis is performed to clarify the grammatical relationships between words, identify structures such as subject-verb-object, modifier-adverb-complement, etc., and construct structured text that includes instruction types, operation objects, parameters, and numerical values.

[0040] In this embodiment, the intent classification model identifies the core intent of user commands based on the structured text. The intent classification model is typically a deep learning model based on a self-attention mechanism. Fine-tuning is performed on the in-vehicle command dataset to improve the ability to recognize specialized vocabulary and command patterns.

[0041] In practice, multi-dimensional features are typically extracted from structured text, including entity type features (entity categories such as devices, parameters, and actions), semantic relationship features (syntactic and semantic associations between words), statistical features (word occurrence frequency, co-occurrence frequency, etc.), and text embedding features generated by pre-trained language models (such as BERT), to provide rich information input for intent classification.

[0042] Deep learning model architectures, such as BiLSTM and Transformer, or fine-tuning pre-trained language models (such as BERT and RoBERTa), can be used to build intent classification models. The model is trained using a large amount of labeled structured text and its corresponding instruction labels. The model parameters are adjusted through optimization algorithms to enable the model to accurately learn the distribution patterns of different intent categories in the feature space.

[0043] The structured text to be recognized is input into the trained intent classification model. The model calculates the probability value of the text belonging to each predefined intent category through forward propagation. The intent category with the highest probability is selected as the recognition result, and the intent label and corresponding confidence score are output. At the same time, the content to be filled in the key semantic slots in the user command, such as the operation entity and operation parameters, is also output.

[0044] After receiving a user instruction, if it is detected that the user instruction is missing information, the user instruction is completed and corrected according to the scene label to obtain the user's true target instruction. Obtain the risk assessment indicators corresponding to the scenario tags; Based on the aforementioned risk assessment indicators, the target instruction is scored, and the execution risk corresponding to the target instruction is determined.

[0045] It should be explained that during in-vehicle voice interaction, user commands often lack information due to brief descriptions or environmental interference, affecting the accuracy and safety of command execution. This problem can be effectively solved by combining scene tags for command completion and correction, along with a risk assessment mechanism.

[0046] Semantic analysis and instruction structure inspection can be used to identify missing information in user instructions. Specific detection dimensions include: missing key entities, ambiguous logical relationships, and incompatible scenario prompts.

[0047] Key entities missing: Check whether the user command is missing key entities such as the object of operation (e.g., only saying "open" without specifying "air conditioning" or "window"), operation parameters (e.g., "adjust temperature" without mentioning specific values), and destination information (e.g., "navigate to" without specifying a location).

[0048] Vague logical relationships: Analyze whether the logical connections between the elements in the user command are clear. For example, "switch mode" does not specify which mode to switch to, or "alert when approaching" does not specify the target location of "approaching".

[0049] Scene incompatibility warning: When a user's command contradicts the current scene label but no specific correction direction is clear, it is judged as potential information missing. For example, if a user says "open the sunroof" while driving at high speed, the vehicle needs to further confirm the user's intention.

[0050] If missing information is detected in a user command, it is necessary to complete and correct the user command based on the scene tag. There are multiple ways to complete and correct the user command.

[0051] In one alternative implementation, user history interaction data can be combined with scenario tags to match user habits. For example, if a user repeatedly issues the command "turn on the air conditioner" in a scenario of "rainy day + city road + congested traffic", and the command is completed as "turn on the air conditioner and set it to recirculation", the vehicle will prioritize completing the command according to this habit when it detects similar scenario tags and missing user commands.

[0052] In one alternative implementation, environmental information can be used for derivation. Command details are supplemented based on external environmental perception information. If the scene label displays "high temperature" and the user command is "turn on the air conditioner," the vehicle can automatically complete it as "turn on the air conditioner and set the temperature to 26 degrees Celsius." If the scene label includes "tunnel + insufficient lighting," and the user says "brighter," it is completed as "brighten the instrument panel and central control screen."

[0053] In one alternative implementation, the command can be completed using attribute information about navigation and driving status in the scene label. When the navigation shows a service area 500 meters ahead, and the user says "stop," the vehicle completes the command as "navigate to the service area ahead and stop." If the vehicle is traveling at high speed and the user says "play music," the command completes the command as "play light music to relieve driving fatigue."

[0054] After completing and correcting the user command, the actual target command can be obtained. If the user command is complete, it is directly used as the actual target command.

[0055] This embodiment will also score the target instruction based on the risk assessment indicators and determine the execution risk corresponding to the target instruction.

[0056] It should be noted that the risk assessment indicators are pre-set, and each scenario label has corresponding risk assessment indicators.

[0057] The construction of risk assessment indicators mainly includes driving safety risks, environmental adaptability risks, operational risks, and user preference risks.

[0058] The main safety risk indicators for driving include the following: Attention Distraction Level: This measures the degree to which a driver's visual, auditory, and manual attention is occupied during the execution of a command. For example, when executing the command "search and play a specific artist's album," the driver needs to manually enter search terms and browse the list multiple times, causing their gaze to be off the road for an extended period, resulting in a higher score for this indicator; while simple voice commands such as "turn up the volume" have little impact on attention distraction and thus a lower score.

[0059] Operational Interference Probability: Assessing the likelihood that command execution will cause driver error or judgment deviation. Complex command combinations (such as simultaneously adjusting navigation and air conditioning settings) and operations requiring multiple steps increase the probability of driver confusion, resulting in a higher score for this indicator; clear, single-step commands, such as "open the sunroof," have a low operational interference probability.

[0060] Emergency Response Delay: This calculates the time from issuing the target instruction to completing the response, and the impact of this time on handling unexpected road conditions (such as emergency braking or pedestrian avoidance). If the driver is unable to pay timely attention to road conditions or responds too slowly during the execution of the target instruction, potentially delaying operations in an emergency, this indicator score will increase significantly.

[0061] The main environmental adaptability risk indicators include the following: Weather compatibility: This assesses how well the command execution matches the current weather conditions. Executing the "open the window" command in heavy rain could easily cause rainwater to enter the vehicle and may also distract the driver from dealing with flooding, making this a high-risk indicator; while executing the same command in sunny weather carries a lower risk.

[0062] Light Adaptability: This considers the visibility and safety of commands under current lighting conditions. Executing the "reduce screen brightness" command at night meets environmental requirements and carries low risk; however, executing the "turn off headlights" command in dimly lit tunnels would severely impair visibility, significantly increasing the light adaptability risk score.

[0063] Noise interference level: This assesses the impact of environmental noise on the accuracy of voice command recognition and the driver's ability to hear feedback. When the vehicle is traveling at high speed, background noise such as wind noise and tire noise is high, which may lead to voice recognition errors or the driver being unable to hear the system's voice prompts, thus increasing the score of this indicator; when traveling at low speed or when the interior of the vehicle is quiet, the noise interference level is low.

[0064] The main operational risk indicators include the following: Device state conflict: Detects whether the target command contradicts the current operating state of the vehicle's devices. If the vehicle's air conditioning is already on, issuing another "turn on the air conditioning" command, while not causing actual harm, creates operational redundancy. If, while the autonomous driving function is activated, a command that might interfere with the autonomous driving system is executed (such as a simulated command to manually and significantly adjust the steering wheel angle), the risk of device state conflict is extremely high.

[0065] Resource Usage: This analyzes the resource usage (such as CPU, memory, and network bandwidth) of the vehicle control system and entertainment system during the execution of target instructions. Playing high-definition videos and running large navigation maps consume significant system resources, potentially causing other vehicle functions to run slowly or even freeze, increasing the risk of system resource overload.

[0066] The main contents of user preference risk indicators include: Historical Behavior Conflict: Compare the fit between the current target command and the user's historical operating habits. If a user often listens to news broadcasts during their commute, but this time issues a command to "play heavy metal music," this conflict with their historical preferences, increasing the risk of this indicator. Commands that align with historical habits, such as "navigate home," have a lower risk.

[0067] Likelihood of accidental triggering: Assess the degree of semantic ambiguity of the command and the probability of confusion with similar commands. The voice command "open sunroof" and "open car window" are similar in pronunciation and are easily misidentified in noisy environments, resulting in a high probability of accidental triggering; commands with clear and unique semantics, such as "open trunk," have a low risk of accidental triggering.

[0068] When calculating risk scores, the risk score of the target instruction can be calculated using a weighted summation formula based on risk assessment indicators.

[0069] For example, the scene tags are (low speed, city road, daytime, sunny), and the user command is "play Zhang San's song." The driving safety risk score is 0.14 with a weight of 0.4; the environmental adaptation risk is 0.1 with a weight of 0.3; the operational risk is 0.12 with a weight of 0.1; and the user preference risk is 0.1 with a weight of 0.2. The final calculated risk score is 0.124.

[0070] Specifically, a risk score less than 0.25 indicates a low level of execution risk; a risk score between 0.25 and 0.75 indicates a medium level of execution risk; and a risk score greater than 0.75 indicates a high level of execution risk.

[0071] If the execution risk is low-level, then the target instruction is executed; If the execution risk is medium risk, a prompt message is generated, and after receiving a confirmation instruction for the prompt message within a preset time, the target instruction is executed; the prompt message can be a voice prompt or a pop-up prompt. If the execution risk is high risk, then the execution of the target instruction is prohibited.

[0072] In summary, the speech recognition method provided in this specification acquires vehicle driving information, external environment perception information, and semantic text, whereby the semantic text refers to the text content obtained after processing the user's speech. The vehicle driving information and the external environment perception information are fused to generate scene tags, which describe the target scene in the current environment. Based on the semantic text and the scene tags, the user's actual target command and the corresponding execution risk are determined. Thus, by fusing multi-dimensional driving status information, the accuracy, security, and interactive coherence of voice command recognition are improved, avoiding misoperation or safety risks caused by scene-related factors.

[0073] Based on the same inventive concept, combined with Figure 2 As shown, embodiments of the present invention also provide a voice recognition device, comprising: The acquisition module is used to acquire vehicle driving information, external environment perception information, and semantic text, wherein the semantic text refers to the text content obtained after processing the user's voice. The fusion processing module is used to fuse the vehicle driving information with the external environment perception information to generate scene tags, which are used to describe the target scene in the current environment. The determination module is used to determine the user's actual target instruction and the execution risk corresponding to the target instruction based on the semantic text and the scene label.

[0074] Optionally, the vehicle driving information includes vehicle location information, navigation information, vehicle speed, and driving mode; the external environment perception information includes weather information, light intensity, ambient temperature, and ambient noise. The step of fusing the vehicle driving information with the external environment perception information to generate scene tags includes: Key features are extracted from the vehicle location information, navigation information, vehicle speed, driving mode, weather information, illumination, ambient temperature, and ambient noise. For each extracted key feature, compare that key feature with the rules in the rule base; Based on the comparison results, determine the attribute information corresponding to this key feature; The scene label is obtained based on the attribute information of each of the key features.

[0075] Optionally, determining the user's actual target instruction and the corresponding execution risk based on the semantic text and the scene tags includes: The trained intent classification model is used to process the semantic text and output user commands. Based on the scene tags, the user commands are completed and corrected to obtain the user's actual target commands; Obtain the risk assessment indicators corresponding to the scenario tags; Based on the aforementioned risk assessment indicators, the target instruction is scored, and the execution risk corresponding to the target instruction is determined.

[0076] Optionally, the step of inputting the speech text into the trained intent classification model and outputting user instructions includes: The semantic text is then subjected to structuring processing to obtain the processed structured text; The structured text is input into the trained intent classification model, which outputs user commands.

[0077] Optionally, the intent classification model is a deep learning model based on a self-attention mechanism.

[0078] Optionally, the method further includes: If the execution risk is low-level, then the target instruction is executed; If the execution risk is medium risk, a prompt message is generated, and after receiving a confirmation instruction for the prompt message within a preset time, the target instruction is executed. If the execution risk is high risk, then the execution of the target instruction is prohibited.

[0079] Optionally, before acquiring vehicle driving information, external environment perception information, and semantic text, the method further includes: Collect user voice; The user's voice is subjected to speech recognition to obtain the semantic text corresponding to the voice.

[0080] In summary, the speech recognition device provided in this specification acquires vehicle driving information, external environment perception information, and semantic text, whereby the semantic text refers to the text content obtained after processing the user's speech. The vehicle driving information and the external environment perception information are fused to generate scene tags, which describe the target scene in the current environment. Based on the semantic text and the scene tags, the user's actual target command and the corresponding execution risk are determined. Thus, by fusing multi-dimensional driving status information, the accuracy, security, and interactive coherence of speech command recognition are improved, avoiding misoperation or safety risks caused by scene-related factors.

[0081] Those skilled in the art will understand that, for the sake of convenience and brevity, the specific working process of the speech recognition device described above can be referred to the corresponding process in the aforementioned method, and will not be elaborated further here.

[0082] Based on the same inventive concept, embodiments of the present invention also provide a controller, which includes a voice recognition device, a memory, a processor, and a communication unit. The memory stores machine-readable instructions that can be executed by the processor. When the controller is running, the processor and the memory communicate through a bus, and the processor executes the machine-readable instructions and performs the voice recognition method.

[0083] The memory, processor, and communication unit are electrically connected directly or indirectly to achieve signal transmission or interaction. For example, these components can be electrically connected to each other through one or more communication buses or signal lines. The voice recognition device includes at least one software functional module that can be stored in the memory in the form of software or firmware. The processor is used to execute the executable module (e.g., the software functional module or computer program included in the voice recognition device) stored in the memory.

[0084] The memory can be, but is not limited to, Random Access Memory (RAM), Read Only Memory (ROM), Programmable Read-Only Memory (PROM), Erasable Programmable Read-Only Memory (EPROM), Electrically Erasable Programmable Read-Only Memory (EEPROM), etc.

[0085] In some embodiments, the processor is used to perform one or more functions described in this embodiment. In some embodiments, the processor may include one or more processing cores (e.g., a single-core processor (S) or a multi-core processor (S)).

[0086] In this embodiment, the memory is used to store the program, and the processor is used to execute the program after receiving the execution instruction. The process definition method disclosed in any implementation of this embodiment can be applied to the processor or implemented by the processor.

[0087] The communication unit is used to establish communication connections between the controller and other devices via the network, and to send and receive data via the network.

[0088] Those skilled in the art will understand that, for the sake of convenience and brevity, the specific working process of the controller described above can be referred to the corresponding process in the aforementioned method, and will not be elaborated further here.

[0089] Based on the same inventive concept, embodiments of the present invention also provide a vehicle, including a vehicle body and a controller installed in the vehicle body, the controller being used to implement the aforementioned voice recognition method.

[0090] Those skilled in the art will understand that, for the sake of convenience and brevity, the specific working process of the vehicle controller described above can be referred to the corresponding process in the aforementioned method, and will not be elaborated further here.

[0091] The above are merely various embodiments of the present invention, but the scope of protection of the present invention is not limited thereto. Any variations or substitutions that can be easily conceived by those skilled in the art within the technical scope disclosed in the present invention should be included within the scope of protection of the present invention. Therefore, the scope of protection of the present invention should be determined by the scope of the claims.

Claims

1. A speech recognition method, characterized in that, include: The system acquires vehicle driving information, external environment perception information, and semantic text, where semantic text refers to the text content obtained after processing the user's voice. The vehicle driving information and the external environment perception information are fused together to generate scene tags, which are used to describe the target scene in the current environment. Based on the semantic text and the scene tags, determine the user's actual target instruction and the execution risk corresponding to the target instruction.

2. The method according to claim 1, characterized in that, The vehicle driving information includes vehicle location information, navigation information, vehicle speed, and driving mode; the external environment perception information includes weather information, light intensity, ambient temperature, and ambient noise. The step of fusing the vehicle driving information with the external environment perception information to generate scene tags includes: Key features are extracted from the vehicle location information, navigation information, vehicle speed, driving mode, weather information, illumination, ambient temperature, and ambient noise. For each extracted key feature, compare that key feature with the rules in the rule base; Based on the comparison results, determine the attribute information corresponding to this key feature; The scene label is obtained based on the attribute information of each of the key features.

3. The method according to claim 1, characterized in that, The step of determining the user's actual target instruction and the corresponding execution risk based on the semantic text and the scene tags includes: The trained intent classification model is used to process the semantic text and output user commands. Based on the scene tags, the user commands are completed and corrected to obtain the user's actual target commands; Obtain the risk assessment indicators corresponding to the scenario tags; Based on the aforementioned risk assessment indicators, the target instruction is scored, and the execution risk corresponding to the target instruction is determined.

4. The method according to claim 3, characterized in that, The step of processing the semantic text using the trained intent classification model and outputting user instructions includes: The semantic text is then subjected to structuring processing to obtain the processed structured text; The structured text is input into the trained intent classification model, which outputs user commands.

5. The method according to claim 4, characterized in that, The intent classification model is a deep learning model based on a self-attention mechanism.

6. The method according to any one of claims 1-3, characterized in that, The method further includes: If the execution risk is low-level, then the target instruction is executed; If the execution risk is medium risk, a prompt message is generated, and after receiving a confirmation instruction for the prompt message within a preset time, the target instruction is executed. If the execution risk is high risk, then the execution of the target instruction is prohibited.

7. The method according to claim 1, characterized in that, Before acquiring vehicle driving information, external environment perception information, and semantic text, the method further includes: Collect user voice; The user's voice is subjected to speech recognition to obtain the semantic text corresponding to the voice.

8. A voice recognition device, characterized in that, include: The acquisition module is used to acquire vehicle driving information, external environment perception information, and semantic text, wherein the semantic text refers to the text content obtained after processing the user's voice. The fusion processing module is used to fuse the vehicle driving information with the external environment perception information to generate scene tags, which are used to describe the target scene in the current environment. The determination module is used to determine the user's actual target instruction and the execution risk corresponding to the target instruction based on the semantic text and the scene label.

9. A controller, characterized in that, The controller includes: a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the computer program to implement the speech recognition method according to any one of claims 1-7.

10. A vehicle, characterized in that, The vehicle includes a vehicle body and a controller installed in the vehicle body, wherein the controller performs the voice recognition method according to any one of claims 1-7.

Images