Local and cloud-based mixed speech recognition response system and method

Abstract

The invention discloses a local and cloud-based mixed speech recognition response system and method, and particularly relates to the technical field of mixed speech recognition, and the method comprises the steps: obtaining a signal fluctuation coefficient of vehicle equipment in a T time period, and judging the network state of the vehicle equipment in the T time period based on the signal fluctuation coefficient, generating a corresponding voice recognition instruction according to the network state; the voice recognition instruction comprises a cloud recognition instruction, a local recognition instruction and a mixed recognition instruction; receiving a local recognition instruction, obtaining local voice feature data of the ith voice to be recognized, inputting the local voice feature data into a pre-constructed local voice recognition model to obtain corresponding local control data, and making a response by the vehicle machine equipment according to the local control data; under the condition of network fluctuation, the mixed identification instruction allows the local control data and the cloud control data to be combined for use, and the accuracy and the response speed of the control data are optimized.

Description

Technical Field

[0001] The present invention relates to the technical field of hybrid speech recognition, and more specifically, to a hybrid speech recognition response system and method based on local and cloud. Background Art

[0002] With the continuous development of intelligent in-vehicle systems, voice interaction technology has become an important part of in-vehicle systems. Traditional voice recognition systems mainly rely on cloud computing, which provides powerful computing power and rich vocabulary support, and can handle complex voice input. However, this network-dependent approach will affect the response speed and recognition accuracy when the network environment is poor, and even cause the system to be unavailable. To solve this problem, local voice recognition has gradually been introduced to improve the stability and instant response capabilities of offline use. There are limitations to relying solely on local voice recognition, such as limited computing resources and a relatively small vocabulary, which will affect the recognition of complex commands and the processing of frequently updated content.

[0003] Among the existing methods, for example, the Chinese patent application with publication number CN108305620A discloses a local cloud hybrid active interactive speech recognition system that relies on big data, including a speech input module, a dialogue engine, an embedded speech recognition module, an embedded natural semantic understanding module, a cloud language recognition module, an arbitration module, and a cloud natural semantic understanding module. The speech input module collects speech information and transmits it to the embedded speech recognition module and the cloud language recognition module respectively. The cloud language recognition module is also connected to the cloud natural semantic understanding module, and the embedded speech recognition module is also connected to the embedded natural semantic understanding module. Although the above method can quickly identify and interact with the user, improving the recognition efficiency and interactive performance, research and application of the above method and the prior art found that the above method and the prior art have at least the following defects:

[0004] Relying on fixed embedded and cloud recognition module division of labor, lacks monitoring and adaptive adjustment of real-time network status. If the network status fluctuates or deteriorates, it may affect the response speed and accuracy of the cloud module, resulting in a decline in recognition and interaction experience.

[0005] To this end, the present invention provides a hybrid speech recognition response system and method based on local and cloud. Summary of the invention

[0006] In order to overcome the above-mentioned defects of the prior art, the present invention provides a hybrid speech recognition response system and method based on local and cloud to solve the problems raised in the above-mentioned background technology.

[0007] To achieve the above objectives, in a first aspect, the present invention provides a hybrid speech recognition response method based on local and cloud, comprising:

[0008] Obtaining the signal fluctuation coefficient of the vehicle equipment within the time period T, judging the network status of the vehicle equipment within the time period T based on the signal fluctuation coefficient, and generating corresponding voice recognition instructions according to the network status; the voice recognition instructions include cloud recognition instructions, local recognition instructions and hybrid recognition instructions;

[0009] Receive a local recognition instruction, obtain local voice feature data of the i-th voice to be recognized, input the local voice feature data into a pre-built local voice recognition model to obtain corresponding local control data, and the vehicle device responds according to the local control data;

[0010] Receive cloud recognition instructions, obtain cloud voice feature data of the i-th voice to be recognized, input the cloud voice feature data into a pre-built cloud voice recognition model to obtain corresponding cloud control data, and the vehicle equipment responds according to the cloud control data;

[0011] Receive the hybrid recognition instruction, obtain the confidence score of each local control data, and predict the first stability index of the cloud control data when the network is in a fluctuating state, and adjust and optimize the local control data according to the first stability index and the confidence score to obtain the hybrid control data. The vehicle equipment responds according to the hybrid control data.

[0012] Furthermore, the network status includes an online status, an offline status, and a network fluctuation status;

[0013] The method for obtaining the signal fluctuation coefficient of the vehicle equipment within the T time period includes:

[0014] Acquire network quality data, wherein the network quality data includes signal strength variance, network delay time variance, data throughput variance, and packet loss rate variance;

[0015] Formulate network quality data to obtain signal fluctuation coefficient ;

[0016] The method for judging the network status of the vehicle equipment within the T time period based on the signal fluctuation coefficient includes:

[0017] Preset fluctuation coefficient threshold, the fluctuation coefficient threshold includes and ,in ; Compare the signal fluctuation coefficient with the preset fluctuation coefficient threshold;

[0018] like > , then the network status of the vehicle equipment in the T time period is marked as the network status;

[0019] like , then the network status of the vehicle equipment in the T time period is marked as a network fluctuation state;

[0020] like , then the network status of the vehicle equipment within the T time period is marked as offline.

[0021] Furthermore, the method for generating a corresponding voice recognition instruction according to the network status includes:

[0022] When the network status of the vehicle equipment is marked as connected within the T time period, a cloud recognition instruction is generated according to the connected status;

[0023] When the network status of the vehicle equipment is marked as offline within the T time period, a local recognition instruction is generated according to the offline status;

[0024] When the network status of the vehicle equipment within the T time period is marked as a network fluctuation state, a hybrid identification instruction is generated according to the network fluctuation state.

[0025] Further, the local speech feature data includes keyword detection data and context state data;

[0026] The local speech recognition model is obtained by training with local speech training data; the local speech training data includes local speech feature data and corresponding local control data;

[0027] The method for generating the local speech recognition model specifically includes:

[0028] Acquire historical local voice training data, and divide the historical local voice training data into a local training set and a local test set; the historical local voice training data includes local voice feature data and its corresponding local control data;

[0029] Construct a first recognition network model, use the local speech feature data in the local training set as the input data of the first recognition network model, use the local control data in the local training set as the output data of the first recognition network model, train the first recognition network model to obtain an initial first recognition network model; use the local test set to verify the initial first recognition network model, and output the initial first recognition network model with a prediction error less than or equal to a preset error threshold as the local speech recognition model, wherein the first recognition network model is a deep neural network model or a deep belief network model.

[0030] Furthermore, the method for acquiring the local control data in the historical local speech training data includes:

[0031] Step a1: obtaining the local speech recognition coefficient of the i-th speech to be recognized;

[0032] The local speech recognition coefficient is obtained by collecting local speech feature data and performing normalization processing; the calculation formula is:

[0033] ;

[0034] In the formula, represents the local speech recognition coefficient, Indicates keyword detection data. Indicates keyword preset data, Represents contextual state data, Represents context state preset data, and is the weight factor, represents the natural exponential function;

[0035] Step a2: setting N local speech recognition coefficient intervals, and setting N local control data corresponding to the N local speech recognition coefficient intervals; each local speech recognition coefficient interval is bound if and only if there is one local control data associated with it;

[0036] Step a3: comparing the local speech recognition coefficient of the i-th speech to be recognized with each local speech recognition coefficient interval, and obtaining the local speech recognition coefficient interval into which the local speech recognition coefficient of the i-th speech to be recognized falls;

[0037] Step a4: according to the local speech recognition coefficient interval into which the local speech recognition coefficient of the i-th speech to be recognized falls, divide the i-th speech to be recognized into the corresponding local control data; and set i=i+1, and jump back to step a1;

[0038] Step a5: Repeat steps a1 to a4 until the loop ends when i=I, so that each of the to-be-recognized voices is matched with N local control data in sequence, where I is the total number of all interactive voices between the user and the vehicle-mounted device within the T time period.

[0039] Furthermore, the cloud voice feature data includes location data and semantic intent data; the cloud voice recognition model is obtained by training the cloud voice training data; the cloud voice training data includes cloud voice feature data and corresponding cloud control data;

[0040] The method for generating the cloud-based speech recognition model specifically includes:

[0041] Acquire historical cloud voice training data, and divide the historical cloud voice training data into a cloud voice recognition training set and a cloud voice recognition test set; the historical cloud voice training data includes cloud voice feature data and its corresponding cloud control data;

[0042] Construct a second recognition network model, use the cloud speech feature data in the cloud speech recognition training set as the input data of the second recognition network model, use the cloud control data in the cloud speech recognition training set as the output data of the second recognition network model, train the second recognition network model to obtain an initial second recognition network model; use the cloud speech recognition test set to verify the initial second recognition network model, and output the initial second recognition network model with a prediction error less than or equal to a preset error threshold as the cloud speech recognition model, wherein the second recognition network model is a deep neural network model or a deep belief network model.

[0043] Furthermore, the cloud control data in the historical cloud voice training data is obtained by:

[0044] Step b1: Obtain the cloud speech recognition coefficient of the i-th speech to be recognized;

[0045] The cloud speech recognition coefficient is obtained by collecting cloud speech feature data and normalizing it, and its calculation formula is:

[0046] ;

[0047] In the formula, represents the cloud-based speech recognition coefficient, Represents location data, Indicates location preset data, Represents semantic intent data, Represents semantic intent preset data, and is the weight factor, represents the natural exponential function;

[0048] Step b2: setting M cloud speech recognition coefficient intervals, and setting M cloud control data corresponding to the M cloud speech recognition coefficient intervals; each cloud speech recognition coefficient interval is bound if and only if there is one cloud control data associated with it;

[0049] Step b3: Compare the cloud speech recognition coefficient of the i-th speech to be recognized with each cloud speech recognition coefficient interval to obtain the cloud speech recognition coefficient interval into which the cloud speech recognition coefficient of the i-th speech to be recognized falls;

[0050] Step b4: according to the cloud speech recognition coefficient interval into which the cloud speech recognition coefficient of the i-th speech to be recognized falls, classify the i-th speech to be recognized into the corresponding cloud control data; and set i=i+1, and jump back to step b1;

[0051] Step b5: Repeat steps b1 to b4 until the loop ends when i=I, so that each of the to-be-recognized voices is matched with M cloud control data in sequence.

[0052] Furthermore, the method for obtaining the confidence score of the local control data includes:

[0053] Extracting the replacement frequency of each piece of local control data and the historical performance data of each piece of local control data, wherein the historical performance data includes the duration, the last running time and the error rate;

[0054] The historical performance data and replacement frequency of each piece of local control data are sequentially input into a pre-built mathematical calculation model to obtain a confidence score for each piece of local control data; the expression of the mathematical calculation model is: ; Where: Score the confidence of the e-th local control data, is the replacement frequency of the local control data in clause e, For the duration of local control data in Article e, The last running time of the local control data in item e, is the error rate of the local control data in the e-th item.

[0055] Furthermore, the method for predicting the first stability index of cloud control data in a network fluctuation state includes:

[0056] Extract the amount M of cloud control data and the signal fluctuation coefficient;

[0057] The amount M of cloud control data and the signal fluctuation coefficient are used as state performance data and input into a preconfigured state prediction model to obtain a first stability index of the cloud control data when the network is in a fluctuation state;

[0058] The method for generating the state prediction model comprises:

[0059] Acquire historical stable training data, and divide the historical stable training data into a stable state training set and a stable state test set; the historical stable training data includes state performance data and its corresponding first stability index;

[0060] Constructing a regression network model, taking the state performance data in the stable state training set as the input data of the regression network model, taking the first stability index in the stable state training set as the output data of the regression network model, training the regression network model, and obtaining an initial stable prediction network;

[0061] The initial stable prediction network is model verified using a stable state test set, and an initial stable prediction network that is less than or equal to a preset second test error threshold is output as a preconfigured state prediction model; the regression network model is a decision tree regression, Pu Bayesian regression, polynomial regression, support vector machine regression, random forest regression, linear regression or a neural network algorithm model.

[0062] Furthermore, the method for generating the first stability index in the historical stability training data includes:

[0063] Record the CPU utilization and replacement time each time the local control data is replaced with cloud control data;

[0064] According to the CPU utilization and the time required for replacement, a formula calculation is performed to obtain the first stability index of the cloud control data; the formula for the formula calculation is:

[0065] ;

[0066] Where: is the first stability index, is the CPU utilization of the mth cloud control data, Indicates the maximum allowed value of CPU utilization. The maximum allowed time required for replacement; The time required to replace the local control data with the mth cloud control data, and is the weight factor.

[0067] Furthermore, the method for adjusting and optimizing the local control data includes:

[0068] Step c1: extract the confidence score of each piece of local control data, sort them from large to small according to the confidence score, and obtain a sorted local control data list;

[0069] Step c2: Determine whether the first stability index belongs to the preset coefficient threshold range. If so, replace the local control data with the cloud control data and use it as hybrid control data, and jump to step c7; if not, jump to step c3;

[0070] Step c3: in the sorted local control data list, remove the local control data with the lowest confidence score, generate a current cloud control data response mapping table, and calculate the signal fluctuation coefficient of the response mapping table;

[0071] Step c4: taking the cloud control data quantity and signal fluctuation coefficient in the response mapping table as inputs, inputting them into the state prediction model, and calculating and obtaining the second stability coefficient;

[0072] Step c5: determining whether the second stability coefficient is within a preset coefficient threshold range, if so, using the cloud control data in the response mapping table as hybrid control data; if not, returning to step c3;

[0073] Step c6: repeat steps c3 to c5 until the second stability coefficient falls within the preset coefficient threshold range, then end the loop and jump to step c7;

[0074] Step c7: Output mixed control data.

[0075] In a second aspect, the present invention provides a hybrid voice recognition response system based on local and cloud, which is used to implement the above-mentioned hybrid voice recognition response method based on local and cloud, including:

[0076] A network connection analysis unit, used to obtain a signal fluctuation coefficient of the vehicle-mounted device within a time period of T, determine the network status of the vehicle-mounted device within the time period of T based on the signal fluctuation coefficient, and generate a corresponding voice recognition instruction according to the network status; the voice recognition instruction includes a cloud recognition instruction, a local recognition instruction, and a hybrid recognition instruction;

[0077] A local connection unit is used to receive a local recognition instruction, obtain local voice feature data of the i-th voice to be recognized, input the local voice feature data into a pre-built local voice recognition model to obtain corresponding local control data, and the vehicle equipment responds according to the local control data;

[0078] The cloud connection unit is used to receive the cloud recognition command, obtain the cloud voice feature data of the i-th voice to be recognized, input the cloud voice feature data into the pre-built cloud voice recognition model to obtain the corresponding cloud control data, and the vehicle equipment responds according to the cloud control data;

[0079] The integrated unit is used to receive hybrid identification instructions, obtain the confidence score of each local control data, and predict the first stability index of the cloud control data when the network is in a fluctuating state, and adjust and optimize the local control data according to the first stability index and the confidence score to obtain hybrid control data, and the vehicle equipment responds according to the hybrid control data.

[0080] Technical effects and advantages of the present invention:

[0081] The present invention obtains the signal fluctuation coefficient and judges the network status based on it. The vehicle system can flexibly select different types of identification instructions (cloud identification instructions, local identification instructions or hybrid identification instructions) according to the actual network conditions. When the network is unstable, local identification instructions are used first to avoid control response failures caused by cloud delays and ensure the stable operation of the vehicle equipment. In the case of network fluctuations, hybrid identification instructions allow local control data to be used in combination with cloud control data to optimize the accuracy and response speed of control data. By adjusting and optimizing local control data, the dependence of network instability on cloud control data is reduced, thereby improving the flexibility and accuracy of the vehicle system response. BRIEF DESCRIPTION OF THE DRAWINGS

[0082] Figure 1 This is a flow chart of the hybrid speech recognition response method based on local and cloud in Example 1;

[0083] Figure 2 This is a flow chart of a method for obtaining local control data from historical local speech training data in Example 1;

[0084] Figure 3 This is a flow chart of the method for adjusting and optimizing local control data in Example 1;

[0085] Figure 4 This is a schematic diagram of the structure of a local and cloud-based hybrid speech recognition response system of Example 2. DETAILED DESCRIPTION

[0086] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0087] In addition, the accompanying drawings are only schematic illustrations of the present invention and are not necessarily drawn to scale. The same reference numerals in the figures represent the same or similar parts, and their repeated description will be omitted. Some of the block diagrams shown in the accompanying drawings are functional entities and do not necessarily correspond to physically or logically independent entities. The functional entities can be implemented in software form, or implemented in one or more hardware modules or integrated circuits, or implemented in different networks and / or processor methods and / or microcontroller methods.

[0088] It should be understood that, although the terms "first", "second", etc. may be used herein to describe various units, these units should not be limited by these terms. These terms are used only to distinguish one unit from another unit. For example, without departing from the scope of the exemplary embodiments, a first unit may be referred to as a second unit, and a similar second unit may be referred to as a first unit. The term "and / or" used herein includes any and all combinations of one or more of the listed associated items.

[0089] Example 1

[0090] See also Figure 1 As shown, this embodiment discloses a hybrid voice recognition response method based on local and cloud. The method is applied to a vehicle device, and the vehicle device is equipped with an intelligent connection management module, which can automatically switch between cellular networks, Wi-Fi and satellite communications to maintain a stable cloud connection. The method includes:

[0091] The signal fluctuation coefficient of the vehicle equipment within the T time period is obtained, the network status of the vehicle equipment within the T time period is determined based on the signal fluctuation coefficient, and the corresponding voice recognition command is generated according to the network status.

[0092] Specifically, the network status includes an online status, an offline status, and a network fluctuation status. In implementation, the method for obtaining the signal fluctuation coefficient of the vehicle equipment within the T time period includes:

[0093] Acquire network quality data, wherein the network quality data includes signal strength variance, network delay time variance, data throughput variance, and packet loss rate variance;

[0094] It should be noted that the unit of the T time period can be seconds, minutes or hours, which is set according to the experience of technicians in this field and is not specifically limited here. The signal strength refers to the strength of the wireless network signal, which is measured by RSSI; the signal strength is usually expressed as a negative dBm value, for example, -30dBm means the signal is very strong, and -90dBm means the signal is very weak. The higher the signal strength, the more stable the network connection and the better the data transmission quality. The network delay time refers to the time required for a data packet to travel from the sender to the receiver; the smaller the delay, the faster the data transmission. The delay is affected by distance, network congestion and signal quality; the data throughput refers to the amount of data that can be transmitted by the network per unit time; the packet loss rate refers to the proportion of data packets lost during data transmission, usually expressed as a percentage. Data packet loss will cause incomplete data or the need for retransmission, and in severe cases, even interrupt the connection. The higher the packet loss rate, the worse the network reliability. It should be supplemented that the signal strength, network delay time, data throughput and packet loss rate are all obtained through the network detection tool in the vehicle equipment. Any method of obtaining network quality data in the prior art may be used, and no further explanation will be given here.

[0095] The calculation formula of the signal strength variance is: ;

[0096] In the formula, It represents the signal strength variance in the T time period. represents the signal strength at time t, Indicates the standard signal strength at time t.

[0097] The calculation formula of the network delay time variance is: ;

[0098] In the formula, It represents the variance of network delay in time period T. Indicates the network delay at time t, Indicates the standard duration of network delay at time t.

[0099] The calculation formula of the data throughput variance is: ;

[0100] In the formula, represents the data throughput variance in the T time period, represents the data throughput at time t, Represents the standard data throughput at time t.

[0101] The calculation formula of the packet loss rate variance is: ;

[0102] In the formula, represents the packet loss rate variance in the T time period, represents the packet loss rate at time t, Indicates the standard packet loss rate at time t.

[0103] The network quality data is processed in a formula to obtain the signal fluctuation coefficient, which is calculated as follows:

[0104] ;

[0105] In the formula, represents the signal fluctuation coefficient, represents the weight factor of the signal strength variance, Represents the weight factor of the network delay variance, represents the weight factor for the data throughput variance, The weight factors representing the packet loss rate variance are set by those skilled in the art based on experience.

[0106] In implementation, the method for determining the network status of the vehicle equipment within the T time period based on the signal fluctuation coefficient includes:

[0107] Preset fluctuation coefficient threshold, the fluctuation coefficient threshold includes and ,in ; Compare the signal fluctuation coefficient with the preset fluctuation coefficient threshold;

[0108] like > , then the network status of the vehicle equipment in the T time period is marked as the networking status.

[0109] like , then the network status of the vehicle equipment within the T time period is marked as a network fluctuation state.

[0110] like , then the network status of the vehicle equipment within the T time period is marked as offline.

[0111] It should be noted that: the preset fluctuation coefficient threshold is that the staff collects vehicle equipment data multiple times during the historical normal working stage of the vehicle equipment, collects Q groups of vehicle equipment data once, calculates the average of Q groups of vehicle equipment data, calculates the signal fluctuation coefficient corresponding to the average of the vehicle equipment data, sorts the multiple signal fluctuation coefficients, takes the maximum value of the multiple signal fluctuation coefficients as the maximum value of the signal fluctuation coefficient threshold, and takes the minimum value of the multiple signal fluctuation coefficients as the minimum value of the signal fluctuation coefficient threshold; the range between the minimum value of the signal fluctuation coefficient threshold to the maximum value of the signal fluctuation coefficient threshold is the preset fluctuation coefficient threshold.

[0112] The voice recognition instructions include cloud recognition instructions, local recognition instructions and hybrid recognition instructions.

[0113] In implementation, the method of generating a corresponding voice recognition instruction according to the network status includes:

[0114] When the network status of the vehicle equipment is marked as connected within the T time period, a cloud recognition instruction is generated according to the connected status;

[0115] When the network status of the vehicle equipment is marked as offline within the T time period, a local recognition instruction is generated according to the offline status.

[0116] When the network status of the vehicle equipment within the T time period is marked as a network fluctuation state, a hybrid identification instruction is generated according to the network fluctuation state.

[0117] In this step, when the network is offline or fluctuating, the system automatically selects the local recognition mode to avoid delays or failures in voice recognition due to network instability. This ensures that the vehicle system can still respond to the user's voice commands even when the signal is weak or the network is disconnected, improving the adaptability and robustness of the vehicle system.

[0118] Receive a local recognition instruction, obtain local voice feature data of the i-th voice to be recognized, input the local voice feature data into a pre-built local voice recognition model to obtain corresponding local control data, and the vehicle device responds according to the local control data;

[0119] It should be noted that: the i-th speech to be recognized is a phrase or sentence of the user's speech.

[0120] The local voice feature data includes keyword detection data and context state data. The keyword detection data is used to determine whether the user's input contains specific keywords or command words, such as "navigation", "make a call" or "turn on the air conditioner", etc. The context state data is the status information of the current vehicle system, such as navigation status, volume or air conditioning temperature, etc., which is used to ensure that the local command is in line with the current situation. By acquiring data from the sensors and modules of the vehicle system (such as air conditioning module, audio module), the context state and voice command are linked.

[0121] It should be noted that the local voice feature data is obtained by collecting voice signals through microphones, and the audio format transcoding converts the collected voice signals into audio files in PCM or WAV format. The endpoint detection removes the invalid parts of the transcoded voice and only retains the valid voice, thereby reducing the subsequent data calculation amount and speeding up the decoding speed. After front-end processing, the voice data is divided into time domain signals in frames, and the time domain signals of each frame are obtained after feature extraction such as Mel cepstral coefficients.

[0122] It should be noted that: the local speech recognition model is obtained by training with local speech training data; the local speech training data includes local speech feature data and corresponding local control data;

[0123] The local control data is used to respond to and execute basic instructions in an offline state, and ensure the basic functional operation of the vehicle equipment through preset control signals or status updates.

[0124] The basic instructions include simple operation instructions such as navigation start and volume adjustment. For example:

[0125] Navigation operation instructions: such as "navigate to home", "navigate to work", and call up saved frequently used addresses.

[0126] Media control commands: such as "Play music", "Pause music", "Next song", "Previous song" and "Volume up".

[0127] Air conditioning and temperature control commands: such as "Turn on air conditioning", "Turn off air conditioning", "Turn temperature up", and "Turn temperature down".

[0128] Basic phone commands: such as “Call [contact],” “Answer the call,” and “Hang up.”

[0129] In-vehicle information control commands: such as "open the window", "close the window" and "turn on the lights".

[0130] In a specific implementation, the method for generating the local speech recognition model specifically includes:

[0131] Historical local voice training data is obtained, and the historical local voice training data is divided into a local training set and a local test set; the historical local voice training data includes local voice feature data and its corresponding local control data.

[0132] See also Figure 2 As shown, the method for obtaining the local control data in the historical local speech training data includes:

[0133] Step a1: obtaining the local speech recognition coefficient of the i-th speech to be recognized;

[0134] It should be noted that the local speech recognition coefficient is obtained by collecting local speech feature data and performing normalization processing; the calculation formula is:

[0135] ;

[0136] In the formula, represents the local speech recognition coefficient, Indicates keyword detection data. Indicates keyword preset data, Represents contextual state data, Represents context state preset data, and is the weight factor, represents the natural exponential function.

[0137] It should be noted that the keyword preset data and context status preset data are preset by technicians in this field based on experience and pre-stored in the vehicle system database.

[0138] Step a2: setting N local speech recognition coefficient intervals, and setting N local control data corresponding to the N local speech recognition coefficient intervals; each local speech recognition coefficient interval is bound if and only if there is one local control data associated with it;

[0139] Step a3: comparing the local speech recognition coefficient of the i-th speech to be recognized with each local speech recognition coefficient interval, and obtaining the local speech recognition coefficient interval into which the local speech recognition coefficient of the i-th speech to be recognized falls;

[0140] Step a4: according to the local speech recognition coefficient interval into which the local speech recognition coefficient of the i-th speech to be recognized falls, divide the i-th speech to be recognized into the corresponding local control data; and set i=i+1, and jump back to step a1;

[0141] Step a5: Repeat the above steps a1 to a4 until the loop ends when i=I, so that each of the to-be-recognized voices is matched with N local control data in sequence, where I is the total number of all interactive voices between the user and the vehicle-mounted device within the T time period.

[0142] Construct a first recognition network model, use the local speech feature data in the local training set as the input data of the first recognition network model, use the local control data in the local training set as the output data of the first recognition network model, train the first recognition network model to obtain an initial first recognition network model; use the local test set to verify the initial first recognition network model, and output the initial first recognition network model with a prediction error less than or equal to a preset error threshold as the local speech recognition model, wherein the first recognition network model is a deep neural network model or a deep belief network model.

[0143] This step is to obtain the speech to be recognized and extract the corresponding local speech feature data. The local control data is obtained through intelligent search of the local speech recognition model, which can improve the response speed and accuracy of the vehicle system when it is disconnected from the network, and rely on the local speech recognition model to respond to basic operation instructions. Through comprehensive analysis of keyword detection data and context status data, the misrecognition rate is effectively reduced, and the local speech recognition model is used to ensure the coordination and consistency of recognition instructions and in-vehicle control functions, thereby improving the user experience.

[0144] Receive cloud recognition instructions, obtain cloud voice feature data of the i-th voice to be recognized, input the cloud voice feature data into a pre-built cloud voice recognition model to obtain corresponding cloud control data, and the vehicle equipment responds according to the cloud control data;

[0145] The cloud voice feature data includes location data and semantic intent data. The location data is the user's current geographic location information, driving status (such as highways, urban roads, etc.), navigation process, etc., and the location information is collected in real time through the GPS module and acceleration sensor of the vehicle equipment and uploaded to the cloud; the semantic intent data is the semantic intent in the recognized voice, such as "navigate to a certain place", "find the nearest gas station and navigate", and other compound instruction information, so as to have a deeper understanding of user needs, perform semantic analysis through natural language processing (NLP), and identify intentions and action goals.

[0146] The cloud-based speech recognition model is trained with cloud-based speech training data; the cloud-based speech training data includes cloud-based speech feature data and corresponding cloud-based control data. The cloud-based control data is used to call cloud-based resources and services and respond to complex instructions, including complex operation instructions such as information query and multi-step navigation. For example:

[0147] Complex navigation instructions: such as “navigate to nearby restaurants and display reviews” and “find the nearest gas station and view prices”.

[0148] Contextual query commands: such as "What's the weather like today?", "Is there a traffic jam on the highway?", "How long is it to the destination now?" and other commands involving real-time information query.

[0149] Detailed information query instructions: such as "play a certain singer's popular songs" and "search for nearby shopping malls and display business hours."

[0150] Natural language processing dialogue: such as "I'm a little cold, please help me adjust the temperature", "What interesting news are there today", "Recommend some relaxing music".

[0151] Complex vehicle diagnostic instructions: such as "display the maintenance status of my vehicle" and "check tire pressure and remind if inflation is needed."

[0152] In a specific implementation, the method for generating the cloud-based speech recognition model specifically includes:

[0153] Acquire historical cloud voice training data, and divide the historical cloud voice training data into a cloud voice recognition training set and a cloud voice recognition test set; the historical cloud voice training data includes cloud voice feature data and its corresponding cloud control data;

[0154] The method for acquiring the cloud control data in the historical cloud voice training data includes:

[0155] Step b1: Obtain the cloud speech recognition coefficient of the i-th speech to be recognized;

[0156] It should be noted that the cloud speech recognition coefficient is obtained by collecting cloud speech feature data and normalizing it, and its calculation formula is:

[0157] ;

[0158] In the formula, represents the cloud-based speech recognition coefficient, Represents location data, Indicates location preset data, Represents semantic intent data, Represents semantic intent preset data, and is the weight factor, represents the natural exponential function.

[0159] It should be noted that the location preset data and semantic intent preset data are both obtained through cloud-based intelligent recognition technology search; wherein, the location preset data generation method is based on a large amount of historical geographic location data, combined with the user's driving trajectory, preferences and behavior patterns, and generates the user's preset information in a specific time period or location through intelligent algorithms (such as cluster analysis, pattern recognition). For example, frequently visited places or routes are automatically identified to generate navigation preferences or traffic preset conditions. Among them, the semantic intent preset data generation method is to use a natural language processing (NLP) algorithm to analyze a large amount of voice instructions and text data, extract the user's common semantic intent, and build a preset model. The algorithm classifies the user's language patterns and expression habits, and maps the keywords and intentions in the instructions to specific operation instructions.

[0160] Step b2: setting M cloud speech recognition coefficient intervals, and setting M cloud control data corresponding to the M cloud speech recognition coefficient intervals; each cloud speech recognition coefficient interval is bound if and only if there is one cloud control data associated with it;

[0161] Step b3: Compare the cloud speech recognition coefficient of the i-th speech to be recognized with each cloud speech recognition coefficient interval to obtain the cloud speech recognition coefficient interval into which the cloud speech recognition coefficient of the i-th speech to be recognized falls;

[0162] Step b4: according to the cloud speech recognition coefficient interval into which the cloud speech recognition coefficient of the i-th speech to be recognized falls, classify the i-th speech to be recognized into the corresponding cloud control data; and set i=i+1, and jump back to step b1;

[0163] Step b5: Repeat the above steps b1 to b4 until the loop ends when i=I, so that each of the to-be-recognized voices is matched with M cloud control data in sequence.

[0164] Construct a second recognition network model, use the cloud speech feature data in the cloud speech recognition training set as the input data of the second recognition network model, use the cloud control data in the cloud speech recognition training set as the output data of the second recognition network model, train the second recognition network model to obtain an initial second recognition network model; use the cloud speech recognition test set to verify the initial second recognition network model, and output the initial second recognition network model with a prediction error less than or equal to a preset error threshold as the cloud speech recognition model, wherein the second recognition network model is a deep neural network model or a deep belief network model.

[0165] This step is to obtain the voice to be recognized and extract the corresponding cloud voice feature data. The cloud control data is obtained through intelligent search of the cloud voice recognition model, which can improve the response speed and accuracy of the vehicle system in the connected state, and rely on the cloud voice recognition model to respond to complex operation instructions. Through comprehensive analysis of location data and semantic intent data, it adapts to personalized scenarios, thereby improving the response efficiency and accuracy of vehicle voice recognition. In addition, the process uses deep learning models to continuously optimize the recognition effect, ensure the intelligence and stability of the service, and bring users a convenient in-vehicle service experience.

[0166] Receive the hybrid recognition instruction, obtain the confidence score of each local control data, and predict the first stability index of the cloud control data when the network is in a fluctuating state, and adjust and optimize the local control data according to the first stability index and the confidence score to obtain hybrid control data, and the vehicle equipment responds according to the hybrid control data;

[0167] In implementation, the method for obtaining the confidence score of the local control data is:

[0168] The replacement frequency of each piece of local control data and the historical performance data of each piece of local control data are extracted, wherein the historical performance data includes the duration, the last running time and the error rate.

[0169] It should be noted that the replacement frequency and historical performance data of each local control data are actually recorded and calculated by the technicians based on the performance of each local control data in the historical vehicle system log; wherein, the replacement frequency refers to the frequency of local control data being replaced by cloud control data. The duration refers to the duration of each local control data since its creation. Whenever a new local control data is created, its creation timestamp is recorded, and the duration is calculated by subtracting the creation time from the current time; and the last running time indicates the length of time the local control data was last used. When a local control data is used, its access timestamp is recorded, and the last running time is calculated by subtracting the last access time from the current time; the error rate refers to the proportion of errors in the interaction process of the local control data, and records the correct and incorrect interactions of each local control data; the error rate is calculated by dividing the number of errors by the total number of interactions.

[0170] The historical performance data and replacement frequency of each piece of local control data are sequentially input into a pre-built mathematical calculation model to obtain a confidence score for each piece of local control data; the expression of the mathematical calculation model is: ; Where: Score the confidence of the e-th local control data, is the replacement frequency of the local control data in clause e, For the duration of local control data in Article e, The last running time of the local control data in item e, is the error rate of the local control data in the e-th item.

[0171] It can be understood that: the confidence score can quantify the credibility of each local control data. The larger the confidence score, the greater the credibility of the corresponding local control data and the higher the confidence. On the contrary, it reflects that the corresponding local control data is less credible and has a lower priority.

[0172] In implementation, the method for predicting the first stability index of cloud control data in a network fluctuation state includes:

[0173] Extract the amount M of cloud control data and the signal fluctuation coefficient;

[0174] The amount M of cloud control data and the signal fluctuation coefficient are used as state performance data and input into a preconfigured state prediction model to obtain a first stability index of the cloud control data when the network is in a fluctuating state.

[0175] Specifically, the method for generating the state prediction model includes:

[0176] Obtain historical stable training data, and divide the historical stable training data into a stable state training set and a stable state test set; the historical stable training data includes state performance data and its corresponding first stability index; wherein, the state performance data in the historical stable training data is collected in advance by technical personnel, and its specific collection process or logic refers to the above description, which will not be repeated here.

[0177] The method for generating the first stability index in the historical stability training data includes:

[0178] Record the CPU utilization and replacement time each time the local control data is replaced with cloud control data;

[0179] According to the CPU utilization and the time required for replacement, a formula calculation is performed to obtain the first stability index of the cloud control data; the formula for the formula calculation is: ; Where: is the first stability index, is the CPU utilization of the mth cloud control data, Indicates the maximum allowed value of CPU utilization. The maximum allowed time required for replacement; The time required to replace the local control data with the mth cloud control data, and is the weight factor.

[0180] It should be noted that the CPU utilization and replacement time required each time the cloud control data is used to replace the local control data are recorded. The technicians collect this data based on the vehicle system log and store it in the vehicle system database. The maximum allowable value of CPU utilization indicates the highest CPU utilization threshold that the system can tolerate; the maximum allowable value of replacement time is the maximum delay time when a single cloud control data replaces the local control data. To ensure the stability and performance of the vehicle system, technicians usually determine these two maximum allowable values ​​through experiments and tests.

[0181] It can be understood that: the larger the first stability index is, the more stable the performance is when the cloud control data replaces the local control data in the networked state; on the contrary, the smaller the first stability index is, the more unstable the performance is when the cloud control data replaces the local control data in the networked state, which may easily affect the normal operation of the cloud voice recognition response.

[0182] A regression network model is constructed, the state performance data in the stable state training set is used as the input data of the regression network model, the first stable index in the stable state training set is used as the output data of the regression network model, the regression network model is trained, and an initial stable prediction network is obtained.

[0183] The initial stable prediction network is model verified using a stable state test set, and an initial stable prediction network that is less than or equal to a preset second test error threshold is output as a preconfigured state prediction model; the regression network model is a decision tree regression, Pu Bayesian regression, polynomial regression, support vector machine regression, random forest regression, linear regression or a neural network algorithm model.

[0184] In implementation, see Figure 3 As shown, the method for adjusting and optimizing the local control data includes:

[0185] Step c1: extract the confidence score of each piece of local control data, sort them from large to small according to the confidence score, and obtain a sorted local control data list;

[0186] Step c2: Determine whether the first stability index belongs to the preset coefficient threshold range. If so, replace the local control data with the cloud control data and use it as hybrid control data, and jump to step c7; if not, jump to step c3;

[0187] Step c3: In the sorted local control data list, remove the local control data with the lowest confidence score, generate a current cloud control data response mapping table, and calculate the signal fluctuation coefficient of the response mapping table.

[0188] Step c4: taking the cloud control data quantity and signal fluctuation coefficient in the response mapping table as inputs, inputting them into the state prediction model, and calculating and obtaining the second stability coefficient;

[0189] It should be noted that the second stability coefficient is of the same type as the first stability index, and the difference is that the different names are used to distinguish the representations in different network states.

[0190] Step c5: determining whether the second stability coefficient is within a preset coefficient threshold range, if so, using the cloud control data in the response mapping table as hybrid control data; if not, returning to step c3;

[0191] It should be noted that: in step c3, each time a removal task is executed, only one local control data is removed. Therefore, each time a loop is executed and step c3 is returned once, only one local control data is removed.

[0192] Step c6: repeat the above steps c3 to c5 until the second stability coefficient falls within the preset coefficient threshold range, then end the loop and jump to step c7;

[0193] Step c7: Output mixed control data.

[0194] This step is used for vehicle-mounted systems in a network fluctuation state. By mutual verification, gradual elimination and calculation of the stability coefficient, it can effectively manage the replacement and fusion of cloud and local control data when the network fluctuates or is unstable, thereby ensuring that the system can still make accurate responses in different network states, thereby improving the stability and performance of the vehicle-mounted system.

[0195] The formulas involved in the above are all calculated by removing dimensions and taking their numerical values. They are a formula that is closest to the actual situation obtained by software simulation of a large amount of collected data. The weight factors in the formula and the preset thresholds in the analysis process are set by technical personnel in this field according to actual conditions or obtained by simulating a large amount of data; the size of the weight factor is to quantify each parameter to obtain a specific value for subsequent comparison. The size of the weight factor depends on the amount of sample data and the corresponding processing coefficient initially set by technical personnel in this field for each group of sample data; as long as it does not affect the proportional relationship between the parameter and the quantized value.

[0196] In this embodiment, by obtaining the signal fluctuation coefficient and judging the network status based on it, the vehicle system can flexibly select different types of identification instructions (cloud identification instructions, local identification instructions or hybrid identification instructions) according to the actual network conditions. When the network is unstable, local identification instructions are used first to avoid control response failures caused by cloud delays and ensure the stable operation of the vehicle equipment. In the case of network fluctuations, hybrid identification instructions allow local control data to be used in combination with cloud control data to optimize the accuracy and response speed of control data. By adjusting and optimizing local control data, the dependence of network instability on cloud control data is reduced, thereby improving the flexibility and accuracy of the vehicle system response.

[0197] Example 2

[0198] See also Figure 4 As shown, this embodiment provides a hybrid speech recognition response system based on local and cloud, including a network connection analysis unit, a local connection unit, a cloud connection unit and an integrated unit; each unit is connected by wire and / or wireless means to achieve data transmission between units; the system includes:

[0199] A network connection analysis unit, used to obtain a signal fluctuation coefficient of the vehicle-mounted device within a time period of T, determine the network status of the vehicle-mounted device within the time period of T based on the signal fluctuation coefficient, and generate a corresponding voice recognition instruction according to the network status; the voice recognition instruction includes a cloud recognition instruction, a local recognition instruction, and a hybrid recognition instruction;

[0200] A local connection unit is used to receive a local recognition instruction, obtain local voice feature data of the i-th voice to be recognized, input the local voice feature data into a pre-built local voice recognition model to obtain corresponding local control data, and the vehicle equipment responds according to the local control data;

[0201] The cloud connection unit is used to receive the cloud recognition command, obtain the cloud voice feature data of the i-th voice to be recognized, input the cloud voice feature data into the pre-built cloud voice recognition model to obtain the corresponding cloud control data, and the vehicle equipment responds according to the cloud control data;

[0202] The integrated unit is used to receive hybrid identification instructions, obtain the confidence score of each local control data, and predict the first stability index of the cloud control data when the network is in a fluctuating state, and adjust and optimize the local control data according to the first stability index and the confidence score to obtain hybrid control data, and the vehicle equipment responds according to the hybrid control data.

[0203] Example 3

[0204] This embodiment provides an electronic device. The electronic device may include one or more processors and one or more memories. The memories store computer-readable codes, and when the computer-readable codes are executed by the one or more processors, the hybrid speech recognition response method based on local and cloud computing may be executed.

[0205] The system according to the implementation mode of the present application can also be implemented with the help of the architecture of an electronic device. The architecture of the electronic device may include a bus, one or more CPUs, a read-only memory (ROM), a random access memory (RAM), a communication port connected to a network, an input / output, a hard disk, etc. A storage device in the electronic device, such as a ROM or a hard disk, can store the hybrid voice recognition response method based on local and cloud provided in the present application. Furthermore, the electronic device may also include a user interface. Of course, the above architecture is only exemplary. When implementing different devices, one or more components in the electronic device may be omitted according to actual needs.

[0206] Example 4

[0207] This embodiment provides a computer-readable storage medium storing instructions. When the instructions are executed on a computer, the computer executes the local and cloud-based hybrid speech recognition response method of Embodiment 1.

[0208] The above is only a specific implementation of the present application, but the protection scope of the present application is not limited thereto. Any person skilled in the art who is familiar with the present technical field can easily think of changes or substitutions within the technical scope disclosed in the present application, which should be included in the protection scope of the present application. Therefore, the protection scope of the present application should be based on the protection scope of the claims.

[0209] Finally: The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions, improvements, etc. made within the spirit and principles of the present invention should be included in the protection scope of the present invention.

Claims

1. A hybrid speech recognition response method based on local and cloud, characterized in that: include: Obtaining the signal fluctuation coefficient of the vehicle equipment within the time period T, judging the network status of the vehicle equipment within the time period T based on the signal fluctuation coefficient, and generating corresponding voice recognition instructions according to the network status; the voice recognition instructions include cloud recognition instructions, local recognition instructions and hybrid recognition instructions; Receive a local recognition instruction, obtain local voice feature data of the i-th voice to be recognized, input the local voice feature data into a pre-built local voice recognition model to obtain corresponding local control data, and the vehicle device responds according to the local control data; Receive cloud recognition instructions, obtain cloud voice feature data of the i-th voice to be recognized, input the cloud voice feature data into a pre-built cloud voice recognition model to obtain corresponding cloud control data, and the vehicle equipment responds according to the cloud control data; Receive the hybrid recognition instruction, obtain the confidence score of each local control data, and predict the first stability index of the cloud control data when the network is in a fluctuating state, and adjust and optimize the local control data according to the first stability index and the confidence score to obtain the hybrid control data. The vehicle equipment responds according to the hybrid control data.

2. The hybrid speech recognition response method based on local and cloud according to claim 1 is characterized in that: The network status includes online status, offline status and network fluctuation status; The method for obtaining the signal fluctuation coefficient of the vehicle equipment within the T time period includes: Acquire network quality data, wherein the network quality data includes signal strength variance, network delay time variance, data throughput variance, and packet loss rate variance; Formulate network quality data to obtain signal fluctuation coefficient ; The method for judging the network status of the vehicle equipment within the T time period based on the signal fluctuation coefficient includes: Preset fluctuation coefficient threshold, the fluctuation coefficient threshold includes and ,in ; Compare the signal fluctuation coefficient with the preset fluctuation coefficient threshold; like > , then the network status of the vehicle equipment in the T time period is marked as the network status; like , then the network status of the vehicle equipment in the T time period is marked as a network fluctuation state; like , then the network status of the vehicle equipment within the T time period is marked as offline.

3. The hybrid speech recognition response method based on local and cloud according to claim 2 is characterized in that: The method for generating a corresponding voice recognition instruction according to a network state includes: When the network status of the vehicle equipment is marked as connected within the T time period, a cloud recognition instruction is generated according to the connected status; When the network status of the vehicle equipment is marked as offline within the T time period, a local recognition instruction is generated according to the offline status; When the network status of the vehicle equipment within the T time period is marked as a network fluctuation state, a hybrid identification instruction is generated according to the network fluctuation state.

4. The hybrid speech recognition response method based on local and cloud according to claim 3 is characterized in that: The local speech feature data includes keyword detection data and context state data; The local speech recognition model is obtained by training with local speech training data; The local speech training data includes local speech feature data and corresponding local control data; The method for generating the local speech recognition model specifically includes: Acquire historical local voice training data, and divide the historical local voice training data into a local training set and a local test set; the historical local voice training data includes local voice feature data and its corresponding local control data; Construct a first recognition network model, use the local speech feature data in the local training set as the input data of the first recognition network model, use the local control data in the local training set as the output data of the first recognition network model, train the first recognition network model to obtain an initial first recognition network model; use the local test set to verify the initial first recognition network model, and output the initial first recognition network model with a prediction error less than or equal to a preset error threshold as the local speech recognition model, wherein the first recognition network model is a deep neural network model or a deep belief network model.

5. The hybrid speech recognition response method based on local and cloud according to claim 4 is characterized in that: The method for acquiring the local control data in the historical local voice training data includes: Step a1: obtaining the local speech recognition coefficient of the i-th speech to be recognized; The local speech recognition coefficient is obtained by collecting local speech feature data and performing normalization processing; the calculation formula is: ； In the formula, represents the local speech recognition coefficient, Indicates keyword detection data. Indicates keyword preset data, Represents contextual state data, Represents context state preset data, and is the weight factor, represents the natural exponential function; Step a2: setting N local speech recognition coefficient intervals, and setting N local control data corresponding to the N local speech recognition coefficient intervals; each local speech recognition coefficient interval is bound if and only if there is one local control data associated with it; Step a3: comparing the local speech recognition coefficient of the i-th speech to be recognized with each local speech recognition coefficient interval, and obtaining the local speech recognition coefficient interval into which the local speech recognition coefficient of the i-th speech to be recognized falls; Step a4: according to the local speech recognition coefficient interval into which the local speech recognition coefficient of the i-th speech to be recognized falls, divide the i-th speech to be recognized into the corresponding local control data; and set i=i+1, and jump back to step a1; Step a5: Repeat steps a1 to a4 until the loop ends when i=I, so that each of the to-be-recognized voices is matched with N local control data in sequence, where I is the total number of all interactive voices between the user and the vehicle-mounted device within the T time period.

6. The hybrid speech recognition response method based on local and cloud according to claim 5, characterized in that: The cloud-based speech feature data includes location data and semantic intent data; the cloud-based speech recognition model is obtained by training with cloud-based speech training data; The cloud voice training data includes cloud voice feature data and corresponding cloud control data; The method for generating the cloud-based speech recognition model specifically includes: Acquire historical cloud voice training data, and divide the historical cloud voice training data into a cloud voice recognition training set and a cloud voice recognition test set; the historical cloud voice training data includes cloud voice feature data and its corresponding cloud control data; Construct a second recognition network model, use the cloud speech feature data in the cloud speech recognition training set as the input data of the second recognition network model, use the cloud control data in the cloud speech recognition training set as the output data of the second recognition network model, train the second recognition network model to obtain an initial second recognition network model; use the cloud speech recognition test set to verify the initial second recognition network model, and output the initial second recognition network model with a prediction error less than or equal to a preset error threshold as the cloud speech recognition model, wherein the second recognition network model is a deep neural network model or a deep belief network model.

7. The hybrid speech recognition response method based on local and cloud according to claim 6, characterized in that: The method for obtaining the cloud control data in the historical cloud voice training data includes: Step b1: Obtain the cloud speech recognition coefficient of the i-th speech to be recognized; The cloud speech recognition coefficient is obtained by collecting cloud speech feature data and normalizing it, and its calculation formula is: ； In the formula, represents the cloud-based speech recognition coefficient, Represents location data, Indicates location preset data, Represents semantic intent data, Represents semantic intent preset data, and is the weight factor, represents the natural exponential function; Step b2: setting M cloud speech recognition coefficient intervals, and setting M cloud control data corresponding to the M cloud speech recognition coefficient intervals; each cloud speech recognition coefficient interval is bound if and only if there is one cloud control data associated with it; Step b3: Compare the cloud speech recognition coefficient of the i-th speech to be recognized with each cloud speech recognition coefficient interval to obtain the cloud speech recognition coefficient interval into which the cloud speech recognition coefficient of the i-th speech to be recognized falls; Step b4: according to the cloud speech recognition coefficient interval into which the cloud speech recognition coefficient of the i-th speech to be recognized falls, classify the i-th speech to be recognized into the corresponding cloud control data; and set i=i+1, and jump back to step b1; Step b5: Repeat steps b1 to b4 until the loop ends when i=I, so that each of the to-be-recognized voices is matched with M cloud control data in sequence.

8. The hybrid speech recognition response method based on local and cloud according to claim 7, characterized in that: The method for obtaining the confidence score of the local control data includes: Extracting the replacement frequency of each piece of local control data and the historical performance data of each piece of local control data, wherein the historical performance data includes the duration, the last running time and the error rate; The historical performance data and replacement frequency of each piece of local control data are sequentially input into a pre-built mathematical calculation model to obtain a confidence score for each piece of local control data; the expression of the mathematical calculation model is: ; Where: Score the confidence of the e-th local control data, is the replacement frequency of the local control data in clause e, For the duration of local control data in clause e, The last running time of the local control data in item e, is the error rate of the local control data in the e-th item.

9. The hybrid speech recognition response method based on local and cloud according to claim 8, characterized in that: The method for predicting the first stability index of cloud control data in a network fluctuation state includes: Extract the amount M of cloud control data and the signal fluctuation coefficient; The amount M of cloud control data and the signal fluctuation coefficient are used as state performance data and input into a preconfigured state prediction model to obtain a first stability index of the cloud control data when the network is in a fluctuation state; The method for generating the state prediction model comprises: Acquire historical stable training data, and divide the historical stable training data into a stable state training set and a stable state test set; the historical stable training data includes state performance data and its corresponding first stability index; Constructing a regression network model, taking the state performance data in the stable state training set as the input data of the regression network model, taking the first stability index in the stable state training set as the output data of the regression network model, training the regression network model, and obtaining an initial stable prediction network; The initial stable prediction network is model verified using a stable state test set, and an initial stable prediction network that is less than or equal to a preset second test error threshold is output as a preconfigured state prediction model; the regression network model is a decision tree regression, Pu Bayesian regression, polynomial regression, support vector machine regression, random forest regression, linear regression or a neural network algorithm model.

10. The hybrid speech recognition response method based on local and cloud according to claim 9, characterized in that: The method for generating the first stability index in the historical stability training data includes: Record the CPU utilization and replacement time each time the local control data is replaced with cloud control data; According to the CPU utilization and the time required for replacement, a formula calculation is performed to obtain the first stability index of the cloud control data; the formula for the formula calculation is: ； Where: is the first stability index, is the CPU utilization of the mth cloud control data, Indicates the maximum allowed value of CPU utilization. The maximum allowed time required for replacement; The time required to replace the local control data with the mth cloud control data, and is the weight factor.

11. The hybrid speech recognition response method based on local and cloud according to claim 10, characterized in that: Methods for adjusting and optimizing local control data include: Step c1: extract the confidence score of each piece of local control data, sort them from large to small according to the confidence score, and obtain a sorted local control data list; Step c2: Determine whether the first stability index belongs to the preset coefficient threshold range. If so, replace the local control data with the cloud control data and use it as hybrid control data, and jump to step c7; if not, jump to step c3; Step c3: in the sorted local control data list, remove the local control data with the lowest confidence score, generate a current cloud control data response mapping table, and calculate the signal fluctuation coefficient of the response mapping table; Step c4: taking the cloud control data quantity and signal fluctuation coefficient in the response mapping table as inputs, inputting them into the state prediction model, and calculating and obtaining the second stability coefficient; Step c5: determining whether the second stability coefficient is within a preset coefficient threshold range, if so, using the cloud control data in the response mapping table as hybrid control data; if not, returning to step c3; Step c6: repeat steps c3 to c5 until the second stability coefficient falls within the preset coefficient threshold range, then end the loop and jump to step c7; Step c7: Output mixed control data.

12. A local and cloud-based hybrid speech recognition response system, used to implement the local and cloud-based hybrid speech recognition response method according to any one of claims 1 to 11, characterized in that: include: A network connection analysis unit, used to obtain a signal fluctuation coefficient of the vehicle-mounted device within a time period of T, determine the network status of the vehicle-mounted device within the time period of T based on the signal fluctuation coefficient, and generate a corresponding voice recognition instruction according to the network status; the voice recognition instruction includes a cloud recognition instruction, a local recognition instruction, and a hybrid recognition instruction; A local connection unit is used to receive a local recognition instruction, obtain local voice feature data of the i-th voice to be recognized, input the local voice feature data into a pre-built local voice recognition model to obtain corresponding local control data, and the vehicle equipment responds according to the local control data; The cloud connection unit is used to receive the cloud recognition command, obtain the cloud voice feature data of the i-th voice to be recognized, input the cloud voice feature data into the pre-built cloud voice recognition model to obtain the corresponding cloud control data, and the vehicle equipment responds according to the cloud control data; The integrated unit is used to receive hybrid identification instructions, obtain the confidence score of each local control data, and predict the first stability index of the cloud control data when the network is in a fluctuating state, and adjust and optimize the local control data according to the first stability index and the confidence score to obtain hybrid control data, and the vehicle equipment responds according to the hybrid control data.

Images