Vehicle-mounted speech recognition model training method, electronic device, storage medium and program product

Abstract

The present application discloses a vehicle-mounted speech recognition model training method, electronic device, storage medium and program product. The embodiment of the present application provides a vehicle-mounted speech recognition model training method for a vehicle-mounted speech recognition model training platform, through which a user can fill in microphone information and customized training information or non-customized training information on the training platform according to the guidance of the task creation interface, thereby generating a model training task and sending it to a training server for vehicle-mounted speech recognition model training, and the user does not need to make relevant investments in hardware and specific implementation methods of the training model, thereby improving the accuracy and efficiency of training the vehicle-mounted speech recognition model.

Description

Technical Field

[0001] The present application relates to the technical field of vehicle-mounted speech recognition, and in particular to a vehicle-mounted speech recognition model training method, electronic equipment, storage medium and program product. Background Art

[0002] In the related art, the in-vehicle speech recognition model has problems such as limited pre-processing effect and inability to accurately locate the target voice when dealing with complex and changeable noise environments. In order to solve the above problems, it is necessary to conduct targeted model training for different in-vehicle environments, but there are still the following problems in model training: ① Time-consuming and labor-intensive: Customized in-vehicle speech recognition models need to be carefully designed and optimized for the noise environment and speech characteristics of specific fields, which often requires a lot of time and R&D manpower investment. In addition, from audio recording to building training and test sets, to model training and optimization, the entire process involves collaboration between multiple links and departments, which increases the complexity and time-consuming nature of the project.

[0003] ② Long scheduling cycle: Due to the complexity and time-consuming nature of the customized in-vehicle speech recognition model training process, project schedules are often long, which may slow down product iteration and affect market competitiveness. Summary of the invention

[0004] The applicant has made the following analysis on the specific factors that cause the long model training time in the related art: ① Computation-intensive services: Model training requires a large amount of GPU and CPU computing resources, which is a computation-intensive service, resulting in increased machine costs. Efficient computing resources not only require high hardware investment, but also require professional operation and management teams to ensure stable operation and efficient use of resources.

[0005] ② Differences in the implementation of the underlying algorithm: The implementation of the underlying algorithm for preprocessing, data processing, network training and model testing is different, which will directly affect the time and effect of the model output. Different algorithm implementations may have different computational complexities and optimization spaces, which need to be selected and adjusted according to specific circumstances.

[0006] Therefore, to address these issues, this application needs to explore more efficient and intelligent signal processing technologies and methods. At the same time, it also needs to strengthen collaboration and communication between departments and optimize process design to improve the execution efficiency and quality of the entire project.

[0007] In order to solve at least one of the above problems, the applicant has decided to develop an in-vehicle multi-zone signal processing training platform after repeated consideration. The platform aims to improve the accuracy and efficiency of in-vehicle speech recognition model training by outputting signal processing resources suitable for specific vehicle environments through customized training.

[0008] The embodiments of the present application provide a vehicle-mounted speech recognition model training method, electronic device, storage medium and program product, which are used to solve at least one of the above-mentioned technical problems.

[0009] In a first aspect, an embodiment of the present application provides a vehicle-mounted speech recognition model training method for a vehicle-mounted speech recognition model training platform, the method comprising: In response to the user's task creation operation on the training platform, an initial task creation interface is generated; wherein the initial task creation interface is provided with a non-customized training mode option and a customized training mode option; When the non-customized training mode option is selected, a first task creation interface is refreshed and generated; wherein the first task creation interface includes a first microphone information filling area and a non-customized training information filling area; When the customized training mode option is selected, a second task creation interface is refreshed and generated; wherein the second task creation interface includes a second microphone information filling area and a customized training information filling area; generating a third task creation interface in response to a user operation, wherein the third task creation interface includes a test information filling area; A model training task is generated based on the information filled in by the user in the first task creation interface or the second task creation interface, and the information filled in by the user in the third task creation interface, and is sent to the training server for vehicle-mounted speech recognition model training.

[0010] In some embodiments, the method further comprises: Add the model training task to a task list, wherein the task list includes the status and operable options of the model training task; wherein the operable options include at least one of details, download, retry, cancel and delete; When the training server completes the vehicle-mounted speech recognition model training and generates model resources, receiving the model resources and updating the status of the model training task to test success; When the training server fails to perform vehicle-mounted speech recognition model training, updating the status of the model training task to test failure; When the training server is in an occupied state, the state of the model training task is updated to be in training queue.

[0011] In some embodiments, the first microphone information filling area includes at least one of the following: a microphone type option, a microphone quantity filling area, a sound zone quantity filling area, a loop quantity filling area, and a microphone arrangement information filling area; the method further includes: When the non-customized training mode option is selected, the training server selects a matching target public version model from multiple preset public version models based on at least one of the microphone type, number of microphones, number of voice zones, number of loops and microphone arrangement information in the model training task for in-vehicle speech recognition model training.

[0012] In some embodiments, the second microphone information filling area includes at least one of the following: a microphone type option, a microphone quantity filling area, a sound zone quantity filling area, a loop quantity filling area, and a microphone arrangement information filling area; the method further includes: When the customized training mode option is selected, the training server performs in-vehicle speech recognition model training on the preset eabNet single model according to at least one of the microphone type, number of microphones, number of sound zones, number of loops and microphone arrangement information in the model training task.

[0013] In some embodiments, the customized training information filling area includes at least one of the following: an RIR training set filling area, a noise training set filling area, a total number of audio channels filling area, and a channel order filling area.

[0014] In some embodiments, the training platform includes a task list interface, the task list interface includes a task list and a task search box; the task search box includes at least one of a task ID search box, a task name search box, a training method search box, a status search box, a creator search box, and a creation time search box; The method further includes: presenting the hit task in the task list according to at least one of a task ID, a task name, a training method, a creator, and a creation time input by a user.

[0015] In a second aspect, the present application also provides a vehicle-mounted speech recognition model training method for training a server, the method comprising: Receive a model training task from an in-vehicle speech recognition model training platform; the model training task is generated by the training platform based on information filled in by the user in a first task creation interface or a second task creation interface, and information filled in by the user in a third task creation interface; wherein the first task creation interface includes a first microphone information filling area and a non-customized training information filling area; the second task creation interface includes a second microphone information filling area and a customized training information filling area; Selecting a training model according to the training task, and performing vehicle-mounted speech recognition model training on the training model; The training resources obtained through training are sent to the training platform.

[0016] In some embodiments, the training platform generates an initial task creation interface in response to a user's task creation operation on the training platform; wherein the initial task creation interface is provided with a non-customized training mode option and a customized training mode option; the first microphone information filling area or the second microphone information filling area includes at least one of the following: a microphone type option, a microphone quantity filling area, a sound zone quantity filling area, a loop quantity filling area, and a microphone arrangement information filling area; The selecting a training model according to the training task and performing vehicle-mounted speech recognition model training on the training model includes: When the non-customized training mode option is selected, the training server selects a matching target public version model from a plurality of preset public version models according to at least one of the microphone type, the number of microphones, the number of sound zones, the number of loops, and the microphone arrangement information in the model training task for in-vehicle speech recognition model training; When the customized training mode option is selected, the training server performs in-vehicle speech recognition model training on the preset eabNet single model according to at least one of the microphone type, number of microphones, number of sound zones, number of loops and microphone arrangement information in the model training task.

[0017] In a third aspect, an embodiment of the present application provides an electronic device comprising: at least one processor, and a memory communicatively connected to the at least one processor, wherein the memory stores instructions executable by the at least one processor, and the instructions are executed by the at least one processor so that the at least one processor can execute the steps of the in-vehicle speech recognition model training method described in any embodiment of the present application.

[0018] In a fourth aspect, an embodiment of the present application provides a storage medium on which a computer program is stored, characterized in that when the program is executed by a processor, the steps of the in-vehicle speech recognition model training method described in any embodiment of the present application are implemented.

[0019] In a fifth aspect, an embodiment of the present application provides a computer program product, including a computer program / instructions, characterized in that when the computer program / instructions are executed by a processor, the steps of the in-vehicle speech recognition model training method described in any embodiment of the present application are implemented.

[0020] The beneficial effect of the embodiments of the present application is that it provides a vehicle-mounted speech recognition model training method for a vehicle-mounted speech recognition model training platform. The user can use this method to fill in microphone information and customized training information or non-customized training information on the training platform according to the guidance of the task creation interface, thereby generating a model training task and sending it to the training server for vehicle-mounted speech recognition model training. The user does not need to make relevant investments in hardware and the specific implementation method of the training model, thereby improving the accuracy and efficiency of training the vehicle-mounted speech recognition model. BRIEF DESCRIPTION OF THE DRAWINGS

[0021] In order to more clearly illustrate the technical solutions of the embodiments of the present application, the drawings required for use in the description of the embodiments will be briefly introduced below. Obviously, the drawings described below are some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.

[0022] Figure 1 A flowchart of an embodiment of the vehicle-mounted speech recognition model training method of the present application; Figure 2 A schematic diagram of an embodiment of a training platform page of the present application; Figure 3 A schematic diagram of an embodiment of a task creation interface of the training platform of the present application; Figure 4 A schematic diagram of another embodiment of a task creation interface of the training platform of the present application; Figure 5 A schematic diagram of another embodiment of a task creation interface of the training platform of the present application; Figure 6 A schematic diagram of another embodiment of a task creation interface of the training platform of the present application; Figure 7 A flowchart of another embodiment of the vehicle-mounted speech recognition model training method of the present application; Figure 8 A flowchart of another embodiment of the vehicle-mounted speech recognition model training method of the present application; Fig. 9 A timing diagram of a user creating a custom resource task through the platform in this application; Fig.10 This is a schematic diagram of the underlying processing logic flow after the training server receives a task in this application; Fig.11 It is a schematic structural diagram of an embodiment of an electronic device of the present application. DETAILED DESCRIPTION

[0023] In order to make the purpose, technical scheme and advantages of the embodiments of the present application clearer, the technical scheme in the embodiments of the present application will be clearly and completely described below in conjunction with the drawings in the embodiments of the present application. Obviously, the described embodiments are part of the embodiments of the present application, rather than all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in the field without making creative work are within the scope of protection of the present application. It should be noted that the embodiments in the present application and the features in the embodiments can be combined with each other without conflict.

[0024] It should also be noted that, in this article, the terms "include" and "comprise" include not only those elements, but also other elements not explicitly listed, or elements inherent to such processes, methods, articles or equipment. In the absence of further restrictions, the elements defined by the sentence "include..." do not exclude the existence of other identical elements in the process, method, article or equipment that includes the elements.

[0025] like Figure 1 As shown, an embodiment of the present application provides a vehicle-mounted speech recognition model training method for a vehicle-mounted speech recognition model training platform, the method comprising: S11. In response to the user's task creation operation on the training platform, an initial task creation interface is generated; wherein the initial task creation interface is provided with a non-customized training mode option and a customized training mode option.

[0026] For example, the user logs in to the vehicle-mounted speech recognition model training platform through account information (eg, user name and password), and clicks on the relevant component on the training platform page to jump to the training platform page. Figure 2 The following is a schematic diagram of an embodiment of the training platform page of the present application. Figure 2 The local integrated training platform shown. Further, the user can click Figure 2 The "Create Task" component in the training platform generates an initial task creation interface (such as Figure 3 In this embodiment, the non-customized training mode option and the customized training mode option correspond to Figure 3 The public version option and the eabNet single model training option after "Training method" in the initial task creation interface are selected or the eabNet single model training option is selected, and this application does not limit this.

[0027] S12. When the non-customized training mode option is selected, refresh and generate a first task creation interface; wherein the first task creation interface includes a first microphone information filling area and a non-customized training information filling area.

[0028] For example, Figure 3 The interface shown is the page where the non-customized training method (corresponding to the public version option) is selected in this application, and this page also corresponds to the first creation task interface generated by refreshing. The user can enter the task name under this interface, select the microphone type and enter the microphone information in the microphone information area, and enter the version number in the training information area. You can also select or switch the training method under this page. In addition, you can also select a low-precision model under this page.

[0029] Exemplarily, the customized training information filling area includes at least one of the following: an RIR training set filling area, a noise training set filling area, a total number of audio channels filling area, and a channel order filling area.

[0030] S13. When the customized training mode option is selected, refresh and generate a second task creation interface; wherein the second task creation interface includes a second microphone information filling area and a customized training information filling area.

[0031] Exemplarily, the second task creation interface includes a second microphone information filling area and the first task creation interface includes a first microphone information filling area, and the only difference is the customized training information filling area. Figure 4 The interface shown is the page for filling in the customized training information area when the customized training method (corresponding to the eabNet single model training option) in this application is selected. Figure 4 The illustrated embodiment includes an RIR training set input box, a noise training set input box, a total number of audio channels input box, and a channel order option and input box (such as Figure 4 The "Standard" and "Custom" options in the dialog box, as well as the channel order input box below, when "Standard" is selected, the channel order adopts the preset order and the channel order input box cannot be edited; when "Custom" is selected, the channel order input box is editable and users can enter a custom channel order).

[0032] S14. Generate a third task creation interface in response to a user operation, wherein the third task creation interface includes a test information filling area.

[0033] Exemplarily, after completing filling in relevant information in the first task creation interface or the second task creation interface, the user can click the next operation to trigger the training platform to generate a third task creation interface in response to the user operation.

[0034] like Figure 5The diagram shows an embodiment of the third task creation interface in the present application. In this embodiment, the user can upload wake-up resources as needed, input the env of the wake-up resources, fill in the wake-up recognition and positioning test set and the sound zone suppression test set, at least one wake-up recognition and positioning test set, upload the annotation file, input the total number of audio channels and the channel order, etc.

[0035] Further, after the user completes filling in the relevant information in the third task creation interface, click Next to enter the fourth task creation interface (such as Figure 6 As shown), in the first task creation interface, the user can enter the project name and project category, and choose whether to enter the project into the formula system, etc., and then click the "Start Training" button to complete the task creation and training.

[0036] S15. Generate a model training task based on the information filled in by the user in the first task creation interface or the second task creation interface, and the information filled in by the user in the third task creation interface, and send it to the training server for vehicle-mounted speech recognition model training.

[0037] An embodiment of the present application provides a vehicle-mounted speech recognition model training method for a vehicle-mounted speech recognition model training platform. Through this method, a user can fill in microphone information and customized training information or non-customized training information on the training platform according to the guidance of the task creation interface, thereby generating a model training task and sending it to a training server for vehicle-mounted speech recognition model training. The user does not need to make relevant investments in hardware and specific implementation methods of the training model, thereby improving the accuracy and efficiency of training the vehicle-mounted speech recognition model.

[0038] like Figure 7 The flowchart of another embodiment of the vehicle-mounted speech recognition model training method of the present application is shown. In this embodiment, the vehicle-mounted speech recognition model training method further includes: S16. Add the model training task to a task list, wherein the task list includes the status and operable options of the model training task; wherein the operable options include at least one of details, download, retry, cancel and delete.

[0039] S17. When the training server completes the in-vehicle speech recognition model training and generates model resources, it receives the model resources and updates the status of the model training task to test success.

[0040] At this time, the operable options include details, download and delete. The user can click the details button after the training task to view the relevant data of the training task. The specific relevant data is not limited in this application; the user can click the download button after the training task to download the trained model resources (for example, the in-vehicle speech recognition model and test report); the user can click the delete button after the training task to delete the training task.

[0041] S18: When the training server fails to train the vehicle-mounted speech recognition model, the status of the model training task is updated to test failure. The operable options at this time include details, retry and delete. The user can click the retry button after the training task to restart the training task.

[0042] S19: When the training server is in an occupied state, the state of the model training task is updated to be in training queue. At this time, the operable options include details and cancel, and the user can click the cancel button after the training task to cancel the training task.

[0043] In this embodiment, by following the execution progress of the training task in the task list and the status of the training task in real time, the user can always grasp the progress of the training task and perform different operations according to different training progress, so that the user can control the entire training process in stages.

[0044] In some embodiments, the first microphone information filling area includes at least one of the following: a microphone type option, a microphone quantity filling area, a sound zone quantity filling area, a loop quantity filling area, and a microphone arrangement information filling area; the method further includes: When the non-customized training mode option is selected, the training server selects a matching target public version model from multiple preset public version models based on at least one of the microphone type, number of microphones, number of voice zones, number of loops and microphone arrangement information in the model training task for in-vehicle speech recognition model training.

[0045] In this embodiment, when the user selects the non-customized training method, a target public version model that matches the relevant information entered by the user in the first microphone information filling area can be selected from a plurality of preset public version models through the training server, and the training of the vehicle-mounted speech recognition model can be implemented based on the public version model. It should be noted that the public version model in this application is a model resource obtained by the training platform in advance according to one or more combinations of different microphone types, number of microphones, number of sound zones, number of loops and microphone arrangement information. Therefore, the training process in this embodiment actually only refers to testing the selected target public version model based on the test data, and there is no need for repeated training, which greatly improves the efficiency of user training and obtaining the vehicle-mounted speech recognition model, and helps to improve the efficiency of the user's overall project.

[0046] In some embodiments, the second microphone information filling area includes at least one of the following: a microphone type option, a microphone quantity filling area, a sound zone quantity filling area, a loop quantity filling area, and a microphone arrangement information filling area; the method further includes: When the customized training mode option is selected, the training server performs in-vehicle speech recognition model training on the preset eabNet single model according to at least one of the microphone type, number of microphones, number of sound zones, number of loops and microphone arrangement information in the model training task.

[0047] In this embodiment, when the user selects the customized training mode, the preset eabNet single model can be obtained through the training server, and the eabNet single model can be trained according to at least one of the microphone type, number of microphones, number of sound zones, number of loops, and microphone arrangement information in the model training task and combined with the training data. The user can train the eabNet single model with relevant information such as the specific microphone type, number of microphones, number of sound zones, number of loops, and microphone arrangement information, and combine the training data to obtain a speech recognition model that is more accurately matched to the vehicle environment.

[0048] During the development process, the applicant also explored several other alternatives or substitutes before proposing the above embodiment. In terms of network training for signal processing training, the applicant mainly considered two methods: eabNet single model training and dccrn multi-model training. However, through in-depth research and analysis, the applicant found that: The eabNet single-model training solution has remarkable flexibility. It can cover the training of all sound zones with one task, which means that during the training process, we only need to submit one task to achieve comprehensive optimization of all sound zones. In addition, the solution also supports the use of different strategies to train multiple models under the same requirement, thereby further improving the success rate of customization.

[0049] The DCCRN multi-model training solution adopts a more detailed training strategy. It determines the number of training tasks based on the number of zones, and each zone has an independent model for training. However, the disadvantages of this solution are: it requires training multiple models, which increases the complexity and management difficulty of the model; at the same time, selecting the optimal checkpoint method is also relatively complicated and requires more time and effort; in addition, since the same zone may need to train the model multiple times, this also leads to a lower customization success rate.

[0050] Taking all the above factors into consideration, the applicant decided to adopt the eabNet solution training customization process to support the customization requirements of 2 microphones and 2 audio zones and 4 microphones and 4 audio zones. This is because the eabNet solution is more in line with the product implementation requirements in terms of overall performance and flexibility, and can also meet the actual usage scenarios of most users.

[0051] In some embodiments, the training platform includes a task list interface, which includes a task list and a task search box; the task search box includes at least one of a task ID search box, a task name search box, a training method search box, a status search box, a creator search box and a creation time search box; the method also includes: presenting the hit task in the task list according to at least one of the task ID, task name, training method, creator, and creation time input by the user.

[0052] The method of this embodiment sets a task search box on the training platform and sets a variety of different search fields for users to set search conditions, so that after users have created more training tasks, they can still accurately search for the training tasks they need and download related model resources as needed.

[0053] like Figure 8 The flowchart of another embodiment of the vehicle-mounted speech recognition model training method of the present application is shown. The vehicle-mounted speech recognition model training method is used to train a server, and the method includes: S21. Receive a model training task from the on-board speech recognition model training platform; the model training task is generated by the training platform based on the information filled in by the user in the first task creation interface or the second task creation interface, and the information filled in by the user in the third task creation interface; wherein, the first task creation interface includes a first microphone information filling area and a non-customized training information filling area; the second task creation interface includes a second microphone information filling area and a customized training information filling area.

[0054] S22: selecting a training model according to the training task, and performing vehicle-mounted speech recognition model training on the training model.

[0055] Exemplarily, the training server parses the received training task and selects a training model based on the parsing result. For example, when the training method in the parsing result is a customized training method, the preset eabNet single model is selected for training; when the training method in the parsing result is a non-customized method, according to the relevant information filled in and submitted by the user in the first microphone information filling area, a target public version model matching the relevant information entered by the user in the first microphone information filling area is selected from multiple preset public version models, and the training of the vehicle-mounted speech recognition model is implemented based on the public version model.

[0056] S23, sending the training resources obtained by training to the training platform. Exemplarily, the training resources include the trained vehicle-mounted speech recognition model and the test results.

[0057] The embodiment of the present application provides a method for training a vehicle-mounted speech recognition model for a training server. Through this method, a user can cooperate with the training method of the user's vehicle-mounted speech recognition model training platform, fill in microphone information and customized training information or non-customized training information on the training platform according to the guidance of the task creation interface, thereby generating a model training task and sending it to the training server for vehicle-mounted speech recognition model training. The training server parses the received training task and selects a training model based on the parsing result. The user does not need to make relevant investments in hardware and the specific implementation method of the training model, thereby improving the accuracy and efficiency of training the vehicle-mounted speech recognition model.

[0058] In some embodiments, the training platform generates an initial task creation interface in response to a user's task creation operation on the training platform; wherein the initial task creation interface is provided with a non-customized training mode option and a customized training mode option; the first microphone information filling area or the second microphone information filling area includes at least one of the following: a microphone type option, a microphone quantity filling area, a sound zone quantity filling area, a loop quantity filling area, and a microphone arrangement information filling area; The selecting a training model according to the training task and performing vehicle-mounted speech recognition model training on the training model includes: When the non-customized training mode option is selected, the training server selects a matching target public version model from a plurality of preset public version models according to at least one of the microphone type, the number of microphones, the number of sound zones, the number of loops, and the microphone arrangement information in the model training task for in-vehicle speech recognition model training; When the customized training mode option is selected, the training server performs in-vehicle speech recognition model training on the preset eabNet single model according to at least one of the microphone type, number of microphones, number of sound zones, number of loops and microphone arrangement information in the model training task.

[0059] The applicant conducted the following analysis in the process of implementing this application: In the field of in-vehicle signal processing, related technologies generally face problems such as poor human voice signal processing, difficulty in eliminating vehicle background noise, and difficulty in accurately separating sound areas. These problems seriously restrict the performance of in-vehicle voice recognition systems, especially in complex and changeable noise environments, where the recognition accuracy is often greatly reduced.

[0060] After in-depth analysis and research, we found that the defects of existing technologies mainly come from the following aspects: first, there is a lack of customized signal processing algorithms for the vehicle environment; second, there is a lack of efficient and accurate sound zone separation technology; third, there is a failure to fully utilize the vehicle hardware resources for optimization.

[0061] In order to solve at least one of the above problems, we have repeatedly considered and decided to develop a vehicle voice signal (for example, vehicle multi-zone signal) processing training platform. The platform aims to improve the accuracy and efficiency of human voice processing by outputting signal processing resources suitable for specific vehicle environments through customized training.

[0062] The overall design concept of this application is to build a comprehensive platform that integrates functions such as audio recording, preprocessing, feature extraction, model training and optimization. The noise reduction, separation, and positioning on the vehicle-mounted platform are implemented using a neural network model. There will also be traditional solutions, such as beamforming, blind source separation and other algorithms, which deeply analyze and process the audio signals in the vehicle environment to achieve effective separation of human voice and noise, accurate division of sound zones, and precise positioning of the target human voice. It should be noted that this application can use relevant existing technologies for noise reduction, separation, neural network models, beamforming, blind source separation and other algorithms on the vehicle-mounted platform, and this application does not limit them.

[0063] The specific principles are as follows: First, audio signals (training audio or test audio) are collected through the on-board microphone array; then, the collected signals are preprocessed, including noise reduction, filtering and other steps; then, the preprocessed signals are subjected to feature extraction using a deep learning algorithm to extract key features that reflect the characteristics of human voice and differences in sound ranges; finally, based on the extracted features, a signal processing model suitable for a specific vehicle environment is trained to output customized signal processing resources.

[0064] The in-vehicle multi-zone signal processing training platform of this application provides users with two types of signal processing resources: public resources and customized resources.

[0065] ①. The difference between public resources and customized resources Public resources are built on a common model and are applicable to a variety of different vehicle environments. Users can obtain corresponding public resources from the platform based on parameters such as the number of microphones in their vehicle system, the number of required sound zones, and the spacing between microphones. Public resources do not require additional model training, so they can be quickly provided to users and initially tested to ensure their applicability.

[0066] Customized resources are trained based on the user's specific vehicle environment and needs. The creation process of customized resources is relatively time-consuming due to the need for in-depth model training and optimization. However, customized resources can more accurately adapt to the user's vehicle environment and provide higher quality signal processing effects.

[0067] ② The timing diagram of a user creating a customized resource task through the platform is as follows: Fig. 9 As shown, the steps are as follows: Step 1: Task creation: Users create a custom resource task through the platform's user interface and enter the required parameter information, such as the number of microphones, the number of sound zones, the microphone spacing, etc.

[0068] Step 2: Task push: After the platform server receives the task created by the user, it pushes the task information to the Redis queue for processing.

[0069] Step 3: Training tasks: The underlying training service listens to redis tasks. Once a new task is pushed to the queue, the underlying algorithm module immediately obtains the task and starts model training. During the training process, the underlying algorithm will perform deep learning and optimization on the model based on the parameters and audio samples provided by the user.

[0070] Step 4: Resource generation and delivery: After a certain period of training, the underlying algorithm module will generate customized resources that meet user needs and save them to the specified storage location.

[0071] Step 5: Notify the server: The training service notifies the server that the task training is successful and provides signal processing resources and test reports.

[0072] Step 6 Download resources: Users can download and use the generated customized resources from the platform at any time.

[0073] ③、The underlying processing logic flow chart after the training service receives the task is as follows Fig.10 As shown, the steps are as follows: Step 1: Get input data: After the training service receives the task, it first gets input data from the task information, including but not limited to: Number of microphones: used to determine the number of channels for audio signal collection.

[0074] Number of audio zones: Defines the number of audio zones to be processed.

[0075] Microphone spacing: affects the spatial distribution characteristics of the audio signal.

[0076] Real vehicle training set: contains audio data collected in the actual vehicle environment, which is used for model training.

[0077] Real vehicle noise: Background noise data in the actual vehicle environment, used to simulate the real environment.

[0078] Microphone distribution information: describes the specific location of microphones in the vehicle.

[0079] Wake-up resource: A specific audio signal used to wake up the system.

[0080] Test set: An audio dataset used to evaluate the performance of the model.

[0081] Step 2: Preprocessing: Check the acquired training data and test data to ensure data quality, which generally includes the following checks: Signal-to-noise ratio check: Evaluates the ratio of audio signal to background noise to ensure the signal is clear.

[0082] Interference, sound effects, and other anomaly detection: Identify and remove abnormal audio that may affect model training.

[0083] Noise data check: Ensure that the noise data does not contain human voice to avoid interference with model training.

[0084] Test set channel order check: Ensure that the channel order of the test set audio data is consistent with the training set.

[0085] Step 3: Data processing: Perform audio format processing on the input training set and test set, and convert the audio in PCM format to WAV format for subsequent processing. Perform white noise processing on the audio data to simulate different noise environments, and adjust relevant parameters to meet the model training requirements. Convert the processed audio data into the format required for model training.

[0086] Step 4: Model training: Use eabnet (a neural network architecture) to learn the processed training data and generate a neural network model. During the training process, the model will continuously adjust parameters to optimize performance.

[0087] Step 5: Model testing: Test the wake-up resources through the wake-up recognition and positioning test set and the sound zone suppression test set, and record the performance of the model on the test set, including the number of successful wake-ups and the number of accurate positioning.

[0088] Step 6: Output model resource .bin and test report.

[0089] Through the above steps, the training service can efficiently process the input data, generate customized model resources suitable for in-vehicle multi-zone signal processing, and provide a detailed test report for user reference.

[0090] In the embodiment of the present application, by using the "vehicle multi-zone signal processing" self-training platform, the user has achieved significant technical effects in vehicle multi-zone signal processing, which is specifically manifested in the following aspects: ① Reduced labor costs: Users can complete model training independently through the platform without relying on external experts or teams. This greatly reduces investment in labor costs and enables enterprises to allocate resources more flexibly.

[0091] ② Shortened project cycle: The platform provides a fast and convenient training process. Users can start training tasks at any time without waiting for project scheduling. This significantly shortens the model training cycle and speeds up product iteration.

[0092] ③ Release of human resources: Due to the high degree of automation of the platform, the workload of research colleagues and testers in the model training process is reduced, allowing them to focus on other more important R&D tasks.

[0093] In summary, the in-vehicle multi-zone signal processing self-training platform of this application has achieved remarkable technical effects in reducing labor costs, shortening project cycles, improving performance, and selecting solutions. These effects not only improve the accuracy and reliability of in-vehicle voice interaction, but also save resources for enterprises, speed up product iteration, and promote the further development of in-vehicle voice technology.

[0094] The full English names and Chinese definitions of the English abbreviations appearing in this application are as follows: pcm: Pulse Code Modulation audio coding method; wav: Waveform Audio File audio file format; .bin: model resources, binary file; cpu:Central Processing Unit central processing unit; gpu:Graphics Processing Unit graphics processor; redis: Remote Dictionary Server remote dictionary service; eabNet: single model training; dccrn: multi-model training; Checkpoint: A snapshot of the model saved at a specific point in time during the training process.

[0095] In some embodiments, an embodiment of the present application provides an electronic device, comprising: at least one processor, and a memory communicatively connected to the at least one processor, wherein the memory stores instructions executable by the at least one processor, and the instructions are executed by the at least one processor so that the at least one processor can execute the steps of the in-vehicle speech recognition model training method described in any embodiment of the present application.

[0096] In some embodiments, an embodiment of the present application provides a storage medium on which a computer program is stored, characterized in that when the program is executed by a processor, the steps of the in-vehicle speech recognition model training method described in any embodiment of the present application are implemented.

[0097] In some embodiments, an embodiment of the present application provides a computer program product, including a computer program / instructions, characterized in that when the computer program / instructions are executed by a processor, the steps of the in-vehicle speech recognition model training method described in any embodiment of the present application are implemented.

[0098] It should be noted that, for the aforementioned method embodiments, for the sake of simplicity of description, they are all expressed as a series of actions combined, but those skilled in the art should be aware that the present application is not limited by the described order of actions, because according to the present application, certain steps can be performed in other orders or simultaneously. Secondly, those skilled in the art should also be aware that the embodiments described in the specification are all preferred embodiments, and the actions and modules involved are not necessarily required by the present application. In the above embodiments, the description of each embodiment has its own emphasis. For parts that are not described in detail in a certain embodiment, please refer to the relevant description of other embodiments.

[0099] Fig.11 is a schematic diagram of the hardware structure of an electronic device for executing a vehicle-mounted speech recognition model training method provided by another embodiment of the present application, such as Fig.11 As shown, the device includes: One or more processors 1110 and memory 1120, Fig.11 A processor 1110 is taken as an example.

[0100] The device for executing the in-vehicle speech recognition model training method may further include: an input device 1130 and an output device 1140 .

[0101] The processor 1110, the memory 1120, the input device 1130 and the output device 1140 may be connected via a bus or other means. Fig.11 The example of connecting through bus is taken in the following.

[0102] The memory 1120, as a non-volatile computer-readable storage medium, can be used to store non-volatile software programs, non-volatile computer executable programs and modules, such as program instructions / modules corresponding to the vehicle-mounted speech recognition model training method in the embodiment of the present application. The processor 1110 executes various functional applications and data processing of the server by running the non-volatile software programs, instructions and modules stored in the memory 1120, that is, the vehicle-mounted speech recognition model training method in the above method embodiment is implemented.

[0103] The memory 1120 may include a program storage area and a data storage area, wherein the program storage area may store an operating system and applications required for at least one function; the data storage area may store data created according to the use of the vehicle-mounted speech recognition model training device, etc. In addition, the memory 1120 may include a high-speed random access memory, and may also include a non-volatile memory, such as at least one disk storage device, a flash memory device, or other non-volatile solid-state storage device. In some embodiments, the memory 1120 may optionally include a memory remotely arranged relative to the processor 1110, and these remote memories may be connected to the vehicle-mounted speech recognition model training device via a network. Examples of the above-mentioned network include, but are not limited to, the Internet, an intranet, a local area network, a mobile communication network, and combinations thereof.

[0104] The input device 1130 can receive input digital or character information and generate signals related to user settings and function control of the vehicle-mounted speech recognition model training device. The output device 1140 can include display devices such as display screens.

[0105] The one or more modules are stored in the memory 1120, and when executed by the one or more processors 1110, the vehicle-mounted speech recognition model training method in any of the above method embodiments is executed.

[0106] The above-mentioned product can execute the method provided in the embodiment of the present application, and has the functional modules and beneficial effects corresponding to the execution method. For technical details not fully described in this embodiment, please refer to the method provided in the embodiment of the present application.

[0107] The electronic devices of the embodiments of the present application exist in various forms, including but not limited to: (1) Mobile communication devices: These devices are characterized by their mobile communication functions and their main purpose is to provide voice and data communications. These terminals include: smart phones (such as iPhone), multimedia phones, functional phones, and low-end phones.

[0108] (2) Ultra-mobile personal computer devices: These devices fall into the category of personal computers, have computing and processing capabilities, and generally also have mobile Internet access features. These terminals include: PDA, MID and UMPC devices, such as iPad.

[0109] (3) Portable entertainment devices: These devices can display and play multimedia content. They include audio and video players (such as iPods), handheld game consoles, e-books, as well as smart toys and portable car navigation devices.

[0110] (4) Server: A device that provides computing services. The server consists of a processor, hard disk, memory, system bus, etc. The server is similar to the general computer architecture, but because it needs to provide highly reliable services, it has higher requirements in terms of processing power, stability, reliability, security, scalability, manageability, etc.

[0111] (5) Other electronic devices with data interaction functions.

[0112] The device embodiments described above are merely illustrative, wherein the units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed on multiple network units. Some or all of the modules may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0113] Through the description of the above implementation methods, those skilled in the art can clearly understand that each implementation method can be implemented by means of software plus a general hardware platform, and of course, by hardware. Based on this understanding, the above technical solution, in essence, or the part that contributes to the relevant technology, can be embodied in the form of a software product, which can be stored in a computer-readable storage medium, such as ROM / RAM, a disk, an optical disk, etc., including a number of instructions for a computer device (which can be a personal computer, a server, or a network device, etc.) to execute the methods described in each embodiment or some parts of the embodiment.

[0114] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present application, rather than to limit it. Although the present application has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or make equivalent replacements for some of the technical features therein. However, these modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the embodiments of the present application.

Claims

1. A vehicle-mounted speech recognition model training method, used for a vehicle-mounted speech recognition model training platform, the method comprising: In response to the user's task creation operation on the training platform, an initial task creation interface is generated; wherein the initial task creation interface is provided with a non-customized training mode option and a customized training mode option; When the non-customized training mode option is selected, a first task creation interface is refreshed and generated; wherein the first task creation interface includes a first microphone information filling area and a non-customized training information filling area; When the customized training mode option is selected, a second task creation interface is refreshed and generated; wherein the second task creation interface includes a second microphone information filling area and a customized training information filling area; generating a third task creation interface in response to a user operation, wherein the third task creation interface includes a test information filling area; A model training task is generated based on the information filled in by the user in the first task creation interface or the second task creation interface, and the information filled in by the user in the third task creation interface, and is sent to the training server for vehicle-mounted speech recognition model training.

2. The method according to claim 1, characterized in that The method further comprises: Add the model training task to a task list, wherein the task list includes the status and operable options of the model training task; wherein the operable options include at least one of details, download, retry, cancel and delete; When the training server completes the vehicle-mounted speech recognition model training and generates model resources, receiving the model resources and updating the status of the model training task to test success; When the training server fails to perform vehicle-mounted speech recognition model training, updating the status of the model training task to test failure; When the training server is in an occupied state, the state of the model training task is updated to be in training queue.

3. The method according to claim 1, characterized in that The first microphone information filling area includes at least one of the following: a microphone type option, a microphone quantity filling area, a sound zone quantity filling area, a loop quantity filling area, and a microphone arrangement information filling area; the method further includes: When the non-customized training mode option is selected, the training server selects a matching target public version model from multiple preset public version models based on at least one of the microphone type, number of microphones, number of voice zones, number of loops and microphone arrangement information in the model training task for in-vehicle speech recognition model training.

4. The method according to claim 1, characterized in that: The second microphone information filling area includes at least one of the following: a microphone type option, a microphone quantity filling area, a sound zone quantity filling area, a loop quantity filling area, and a microphone arrangement information filling area; the method further includes: When the customized training mode option is selected, the training server performs in-vehicle speech recognition model training on the preset eabNet single model according to at least one of the microphone type, number of microphones, number of sound zones, number of loops and microphone arrangement information in the model training task.

5. The method according to any one of claims 2 to 4, characterized in that: The training platform includes a task list interface, which includes a task list and a task search box; the task search box includes at least one of a task ID search box, a task name search box, a training method search box, a status search box, a creator search box, and a creation time search box; The method further includes: presenting the hit task in the task list according to at least one of a task ID, a task name, a training method, a creator, and a creation time input by a user.

6. A vehicle-mounted speech recognition model training method, used for training a server, the method comprising: Receive model training tasks from the in-vehicle speech recognition model training platform; The model training task is generated by the training platform according to the information filled in by the user in the first task creation interface or the second task creation interface, and the information filled in by the user in the third task creation interface; wherein the first task creation interface includes a first microphone information filling area and a non-customized training information filling area; the second task creation interface includes a second microphone information filling area and a customized training information filling area; Selecting a training model according to the training task, and performing vehicle-mounted speech recognition model training on the training model; The training resources obtained through training are sent to the training platform.

7. The method according to claim 6, characterized in that The training platform generates an initial task creation interface in response to the user's task creation operation on the training platform; wherein the initial task creation interface is provided with a non-customized training mode option and a customized training mode option; the first microphone information filling area or the second microphone information filling area includes at least one of the following: a microphone type option, a microphone quantity filling area, a sound zone quantity filling area, a loop quantity filling area, and a microphone arrangement information filling area; The selecting a training model according to the training task and performing vehicle-mounted speech recognition model training on the training model includes: When the non-customized training mode option is selected, the training server selects a matching target public version model from a plurality of preset public version models according to at least one of the microphone type, the number of microphones, the number of sound zones, the number of loops, and the microphone arrangement information in the model training task for in-vehicle speech recognition model training; When the customized training mode option is selected, the training server performs in-vehicle speech recognition model training on the preset eabNet single model according to at least one of the microphone type, number of microphones, number of sound zones, number of loops and microphone arrangement information in the model training task.

8. An electronic device, comprising: At least one processor, and a memory communicatively connected to the at least one processor, wherein the memory stores instructions executable by the at least one processor, and the instructions are executed by the at least one processor so that the at least one processor can perform the steps of the method described in any one of claims 1 to 7.

9. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the program is executed by a processor, the steps of the method described in any one of claims 1 to 7 are implemented.

10. A computer program product comprising a computer program / instructions, characterized in that When the computer program / instructions are executed by a processor, the steps of the method described in any one of claims 1 to 7 are implemented.

Images