Automobile audio control method and system

Abstract

The invention relates to the technical field of information, and provides an automobile audio control method and system. The automobile audio control method comprises the following steps: acquiring voice data and limb infrared image data of a user in a current automobile; obtaining voice data features based on the voice data, and obtaining limb infrared image data features based on the limb infrared image data; performing feature fusion on the voice data features and the limb infrared image data features according to a confidence ratio to obtain fusion features; identifying the fusion feature to obtain a control instruction of the automobile audio; acquiring the real-time volume outside the vehicle and the real-time volume inside the vehicle; based on the real-time sound volume outside the automobile and the real-time volume inside the automobile, obtaining a correction coefficient of the automobile audio; and correcting the control instruction of the automobile audio based on the correction coefficient of the automobile audio to obtain a corrected control instruction of the automobile audio, and sending the corrected control instruction of the automobile audio to an automobile end for control.

Description

Technical Field

[0001] The present application relates to the field of information technology, and in particular to a method and system for controlling car audio. Background Art

[0002] People's demand for cars has changed from a simple means of transportation to a means of transportation that meets personalization and comfort needs. In this context, car audio systems came into being and gradually became an important part of cars. However, traditional car audio systems mainly rely on manual control by the driver, which will distract the driver's attention and cause unnecessary risks. In addition, users need to obtain sound information from outside the car while driving the vehicle. This sound information is of great significance for improving driving safety and assisting driving decisions. However, if the volume of the vehicle's audio is too loud, the user will not be able to accurately obtain sound information from outside the car, causing safety hazards.

[0003] The current technology has flaws and urgently needs to be improved. Summary of the invention

[0004] One object of the present application is to provide a method for controlling car audio, at least to solve the problem of how to improve the accuracy of user control of car audio under the premise of ensuring safe driving.

[0005] To achieve the above objectives, some embodiments of the present application provide the following aspects:

[0006] In a first aspect, some embodiments of the present application further provide a method for controlling a car audio system, the method comprising:

[0007] Acquire the voice data and body infrared image data of the user currently in the car;

[0008] Based on the voice data, voice data features are acquired, and based on the limb infrared image data, limb infrared image data features are acquired;

[0009] The voice data features and the limb infrared image data features are subjected to feature fusion according to the confidence ratio to obtain fusion features; wherein the confidence ratio is determined according to the volume score of the voice data and the quality score of the limb infrared image data;

[0010] Identifying the fusion features to obtain a control instruction for the car audio;

[0011] Get the real-time volume outside the car and the real-time volume inside the car;

[0012] Based on the real-time sound volume outside the vehicle and the real-time sound volume inside the vehicle, obtaining a correction coefficient for the vehicle audio;

[0013] The control instruction of the car audio is corrected based on the correction coefficient of the car audio to obtain the corrected control instruction of the car audio, and the corrected control instruction of the car audio is sent to the car end for control.

[0014] In a second aspect, some embodiments of the present application further provide a car audio control system, the system comprising:

[0015] A voice data and body infrared image data module, which is configured to obtain voice data and body infrared image data of a user currently in the car;

[0016] A feature module, configured to obtain voice data features based on the voice data, and to obtain limb infrared image data features based on the limb infrared image data;

[0017] A feature fusion module, configured to fuse the voice data features and the limb infrared image data features according to a confidence ratio to obtain a fused feature; wherein the confidence ratio is determined according to a volume score of the voice data and a quality score of the limb infrared image data;

[0018] A fusion feature recognition module, which is configured to recognize the fusion feature and obtain a control instruction for the car audio;

[0019] A real-time sound volume acquisition module inside and outside the vehicle, which is configured to acquire the real-time volume outside the vehicle and the real-time volume inside the vehicle;

[0020] A correction coefficient acquisition module for car audio, configured to acquire a correction coefficient for car audio based on the real-time sound volume outside the car and the real-time sound volume inside the car;

[0021] The car audio control module is configured to correct the car audio control instruction based on the car audio correction coefficient, obtain the corrected car audio control instruction, and send the corrected car audio control instruction to the car end for control.

[0022] In a third aspect, some embodiments of the present application further provide an electronic device, comprising: one or more processors; and a memory storing computer program instructions, wherein the computer program instructions, when executed, cause the processor to perform the steps of the method described above.

[0023] In a fourth aspect, some embodiments of the present application further provide a computer-readable medium having computer program instructions stored thereon, wherein the computer program instructions can be executed by a processor to implement the method as described above.

[0024] In a fifth aspect, some embodiments of the present application further provide a computer program product, comprising a computer program / instruction, which implements the steps of the method described above when executed by a processor.

[0025] Compared with the related art, in the solution provided by the embodiment of the present application, voice data and limb infrared image data of the user currently in the car are obtained; based on the voice data, voice data features are obtained, and based on the limb infrared image data, limb infrared image data features are obtained; the voice data features and the limb infrared image data features are feature fused according to the confidence ratio to obtain fused features; wherein the confidence ratio is determined according to the volume score of the voice data and the quality score of the limb infrared image data, the fused features are identified to obtain a control instruction for the car audio; the real-time volume outside the car and the real-time volume inside the car are obtained; based on the real-time sound volume outside the car and the real-time volume inside the car, a correction coefficient for the car audio is obtained; based on the correction coefficient of the car audio, the control instruction of the car audio is corrected to obtain a corrected control instruction for the car audio, and the corrected control instruction for the car audio is sent to the car end for control. Through the above configuration, the present application first obtains the voice data and limb infrared image data of the user in the current car, and then obtains the voice data features based on the voice data, obtains the limb infrared image data features based on the limb infrared image data, and fuses the voice data features and the limb infrared image data features according to the confidence ratio. Since the confidence ratio is determined according to the volume score of the voice data and the quality score of the limb infrared image data, when the quality score of the limb infrared image data is low, the confidence ratio is used to perform feature fusion, which can reduce the proportion of the quality of the limb infrared image data in the fusion feature, so as to ensure the driver's true intention as much as possible. Since the audio sensor in the car (set on the side close to the main driver, when the volume score of the user's voice data is low, the confidence ratio is used to perform feature fusion, which can reduce the proportion of the volume of the user's voice data in the fusion feature, so as to ensure the driver's true intention as much as possible. In order to improve the user's driving experience, since the fusion feature (multimodal feature) is composed of the voice data feature and the limb infrared image data feature, it can better express the driver's ideas compared with the single mode, improve the control accuracy and intelligence of the car audio, identify the fusion feature, and obtain the control command of the car audio. Since the user needs to obtain sound information from outside the car during the process of driving the vehicle, these sound information are crucial for driving safety. Therefore, this application calculates the correction coefficient of the car audio according to the real-time volume outside the car and the real-time volume inside the car, and corrects the control command of the car audio based on the correction coefficient of the car audio, obtains the corrected control command of the car audio, and sends the corrected control command of the car audio to the car end for control, so as to ensure that the user can obtain sound information from outside the car during the process of driving the vehicle. Therefore, this application can improve the accuracy of the user's control of the car audio under the premise of ensuring safe driving. BRIEF DESCRIPTION OF THE DRAWINGS

[0026] One or more embodiments are exemplarily described by pictures in the corresponding drawings, and these exemplified descriptions do not constitute limitations on the embodiments. Elements with the same reference numerals in the drawings represent similar elements, and unless otherwise stated, the figures in the drawings do not constitute proportional limitations.

[0027] Figure 1 It is a schematic flow chart of main steps of a method for controlling car audio according to an embodiment of the present application;

[0028] Figure 2 It is a schematic diagram of the main components of a car audio control system according to an embodiment of the application;

[0029] Figure 3 It is a schematic diagram of an exemplary structure of an electronic device according to the present application. DETAILED DESCRIPTION

[0030] In order to make the purpose, technical solution and advantages of the embodiments of the present application clearer, the technical solution in the embodiments of the present application will be clearly and completely described below in conjunction with the drawings in the embodiments of the present application. Obviously, the described embodiments are part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of this application.

[0031] See attached Figure 1 , Figure 1 1 is a flow chart of the main steps of a method according to an embodiment of the present application. Figure 1 As shown, the car audio control method in the embodiment of the present application mainly includes the following steps S101 to S107.

[0032] Step S101: Acquire the voice data and body infrared image data of the user currently in the car.

[0033] Step S102: based on the voice data, voice data features are acquired; based on the limb infrared image data, limb infrared image data features are acquired.

[0034] Step S103: Fusing the features of the speech data and the features of the infrared image data of the limbs according to the confidence ratio to obtain fused features, wherein the confidence ratio is determined according to the volume score of the speech data and the quality score of the infrared image data of the limbs.

[0035] Step S104: Identify the fusion features to obtain a control instruction for the car audio.

[0036] Step S105: Acquire the real-time volume outside the vehicle and the real-time volume inside the vehicle.

[0037] Step S106: obtaining a correction coefficient of the car audio based on the real-time sound volume outside the car and the real-time sound volume inside the car.

[0038] Step S107: the control instruction of the car audio is corrected based on the correction coefficient of the car audio to obtain the corrected control instruction of the car audio, and the corrected control instruction of the car audio is sent to the car end for control.

[0039] The audio sensor for acquiring the voice data of the user currently in the vehicle is arranged on the driver's side.

[0040] In this embodiment, the voice data of the user in the current car can be collected by the audio sensor in the car (set on the side close to the main driver), and the limb infrared image data of the user in the current car can be collected by the human infrared sensor in the car, and then the voice data is feature extracted to obtain the voice data feature, and the limb infrared image data is feature extracted to obtain the limb infrared image data feature, and the voice data feature and the limb infrared image data feature are feature fused according to the confidence ratio to obtain the fused feature. The confidence ratio can be obtained by formula (1).

[0041]

[0042] in, is the confidence ratio, β is the quality score of the limb infrared image data, and those skilled in the art can customize the quality score of the limb infrared image data (between 10 and 100) according to actual usage or obtain the quality score of the limb infrared image data by inputting the limb infrared image data into a reliable scoring model. is the volume score of the user's voice data, which can be set to 10 points for 0-30 decibels, 70 for 40-80 decibels, and 100 points for greater than 80 decibels. When the quality of the limb infrared image data is low, the confidence level is used for feature fusion to reduce the proportion of low-quality limb infrared image data in the fusion feature to ensure the accuracy of the final detection result. Since the audio sensor in the car is set on the side close to the main driver, other passengers far away from the audio sensor and the driver closest to the audio sensor simultaneously make the same decibel sound. The audio sensor will detect that the driver's volume is louder than that of other passengers. When the volume score of the user's voice data is low, the confidence level is used for feature fusion to reduce the proportion of the volume of the user's voice data in the fusion feature to ensure the driver's true intention as much as possible, so as to ensure the accuracy of the final detection result. The fusion feature is input into the preset deep learning model to obtain the control instructions of the car audio. The control instructions of the car audio include switching audio, increasing the volume, and decreasing the volume. However, since the user does need to obtain sound information from outside the car during driving, this sound information is crucial for driving safety. Such as traffic signal sounds: such as sirens, ambulances, etc., these sounds can remind the driver to pay attention to special vehicles or emergency situations and avoid them in time. Vehicle sounds: horns and brakes of other vehicles, etc. These sounds can convey the driving intentions or warning information of other vehicles and help the driver judge the dynamics of surrounding vehicles. Therefore, this application obtains the real-time volume outside the car by setting an audio sensor outside the car, and obtains the real-time volume inside the car through an audio sensor inside the car, and then obtains the correction coefficient of the car audio according to the real-time sound volume outside the car and the real-time volume inside the car. The correction coefficient of the car audio can be obtained by Formula 2:

[0043]

[0044] Among them, X is the correction coefficient of the car audio, F(W) is the real-time volume outside the car, F(N) is the real-time volume inside the car, F(W) and F(N) are both the average volume within a period of time, which can better reflect the overall volume size, thereby ensuring the correction accuracy, a represents the end time of collecting audio data, b represents the end time of collecting audio data, and K is the sound insulation coefficient of the car. Due to the different models of vehicles, their sound insulation coefficients are also different. Those skilled in the art can customize the sound insulation coefficient of the car according to actual use. Generally, the vehicle can be set to 1.1. The correction coefficient of the car audio only corrects the volume adjustment in the control command of the car audio. Specifically, when X is less than 0.8, the volume increase command in the control command of the car audio is not effective, and other commands are effective, thereby controlling the car volume within a reasonable range to ensure that the driver can accurately obtain information from outside the car, thereby ensuring driving safety. Finally, the control command of the car audio after the correction is sent to the car end for control.

[0045] In one embodiment, step S102 may further include steps S1021 to S1022:

[0046] Step S1021: Send the voice data to the trained audio extraction model to obtain the voice data features.

[0047] Step S1022: sending the limb infrared image data to a trained human infrared data extraction model to obtain the limb infrared image data features.

[0048] In one embodiment, the trained human infrared data extraction model is obtained according to the following steps S10221 to S10245:

[0049] Step S10221: Acquire historical limb infrared image data; wherein the historical limb infrared image data includes historical single-person limb infrared image data and historical multi-person limb infrared image data;

[0050] Step S10222: annotating the historical infrared image data of a single person's limbs to obtain annotated historical infrared image data of a single person's limbs;

[0051] Step S10223: annotating the limb infrared image data of the main driver in the historical limb infrared image data of multiple people, to obtain annotated historical limb infrared image data of multiple people;

[0052] Step S10224: acquiring a historical sample data set based on the annotated historical infrared image data of a single person's limbs and the annotated historical infrared image data of multiple people's limbs;

[0053] Step S10225: Based on the historical sample data set, the initial human infrared data extraction model is trained to obtain the trained human infrared data extraction model.

[0054] In this embodiment, historical limb infrared image data is obtained; wherein the historical limb infrared image data includes historical single-person limb infrared image data and historical multi-person limb infrared image data, the historical single-person limb infrared image data is data-labeled to obtain the labeled historical single-person limb infrared image data, such as labeling the left tilt of the user's limb as reducing the volume, and labeling the right tilt of the user's limb as increasing the volume, labeling the limb infrared image data of the main driver in the historical multi-person limb infrared image data, obtaining the labeled historical multi-person limb infrared image data, and obtaining the historical sample data set based on the labeled historical single-person limb infrared image data and the labeled historical multi-person limb infrared image data. Then, based on the historical sample data set, the initial human infrared data extraction model is trained to obtain the trained human infrared data extraction model.

[0055] In one embodiment, the human infrared data extraction model can use a convolutional neural network (CNN), which is a deep learning model that performs well in image processing and computer vision tasks. CNN can automatically learn features in images by imitating the working mode of the human visual system.

[0056] In one embodiment, after step S10221, step S102216 may be further included:

[0057] Step S102216: Perform data enhancement processing on the historical limb infrared image data.

[0058] In this embodiment, the image can be rotated by a certain angle, such as 90 degrees, 180 degrees or 270 degrees. Flipping: flipping the image horizontally or vertically.

[0059] Scaling: Change the size of an image. Cropping: Randomly select a part of an area from an image. Translation: Move an image in the x or y direction. Color Variation: Adjust the brightness, contrast, saturation and hue of an image. Noise Injection: Add noise to an image, such as Gaussian noise, salt and pepper noise, etc. Filtering: Apply different filters, such as Gaussian blur, sharpening, edge detection, etc. Occlusion: Add occlusions to an image to simulate occlusion. Affine Transformation: Perform a combined transformation of tilting, scaling and rotating an image. Data enhancement processing is performed on the historical limb infrared image data to increase the richness of the data set, thereby improving the accuracy of the model.

[0060] In one embodiment, after step S101, step S108 is further included:

[0061] Step S108: performing a first preprocessing on the limb infrared image data, and using the preprocessed limb infrared image data as input data of the trained human infrared data extraction model.

[0062] In this embodiment, the first preprocessing may include image cropping, image flipping and noise removal. The preprocessed limb infrared image data is used as input data of the trained human infrared data extraction model to improve the recognition accuracy of the final model.

[0063] In one embodiment, after step S101, step S109 is further included:

[0064] Step S109: performing a second preprocessing on the speech data, and using the preprocessed speech data as input data for the trained audio extraction model.

[0065] In this embodiment, the first preprocessing may include de-manipulation processing. The preprocessed speech data is used as input data of the trained audio extraction model to improve the recognition accuracy of the final model.

[0066] It should also be noted that the training of the audio feature extraction model is an iterative process, which usually follows the following steps: Data preprocessing: First, the raw audio data is preprocessed, which includes steps such as denoising, normalization, and segmentation, so that the model can better learn and extract features. Data enhancement: In order to improve the generalization ability of the model, the preprocessed data can be enhanced through various techniques such as time stretching, pitch change, adding noise, etc. Feature extraction: The enhanced preprocessed data is input into the audio feature extraction model, and the model will try to extract features that are helpful for subsequent tasks (such as speech recognition, etc.). Loss calculation: According to the extracted features and the corresponding labels (in this scenario, the labels for the audio data of the main speaker), the value of the loss function is calculated to evaluate the performance of the model. Back propagation and parameter update: Back propagation is performed using the loss value to calculate the gradient of each parameter, and the parameters of the model are updated according to the gradient to reduce the loss. Convergence judgment: After each iteration, check whether the model has converged. This is usually judged by monitoring whether the value of the loss function no longer decreases significantly within a certain threshold. Model saving: If the model has converged, save the trained audio feature extraction model; if it has not converged, continue with the next round of iterative training. End training: When the model converges or reaches the preset number of iterations, the training process ends.

[0067] It should also be noted that the audio feature extraction model can use a recurrent neural network (RNN), which is a neural network suitable for processing sequence data. It has a memory function and can capture the temporal dynamic characteristics of the input data. RNN is very useful in processing data with sequence characteristics such as text, speech, and time series.

[0068] Based on the above steps S101 to S107, voice data and limb infrared image data of the user currently in the car are obtained; based on the voice data, voice data features are obtained, and based on the limb infrared image data, limb infrared image data features are obtained; the voice data features and the limb infrared image data features are feature fused according to the confidence ratio to obtain fused features; wherein the confidence ratio is determined according to the volume score of the voice data and the quality score of the limb infrared image data, the fused features are identified to obtain a control instruction for the car audio; the real-time volume outside the car and the real-time volume inside the car are obtained; based on the real-time sound volume outside the car and the real-time volume inside the car, a correction coefficient for the car audio is obtained; based on the correction coefficient of the car audio, the control instruction for the car audio is corrected to obtain a corrected control instruction for the car audio, and the corrected control instruction for the car audio is sent to the car end for control. Through the above configuration, the present application first obtains the voice data and limb infrared image data of the user in the current car, and then obtains the voice data features based on the voice data, obtains the limb infrared image data features based on the limb infrared image data, and fuses the voice data features and the limb infrared image data features according to the confidence ratio. Since the confidence ratio is determined according to the volume score of the voice data and the quality score of the limb infrared image data, when the quality score of the limb infrared image data is low, the confidence ratio is used to perform feature fusion, which can reduce the proportion of the quality of the limb infrared image data in the fusion feature, so as to ensure the driver's true intention as much as possible. Since the audio sensor in the car (set on the side close to the main driver, when the volume score of the user's voice data is low, the confidence ratio is used to perform feature fusion, which can reduce the proportion of the volume of the user's voice data in the fusion feature, so as to ensure the driver's true intention as much as possible. In order to improve the user's driving experience, since the fusion feature (multimodal feature) is composed of the voice data feature and the limb infrared image data feature, it can better express the driver's ideas compared with the single mode, improve the control accuracy and intelligence of the car audio, identify the fusion feature, and obtain the control command of the car audio. Since the user needs to obtain sound information from outside the car during the process of driving the vehicle, these sound information are crucial for driving safety. Therefore, this application calculates the correction coefficient of the car audio according to the real-time volume outside the car and the real-time volume inside the car, and corrects the control command of the car audio based on the correction coefficient of the car audio, obtains the corrected control command of the car audio, and sends the corrected control command of the car audio to the car end for control, so as to ensure that the user can obtain sound information from outside the car during the process of driving the vehicle. Therefore, this application can improve the accuracy of the user's control of the car audio under the premise of ensuring safe driving.

[0069] The step division of the above methods is only for the purpose of clear description. When implemented, they can be combined into one step or some steps can be split and decomposed into multiple steps. As long as they include the same logical relationship, they are all within the scope of protection of this patent; adding insignificant modifications to the algorithm or process or introducing insignificant designs without changing the core design of the algorithm and process are all within the scope of protection of this patent.

[0070] See attached Figure 2 , Figure 2 1 is a schematic diagram of the main components of a car audio control system according to an embodiment of the application. Figure 2 As shown, the control system of the car audio in the embodiment of the present application mainly includes a voice data and body infrared image data module 11, a feature module 12, a feature fusion module 13, a fusion feature recognition module 14, a real-time sound volume acquisition module 15 inside and outside the car, a car audio correction coefficient acquisition module 16 and a car audio control module 17. In some embodiments, one or more of the voice data and body infrared image data module 11, the feature module 12, the feature fusion module 13, the fusion feature recognition module 14, the real-time sound volume acquisition module 15 inside and outside the car, the car audio correction coefficient acquisition module 16 and the car audio control module 17 can be combined into one module. In some embodiments, a voice data and limb infrared image data module 11 is configured to obtain voice data and limb infrared image data of a user currently in the car; a feature module 12 is configured to obtain voice data features based on the voice data, and obtain limb infrared image data features based on the limb infrared image data; a feature fusion module 13 is configured to fuse the voice data features and the limb infrared image data features according to the confidence ratio to obtain fusion features; a fusion feature recognition module 14 is configured to recognize the fusion features and obtain a control instruction for the car audio; a real-time sound volume acquisition module 15 inside and outside the car is configured to obtain the real-time volume outside the car and the real-time volume inside the car; a car audio correction coefficient acquisition module 16 is configured to obtain a car audio correction coefficient based on the real-time sound volume outside the car and the real-time volume inside the car; a car audio control module 17 is configured to correct the car audio control instruction based on the car audio correction coefficient to obtain the corrected car audio control instruction, and send the corrected car audio control instruction to the car end for control.

[0071] It is worth mentioning that all modules involved in this embodiment are logic modules. In practical applications, a logic unit can be a physical unit, a part of a physical unit, or a combination of multiple physical units. In addition, in order to highlight the innovative part of this application, this embodiment does not introduce units that are not closely related to solving the technical problems proposed by this application, but this does not mean that there are no other units in this embodiment.

[0072] In addition, some embodiments of the present application also provide an electronic device. The electronic device may be a digital computer in various forms, such as a laptop computer, a desktop computer, a workbench, a personal digital assistant, a server, a blade server, a mainframe computer, etc. The electronic device may also be a mobile device in various forms, such as a personal digital processing, a cellular phone, a smart phone, a wearable device, and other similar computing devices.

[0073] The electronic device includes: one or more processors; and a memory storing computer program instructions, wherein when the computer program instructions are executed, the processor executes the steps of the method provided in any one or more of the above embodiments. Figure 3 An exemplary structural diagram of the electronic device is disclosed. Figure 3 As shown, the electronic device includes: one or more processors 1101, memory 1102, and interfaces for connecting various components, including high-speed interfaces and low-speed interfaces. The various components are connected to each other using different buses, and can be installed on a common mainboard or installed in other ways as needed. The processor can process instructions executed in the electronic device, including instructions stored in or on the memory to display the graphical information of the GUI on an external input / output device (such as a display device coupled to the interface). In some other embodiments, if necessary, multiple processors and / or multiple buses can be used together with multiple memories and multiple memories. Similarly, multiple electronic devices can be connected, and each device provides some necessary operations (for example, as a server array, a group of blade servers, or a multi-processor system). Among them, the components shown in this article, their connections and relationships, and their functions are only used as examples, and are not intended to limit the implementation of the present application described and / or required herein.

[0074] The electronic device may further include: an input device 1103 and an output device 1104. The processor 1101, the memory 1102, the input device 1103 and the output device 1104 may be connected via a bus or other means. Figure 3 The example of connecting through bus is taken in the following.

[0075] The input device 1103 can receive input digital or character information, and generate key signal input related to the user settings and function control of the electronic device, such as a touch screen, a keypad, a mouse, a track pad, a touch pad, an indicator bar, one or more mouse buttons, a trackball, a joystick and other input devices. The output device 1104 may include a display device, an auxiliary lighting device (e.g., an LED) and a tactile feedback device (e.g., a vibration motor), etc. The display device may include, but is not limited to, a liquid crystal display (LCD), a light emitting diode (LED) display and a plasma display. In some embodiments, the display device may be a touch screen.

[0076] To provide interaction with a user, the electronic device may be a computer. The computer has: a display device (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor) for displaying information to the user; and a keyboard and a pointing device (e.g., a mouse or a trackball), through which the user can provide input to the computer. Other types of devices may also be used to provide interaction with the user; for example, the feedback provided to the user may be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user may be received in any form (including acoustic input, voice input, or tactile input).

[0077] In the embodiments of the present application, a computer program / instruction is stored on a computer-readable medium, and when the computer program / instruction is executed by a processor, the steps of the method provided by any one or more of the above embodiments are implemented. The computer-readable medium may be included in the electronic device described in the above embodiments; or it may exist independently without being assembled into the device. The above computer-readable medium carries one or more computer-readable instructions.

[0078] The memory 1102 can be used as a non-transient computer-readable storage medium, which can be used to store non-transient software programs, non-transient computer executable programs and modules. The processor 1101 executes various functional applications and data processing of the server by running the non-transient software programs, instructions and modules stored in the memory 1102, so as to implement the program instructions / modules corresponding to the method provided by any one or more embodiments in the embodiments of the present application.

[0079] The memory 1102 may include a program storage area and a data storage area, wherein the program storage area may store an operating system, an application required for at least one function; the data storage area may store data created according to the use of the electronic device, etc. In addition, the memory 1102 may include a high-speed random access memory, and may also include a non-transient memory, such as at least one disk storage device, a flash memory device, or other non-transient solid-state storage device. In some embodiments, the memory 1102 may optionally include a memory remotely arranged relative to the processor 1101, and these remote memories may be connected to the electronic device via a network. Examples of the above-mentioned network include, but are not limited to, the Internet, an intranet, a local area network, a mobile communication network, and combinations thereof.

[0080] It should be noted that the computer-readable medium described in the present application may be a computer-readable signal medium or a computer-readable storage medium or any combination of the above two. The computer-readable medium may be, for example, but not limited to, an electrical, magnetic, optical, electromagnetic, infrared or semiconductor system, device or device, or any combination of the above. More specific examples of computer-readable storage media may include, but are not limited to: an electrical connection with one or more wires, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the above. In the present application, a computer-readable medium may be any tangible medium containing or storing a program that can be used by or in combination with an instruction execution system, device or device.

[0081] Computer readable media include permanent and non-permanent, removable and non-removable media, and can be implemented by any method or technology to store information. Information can be computer readable instructions, data structures, modules of programs or other data. Examples of computer storage media include, but are not limited to, phase change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technology, read-only compact disk (CD-ROM), digital versatile disk (DVD) or other optical storage, magnetic cassettes, magnetic tape disk storage or other magnetic storage devices or any other non-transmission medium that can be used to store information that can be accessed by a computing device.

[0082] Computer program code for performing the operations of the present application may be written in one or more programming languages ​​or a combination thereof, including object-oriented programming languages, such as Java, Smalltalk, C++, and conventional procedural programming languages, such as "C" language or similar programming languages. The program code may be executed entirely on the user's computer, partially on the user's computer, as an independent software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In the case of a remote computer, the remote computer may be connected to the user's computer via any type of network, including a local area network (LAN) or a wide area network (WAN), or may be connected to an external computer (e.g., using an Internet service provider to connect via the Internet).

[0083] In the above-described embodiments, it can be implemented in whole or in part by software, hardware, firmware or any combination thereof. For example, it can be implemented by using an application specific integrated circuit (ASIC), a general-purpose computer or any other similar hardware device. In certain embodiments, the software program of the present application can be executed by a processor to implement the above steps or functions. Similarly, the software program of the present application (including relevant data structures) can be stored in a computer-readable recording medium, for example, a RAM memory, a magnetic or optical drive or a floppy disk and similar devices. In addition, some steps or functions of the present application can be implemented by hardware, for example, as a circuit that cooperates with a processor to perform each step or function.

[0084] The computer program product provided in the embodiment of the present application includes one or more computer programs / instructions, and when the computer program / instructions are executed by the processor, the process or function described in the embodiment of the present application is generated in whole or in part. The computer may be a general-purpose computer, a special-purpose computer, a computer network, or other programmable devices. The computer instructions may be stored in a computer-readable storage medium, or transmitted from one computer-readable storage medium to another computer-readable storage medium, for example, the computer instructions may be transmitted from a website site, a computer, a server or a data center by wired (e.g., coaxial cable, optical fiber, digital subscriber line (DSL)) or wireless (e.g., infrared, wireless, microwave, etc.) mode to another website site, computer, server or data center. The computer-readable storage medium may be any available medium that a computer can access or a data storage device such as a server or a data center that includes one or more available media integrations. The available medium may be a magnetic medium, (e.g., a floppy disk, a hard disk, a tape), an optical medium (e.g., a DVD), or a semiconductor medium (e.g., a solid-state drive so lid state di sk (SSD)), etc.

[0085] The flow chart or block diagram in the accompanying drawings shows the possible architecture, function and operation of the equipment, method and computer program product according to various embodiments of the present application. In this regard, each box in the flow chart or block diagram can represent a module, a program segment or a part of a code, and the module, a program segment or a part of a code contains one or more executable instructions for realizing the specified logical function. It should also be noted that in some alternative implementations, the functions marked in the box can also occur in a different order from the order marked in the accompanying drawings. For example, two boxes represented in succession can actually be executed substantially in parallel, and they can sometimes be executed in the opposite order, depending on the functions involved. It should also be noted that each box in the block diagram and / or flow chart, and the combination of the boxes in the block diagram and / or flow chart can be implemented with a dedicated system for hardware that performs a specified function or operation, or can be implemented with a combination of dedicated hardware and computer instructions.

[0086] The scope of the present application is defined by the appended claims rather than the above description, and it is intended that all changes that fall within the meaning and scope of the equivalent elements of the claims are included in the present application. Any figure mark in the claims should not be regarded as limiting the claims involved. In addition, it is clear that the word "comprising" does not exclude other units or steps, and the singular does not exclude the plural. Multiple units or devices stated in the device claim may also be implemented by one unit or device through software or hardware. The words "first", "second", etc. are only used to distinguish the description, and do not indicate any particular order, nor can they be understood as indicating or implying relative importance.

[0087] The above is only a specific implementation of the present application, but the protection scope of the present application is not limited thereto. Any person skilled in the art can easily mention changes or substitutions within the technical scope disclosed in the present application, which should be included in the protection scope of the present application. Therefore, the protection scope of the present application should be based on the protection scope of the claims, and the above embodiments should be regarded as exemplary and non-restrictive.

Claims

1. A method for controlling car audio, characterized in that: The method comprises: Acquire the voice data and body infrared image data of the user currently in the car; Based on the voice data, voice data features are acquired, and based on the limb infrared image data, limb infrared image data features are acquired; The voice data features and the limb infrared image data features are subjected to feature fusion according to the confidence ratio to obtain fusion features; wherein the confidence ratio is determined according to the volume score of the voice data and the quality score of the limb infrared image data; Identifying the fusion features to obtain a control instruction for the car audio; Get the real-time volume outside the car and the real-time volume inside the car; Based on the real-time sound volume outside the vehicle and the real-time sound volume inside the vehicle, obtaining a correction coefficient for the vehicle audio; Correcting the control instruction of the car audio based on the correction coefficient of the car audio to obtain a corrected control instruction of the car audio, and sending the corrected control instruction of the car audio to the car end for control; The audio sensor for acquiring the voice data of the user currently in the vehicle is arranged on the driver's side.

2. The method for controlling the car audio according to claim 1, characterized in that: The method of acquiring voice data features based on the voice data and acquiring limb infrared image data features based on the limb infrared image data includes: Sending the voice data to the trained audio extraction model to obtain the voice data features; The limb infrared image data is sent to a trained human infrared data extraction model to obtain the limb infrared image data features.

3. The method for controlling the car audio according to claim 2, characterized in that: The method further comprises obtaining the trained human infrared data extraction model according to the following steps: Acquire historical limb infrared image data; wherein the historical limb infrared image data includes historical single-person limb infrared image data and historical multi-person limb infrared image data; Performing data labeling on the historical infrared image data of a single person's limbs to obtain the labeled historical infrared image data of a single person's limbs; Performing data labeling on the limb infrared image data of the main driver in the historical limb infrared image data of multiple people to obtain the labeled historical limb infrared image data of multiple people; Based on the annotated historical infrared image data of a single person's limbs and the annotated historical infrared image data of multiple people's limbs, a historical sample data set is acquired; Based on the historical sample data set, the initial human infrared data extraction model is trained to obtain the trained human infrared data extraction model.

4. The method for controlling the car audio according to claim 3, characterized in that: After acquiring the historical limb infrared image data, the method further includes: Data enhancement processing is performed on the historical limb infrared image data.

5. The method for controlling the car audio according to claim 2, characterized in that: After acquiring the voice data and body infrared image data of the user currently in the car, the method further includes: The limb infrared image data is subjected to a first preprocessing, and the preprocessed limb infrared image data is used as input data of the trained human infrared data extraction model.

6. The method for controlling the car audio system according to claim 2, characterized in that: After acquiring the voice data and body infrared image data of the user currently in the car, the method further includes: The speech data is subjected to a second preprocessing, and the preprocessed speech data is used as input data of the trained audio extraction model.

7. A car audio control system, characterized in that: The system comprises: A voice data and body infrared image data module, which is configured to obtain voice data and body infrared image data of a user currently in the car; A feature module, configured to obtain voice data features based on the voice data, and to obtain limb infrared image data features based on the limb infrared image data; A feature fusion module, configured to fuse the voice data features and the limb infrared image data features according to a confidence ratio to obtain a fused feature; A fusion feature recognition module, which is configured to recognize the fusion feature and obtain a control instruction for the car audio; A real-time sound volume acquisition module inside and outside the vehicle, which is configured to acquire the real-time volume outside the vehicle and the real-time volume inside the vehicle; A correction coefficient acquisition module for car audio, configured to acquire a correction coefficient for car audio based on the real-time sound volume outside the car and the real-time sound volume inside the car; The car audio control module is configured to correct the car audio control instruction based on the car audio correction coefficient, obtain the corrected car audio control instruction, and send the corrected car audio control instruction to the car end for control.

8. An electronic device, characterized in that: The electronic device comprises: one or more processors; and A memory storing computer program instructions, which, when executed, cause the processor to perform the steps of the method as claimed in any one of claims 1 to 6.

9. A computer readable medium having a computer program / instruction stored thereon, characterized in that: When the computer program / instructions are executed by a processor, the steps of the method according to any one of claims 1 to 6 are implemented.

10. A computer program product comprising a computer program / instructions, characterized in that When the computer program / instructions are executed by a processor, the steps of the method according to any one of claims 1 to 6 are implemented.

Images