Multimedia control method and system, vehicle-mounted terminal and medium

Abstract

The invention relates to a multimedia control method and system, a vehicle-mounted terminal and a medium. The method is applied to a multimedia system, the multimedia system corresponds to at least two groups of multimedia equipment, each group of multimedia equipment comprises input equipment and output equipment, and the method comprises the following steps: acquiring a blowing audio set of each group of input equipment within a preset time range; performing sound source localization on each acquired blowing audio set, and screening and extracting a target audio; inputting each extracted target audio into an instruction generation model, extracting the number of blowing segments or the duration of the blowing segments from the target audio, and generating a control instruction corresponding to each target audio according to the number of the blowing segments or the duration of the blowing segments; and controlling a corresponding output device to execute a preset corresponding operation based on the control instruction. The problem that the user operation instruction cannot be accurately recognized in the prior art is solved.

Description

Technical Field

[0001] The present invention relates to the field of multimedia control, and in particular to a multimedia control method, system, vehicle-mounted terminal and medium. Background Art

[0002] With the development of intelligent technology, in-vehicle entertainment systems are becoming more and more popular in modern cars. The in-vehicle entertainment system not only provides functions such as navigation, music playback, and video playback, but also can perform related operations through voice control, gesture control, and the like. However, the current control method still has many disadvantages in certain usage scenarios. For example, when the control method is voice control, the music or video content being played will be forced to pause while the user issues a command to activate the voice assistant, resulting in a decrease in user experience. When the control method is gesture control, if the user operates in a lying position, the accuracy of gesture recognition will also be reduced. Therefore, it is necessary to provide a multimedia control method, system, in-vehicle terminal, and medium. Summary of the invention

[0003] The present invention provides a multimedia control method, which improves the problem in the prior art that the user operation instructions cannot be accurately identified.

[0004] A multimedia control method provided by the present invention is applied to a multimedia system, wherein the multimedia system corresponds to at least two groups of multimedia devices, each group of multimedia devices includes an input device and an output device, and the method includes: obtaining a blowing audio set of each group of input devices within a preset time range; performing sound source positioning on each acquired blowing audio set, screening and extracting target audio; inputting each extracted target audio into an instruction generation model respectively, extracting the number of blowing segments or the duration of the blowing segments from the target audio, and generating a control instruction corresponding to each target audio according to the number of blowing segments or the duration of the blowing segments; and based on the control instruction, controlling the corresponding output device to perform a preset corresponding operation.

[0005] In one embodiment of the present invention, the obtaining of the blowing audio set of each group of input devices within a preset time range includes: obtaining an initial blowing audio set of each group of input devices within the preset time range; filtering each blowing audio in the initial blowing audio set to remove noise in the blowing audio, and using the remaining blowing audios as the blowing audio after noise filtering; and performing echo elimination on each blowing audio after noise filtering to obtain the blowing audio set.

[0006] In one embodiment of the present invention, the sound source positioning of each acquired blowing audio set, screening and extracting the target audio, includes: for each blowing audio set: selecting a blowing audio as a candidate audio, extracting the audio features and time mark of the candidate audio; screening the blowing audio with the same audio features from other blowing audio sets as the reference audio, and obtaining the corresponding time mark; wherein the time mark is the time when the blowing audio arrives at the corresponding input device; using the time difference positioning method, according to the time mark of the candidate audio and the time mark of all the reference audios, determining the sound source position of the reference audio, and judging whether the sound source position is located in a preset valid area: if so, extracting the candidate audio as the target audio of the blowing audio set; otherwise, continuing to select another blowing audio as the candidate audio, and repeating the confirmation of the target audio.

[0007] In one embodiment of the present invention, the time difference positioning method is used to determine the sound source position of the reference audio according to the time mark of the candidate audio and the time mark of all reference audios, and to judge whether the sound source position is located in a preset valid area, including: calculating the time difference between the candidate audio and each reference audio according to the time mark of the candidate audio and the time mark of all reference audios; determining the sound source position of the candidate audio based on each time difference and the preset position of the corresponding input device; judging whether the sound source position is within the preset valid area of ​​the input device corresponding to the candidate audio: if so, the blown audio is the target audio of the reference set; otherwise, the blown audio is the non-target audio of the reference set.

[0008] In one embodiment of the present invention, the output device is a speaker, and based on the control instruction, controlling the corresponding output device to perform a corresponding operation includes: parsing the control instruction, determining the operation content and the speaker identification that executes the operation content: if the operation content is switching, switching to the next audio, and sending the switched audio to the corresponding speaker according to the speaker identification, so that the speaker plays the corresponding audio within a preset sound zone; if the operation content is pausing, controlling the corresponding speaker to stop playing according to the speaker identification.

[0009] In one embodiment of the present invention, the output device includes a speaker and a screen, and the corresponding output device is controlled to perform a corresponding operation based on the control instruction, including: parsing the control instruction, determining the operation content and the speaker identifier and screen identifier for executing the operation content; performing corresponding operations on the currently played audio content and the corresponding video content according to the operation content; sending the operated video content to the corresponding screen and the operated audio content to the corresponding speaker based on the screen identifier and the speaker identifier, so as to control the screen to present the operated video content, and control the speaker to play the operated audio content within a preset sound zone.

[0010] In one embodiment of the present invention, after controlling the corresponding output device to perform the corresponding operation based on the control instruction, it also includes: determining whether an operation success response is received from the corresponding output device within a preset receiving time: if so, recording the operation success information and keeping the current control state of the output device unchanged; otherwise, recording the operation failure information and issuing a prompt to check the device connection.

[0011] In one embodiment of the present invention, a multimedia system is further provided, wherein the multimedia system corresponds to at least two groups of multimedia devices, each group of multimedia devices includes an input device and an output device, and the system includes: a data acquisition module, used to acquire a blowing audio set of each group of input devices within a preset time range; an audio extraction module, used to perform sound source positioning on each acquired blowing audio set, filter and extract target audio; an instruction generation module, used to input each extracted target audio into an instruction generation model respectively, extract the number of blowing segments or the duration of the blowing segments from the target audio, and generate a control instruction corresponding to each target audio according to the number of blowing segments or the duration of the blowing segments; and a control module, used to control the corresponding output device to perform a preset corresponding operation based on the control instruction.

[0012] In one embodiment of the present invention, a vehicle-mounted terminal is also provided, comprising: one or more processors; a storage device for storing one or more programs, and when the one or more programs are executed by the one or more processors, the vehicle-mounted terminal implements any of the multimedia control methods described above.

[0013] In one embodiment of the present invention, a computer-readable storage medium is further provided, on which a computer program is stored. When the computer program is executed by a processor of a computer, the computer executes any of the multimedia control methods described above.

[0014] The present invention proposes a multimedia control method, system, vehicle-mounted terminal and medium. By acquiring and analyzing the blowing audio set of each input device within a preset time range, using sound source localization technology to accurately locate and filter out the target audio, inputting the target audio into the command generation model, and generating corresponding control instructions according to the number of blowing segments, no matter what posture the user is in, complex multimedia control can be achieved through simple blowing actions. In addition, this method can ensure that the multimedia content is played continuously without interruption, effectively improve the user experience, and improve the problem that the existing method cannot accurately identify user operation instructions. BRIEF DESCRIPTION OF THE DRAWINGS

[0015] Figure 1 A schematic diagram of a flow chart of a multimedia control method provided by an embodiment of the present invention;

[0016] Figure 2 Shown is a schematic diagram of the positions of multimedia devices according to an embodiment of the present invention;

[0017] Figure 3 Shown is a structural block diagram of a multimedia system provided by an embodiment of the present invention;

[0018] Figure 4 Shown is a structural schematic diagram of a vehicle-mounted terminal according to an embodiment of the present invention. DETAILED DESCRIPTION

[0019] The following describes the embodiments of the present invention by specific examples, and those skilled in the art can easily understand other advantages and effects of the present invention from the contents disclosed in this specification. The present invention can also be implemented or applied through other different specific embodiments, and the details in this specification can also be modified or changed in various ways based on different viewpoints and applications without departing from the spirit of the present invention. It should be noted that the following embodiments and features in the embodiments can be combined with each other without conflict.

[0020] It should be noted that the illustrations provided in the following embodiments are only schematic illustrations of the basic concept of the present invention, and thus the drawings only show components related to the present invention rather than being drawn according to the number, shape and size of components in actual implementation. In actual implementation, the type, quantity and proportion of each component may be changed arbitrarily, and the component layout may also be more complicated.

[0021] In the following description, numerous details are discussed to provide a more thorough explanation of the embodiments of the present invention. However, it is obvious to those skilled in the art that the embodiments of the present invention can be implemented without these specific details. In other embodiments, well-known structures and devices are shown in the form of block diagrams rather than in detail to avoid making the embodiments of the present invention difficult to understand.

[0022] The present invention provides a multimedia control method, which acquires and analyzes the blowing audio set of each input device within a preset time range, uses sound source localization technology to accurately locate and filter out the target audio, inputs the target audio into the instruction generation model, generates corresponding control instructions according to the number of blowing segments, and realizes complex multimedia control through simple blowing actions regardless of the user's posture. The problem that the existing method cannot accurately identify the user's operation instructions is improved. The method of the present invention is applicable to multi-posture operations. Whether the user is lying or sitting, the user can use this method to control the corresponding multimedia device without using more tiring body interaction, and this method will not generate additional voice interference, will not interfere with other users, and improve the user experience. In addition, compared with voice commands that interrupt the playback of content, the method described in the present invention can ensure that the continuous playback of multimedia content is not interrupted, effectively improving the user experience. Furthermore, this method does not need to be specially developed for language and has wide universality.

[0023] See also Figure 1 The multimedia control method is applied to a multimedia system, wherein the multimedia system corresponds to a plurality of groups of multimedia devices, each group of multimedia devices includes an input device and an output device, and comprises the following steps:

[0024] S1. Obtain a set of blowing audios from each group of input devices within a preset time range.

[0025] A set of multimedia devices is configured at each position of the vehicle to realize the configuration of each user having an independent sound zone, wherein the multimedia devices include input devices and output devices. When a user needs to control his multimedia device, he will blow towards the corresponding input device. The blowing sound is used as a trigger signal. The timing function of the multimedia system will be triggered at the moment when the user starts blowing. Within a preset time range, the multimedia system will continuously collect audio data from each input device to form a blowing audio set of each input device. Among them, the input device can be an omnidirectional microphone or a directional microphone. In order to further improve the accuracy of sound capture and reduce the interference of environmental noise, a directional microphone can be used as an input device to capture sound in a specific direction. The output device is an independent media terminal for each user, which includes headphones or a screen. The blowing audio set refers to all audio data generated by the user's blowing captured by the corresponding input device within a preset time range. The audio data contains one or more blowing sounds, and each blowing sound is used as a separate blowing segment to trigger the corresponding output device to perform related functions.

[0026] In one embodiment of the present invention, obtaining a blowing audio set of each group of input devices within a preset time range includes:

[0027] Get the initial blowing audio set of each group of input devices within a preset time range;

[0028] Filtering each blowing audio in the initial blowing audio set to remove noise in the blowing audio, and using the remaining blowing audio as the blowing audio after noise filtering;

[0029] Echo cancellation is performed on each blowing audio after noise filtering to obtain a blowing audio set.

[0030] When the user starts to blow towards a certain input device, within a preset time range (such as 6 seconds after the user starts to blow), the multimedia system will continue to collect the blowing audio captured by all input devices to form an initial blowing audio set collected by each input device. Considering that the blowing audio set obtained in this way may include background noise such as noise, in order to improve the accuracy of subsequent audio recognition, the present invention will filter each blowing audio in the initial blowing audio set, and by setting the frequency range of the bandpass filter, retain the audio signal within the human voice frequency range, and filter out high-frequency and low-frequency signals. These processed audios are used as noise-filtered blowing audio for subsequent processes. Considering that when a user blows in a vehicle, the blowing sound of an adjacent user will form an echo, thereby reducing the clarity of the sound. In order to improve the clarity of the sound and improve the accuracy of subsequent model recognition, the above-mentioned noise-filtered blowing audios will be echo-eliminated one by one to eliminate the echo components therein. The processed blowing audio of each input device forms the blowing audio set of the device.

[0031] S2. Localize the sound source of each acquired blowing audio set, filter and extract the target audio.

[0032] In order to identify and respond to the instructions of each user, a corresponding input device is set in front of each user, for example, device A is in front of user A, and device B is in front of user B. However, due to the sound collection function of the microphone, each blowing audio set may include both the blowing audio of the corresponding user and the interference audio of other users. In particular, when two or more users in the car blow air to their respective devices at the same time, such interference is inevitable. For example, Figure 2As shown, if user A blows into input device A, and user B also blows into input device B, then the blowing audio collection collected by input device A may include both the blowing sound of user A and the blowing sound of user B. Therefore, it is necessary to use sound source localization technology to accurately determine the specific location of the blowing sound, so as to distinguish the audio from different users, so as to enable each user to control an independent device. When it is confirmed that the sound comes from the input device in front of the corresponding user, the sound is used as the target audio. For example, if user A blows into device A, the blowing sound will be identified as the target audio. On the contrary, if the voice of user A is captured by device B, it will not be regarded as the target audio of device B.

[0033] Specifically, in one embodiment of the present invention, the sound source positioning of each acquired blowing audio set, screening and extracting the target audio includes:

[0034] For each blowing audio set:

[0035] Selecting a blowing audio as a candidate audio, and extracting audio features and time marks of the candidate audio;

[0036] Selecting a blowing audio with the same audio features as a reference audio from other blowing audio sets, and obtaining a corresponding time mark; wherein the time mark is the time when the blowing audio arrives at the corresponding input device;

[0037] Using the time difference positioning method, the sound source position of the reference audio is determined according to the time mark of the candidate audio and the time mark of all reference audios, and it is determined whether the sound source position is within the preset valid area:

[0038] If yes, extract the candidate audio as the target audio of the blowing audio set;

[0039] Otherwise, continue to select another blowing audio as a candidate audio and repeat the confirmation of the target audio.

[0040] Since each blowing audio set includes blowing audios of one or more users within a specific time range, a blowing audio is first selected from the blowing audio set as a candidate audio. Audio features are extracted from the selected candidate audio, and the time mark in the candidate audio is obtained. Among them, the audio features include but are not limited to frequency, amplitude, etc., and the time mark indicates the time when the candidate audio arrives at the corresponding input device. According to the audio features of the candidate audio, the remaining blowing audio sets are searched for sounds with the same audio features, these sounds are used as reference audios, and the time mark of each reference audio is extracted. Using the time difference positioning method, the time difference between the candidate audio and each reference audio is calculated according to the time mark, and the specific location of the sound source can be determined according to these time differences. According to the sound source position, it is determined whether it is within the preset valid area. If so, the candidate audio will be used as the target audio of the blowing audio set. On the contrary, if the sound source position is not within the valid area, another blowing audio is selected from the blowing audio set as a candidate audio, and the aforementioned confirmation process is repeated until all the blowing audios in the blowing audio set are selected.

[0041] In one embodiment of the present invention, the time difference positioning method is used to determine the sound source position of the reference audio according to the time mark of the candidate audio and the time mark of all reference audios, and judge the sound source position, including:

[0042] Calculate the time difference between the candidate audio and each reference audio according to the time identifier of the candidate audio and the time identifiers of all reference audios;

[0043] Determine the location of the sound source emitting the candidate audio based on each time difference and the preset location of the corresponding input device;

[0044] Determine whether the sound source position is within the valid area preset by the input device corresponding to the candidate audio:

[0045] If yes, the blowing audio is the target audio of the reference set;

[0046] Otherwise, the blow audio is a non-target audio of the reference set.

[0047] According to the time identification of the candidate audio and each reference audio, for each reference audio, the difference between its arrival time and the arrival time of the candidate audio is calculated. After all the differences are calculated, the time difference is converted into a distance difference using the speed of sound. Since the position of each input device is known, the sound source position can be determined based on these distance differences by triangulation. After determining the sound source position, the multimedia system will detect whether the sound source position is within the preset effective area of ​​the input device corresponding to the candidate audio, wherein the effective area may be an area within a preset radius around the input device, and only the sound source within this area is considered valid. When the sound source position is within the effective area, the candidate audio is used as the target audio and further detection is terminated. Conversely, when the sound source position is not within the effective area, the candidate audio will be used as a non-target audio and will not trigger the system's response. Through this sound source positioning method, the system's response is consistent with the predetermined user intention, avoiding the situation where user A blows and controls device B.

[0048] S3. Input each extracted target audio into the instruction generation model respectively, extract the number of blowing segments or the duration of the blowing segments from the target audio, and generate a control instruction corresponding to each target audio according to the number of blowing segments or the duration of the blowing segments.

[0049] For each extracted target audio, the target audio is input into the trained instruction generation model, the blowing features are extracted from the audio, and the blowing features are analyzed, and each continuous blowing of the user is regarded as a blowing segment, so as to obtain the total number of blowing segments of the user within a preset time range, or the duration of the blowing segments of the user within the time range. According to the total number of blowing segments or the duration of blowing segments, according to the preset rules, the operation instructions corresponding to the total number of blowing segments or the duration are found. It should be noted that in order to improve the accuracy of recognition, when training the instruction generation model in the present invention, the negative samples input to the model include interference noises such as coughs and sighs, so that the model can more accurately distinguish the sound of blowing. In addition, when the model is trained, targeted training will be carried out according to the intensity of blowing, the duration of blowing, etc. to ensure that the model can recognize blowing sounds of different blowing intensities and durations. It can be understood that the instruction generation model includes but is not limited to HMM, LSTM, CNN, GAN, WaveNet, Tacotron, Transformer, etc.

[0050] S4. Based on the control instruction, control the corresponding output device to perform a preset corresponding operation.

[0051] After the instruction generation model generates a control instruction based on the target audio, the multimedia system parses the control instruction to obtain specific operations, such as play, pause, skip or adjust volume, etc. The parsed instruction is sent to the corresponding output device, and the output device is controlled to operate according to the instruction.

[0052] Specifically, in an embodiment of the present invention, the output device is a speaker, and the controlling the corresponding output device to perform a corresponding operation based on the control instruction includes:

[0053] Parse the control instruction to determine the operation content and the speaker identification that executes the operation content:

[0054] If the operation content is switching, switching to the next audio, and sending the switched audio to the corresponding speaker according to the speaker identifier, so that the speaker plays the corresponding audio within the preset sound zone;

[0055] If the operation content is pause, the corresponding speaker is controlled to stop playing according to the speaker identifier.

[0056] In this embodiment, the output device is a speaker. When the user listens to songs or other audio, and wants to switch or pause, etc., he blows into the microphone corresponding to the speaker, extracts the number of blows through the instruction generation model, and generates operation instructions according to the preset control rules. Among them, the speaker described in the present invention includes but is not limited to headrest speakers, headphones, in-ear headphones, etc., and those skilled in the art can adaptably choose based on actual needs, which is not limited here. Exemplarily, when the speaker is a headphone or in-ear headphones, the multimedia system parses and processes the control instructions generated by the instruction generation model, obtains the specific operation content, and controls the headphones to perform corresponding operations based on the parsed operation content. Exemplarily, the control rule is: within the preset time range, when the number of blows is once, the play or pause operation is realized, when the number of blows is two, the operation of switching to the next music is realized, and when the number of blows is three, the operation of rewinding to the previous one is realized. Therefore, according to the number of blows, the corresponding headphones can be controlled to switch songs, pause, rewind or continue playing. It can be understood that in order to improve the user experience, the speaker can adopt active noise reduction technology to improve the clarity of the in-ear sound. It is understandable that the earphones themselves have the characteristic of sound zone isolation, which can effectively prevent the intrusion of external noise. Furthermore, when the speaker is a headrest speaker, in order to improve the user experience, a headrest speaker can be built into the headrest of each car seat, and the user can enjoy a high-quality audio experience without wearing headphones or other devices. Specifically, since each seat headrest has an independent speaker module, each seat forms an independent audio zone. When in use, each headrest speaker only provides audio for the user of the corresponding seat, and based on directional audio technology, the played audio is concentrated in the user's ear area to reduce audio leakage, so that each user can listen to the audio content of their choice without disturbing others, thereby improving user privacy.

[0057] In another embodiment of the present invention, the output device includes a speaker and a screen, and the controlling the corresponding output device to perform a corresponding operation based on the control instruction includes:

[0058] Parsing the control instruction to determine the operation content and the speaker identifier and screen identifier for executing the operation content;

[0059] Performing corresponding operations on the currently played audio content and the corresponding video content according to the operation content;

[0060] According to the screen identifier and the speaker identifier, the video content after the operation is sent to the corresponding screen, and the audio content after the operation is sent to the corresponding speaker, so as to control the screen to present the video content after the operation, and control the speaker to play the audio content after the operation within a preset sound zone.

[0061] When the output device includes a speaker and a screen, after the instruction generation model generates the corresponding control instructions according to the target audio, the multimedia system parses the generated control instructions to obtain the specific operation content in the instructions. Then, according to the operation content obtained by the analysis, the relevant operations on the audio and video content are performed on the output device, and the playback status of the speaker and the screen are updated synchronously to ensure that the audio and video content remain consistent. For example, if the operation content is "play", the audio and video content is started or continued to be played; if the operation content is "turn the page up", the interface of the previous page is displayed on the screen, and the audio of the previous page is played.

[0062] In an embodiment of the present invention, after controlling the corresponding output device to perform a corresponding operation based on the control instruction, the method further includes:

[0063] Determine whether a successful operation response from the corresponding output device is received within the preset receiving time:

[0064] If yes, record the successful operation information and keep the current control status of the output device unchanged;

[0065] Otherwise, the operation failure information is recorded and a prompt to check the device connection is issued.

[0066] The multimedia system will preset a receiving time range, and determine whether it has received feedback from the output device within the receiving time range. If the system receives a successful response to the operation feedback from the output device, it will record this information and maintain the current control state of the output device unchanged. On the contrary, if the system has not received a successful response to the operation within the receiving time range, it means that the output device may not respond to the operation in time due to a fault or other reasons. At this time, the multimedia device will record detailed information about the failed operation for later fault diagnosis. At the same time, the multimedia device will also issue a prompt to check the device connection to guide the user to detect device failures.

[0067] See also Figure 3 , the multimedia system 100 includes: a data acquisition module 110, an audio extraction module 120, an instruction generation module 130 and a control module 140. The data acquisition module 110 is used to obtain a set of blowing audios of each group of input devices within a preset time range. The audio extraction module 120 is used to locate the sound source of each acquired blowing audio set, filter and extract the target audio. The instruction generation module 130 is used to input each extracted target audio into the instruction generation model, extract the number of blowing segments or the duration of the blowing segments from the target audio, and generate a control instruction corresponding to each target audio according to the number of blowing segments or the duration of the blowing segments. The control module 140 is used to control the corresponding output device to perform a preset corresponding operation based on the control instruction.

[0068] For the specific definition of the multimedia system, please refer to the definition of the multimedia control method above, which will not be repeated here. Each module in the above multimedia system can be implemented in whole or in part by software, hardware and a combination thereof. Each of the above modules can be embedded in or independent of the processor in the computer device in hardware format, or can be stored in the memory of the computer device in software format, so that the processor can call the corresponding operations of each of the above modules.

[0069] It should be noted that, in order to highlight the innovative part of the present invention, the present embodiment does not introduce modules that are not closely related to solving the technical problem proposed by the present invention, but this does not mean that there are no other modules in the present embodiment.

[0070] See also Figure 4 The vehicle terminal 1 may include a memory 12, a processor 13 and a bus, and may also include a computer program stored in the memory 12 and executable on the processor 13, such as a multimedia control program.

[0071] Among them, the memory 12 includes at least one type of readable storage medium, and the readable storage medium includes flash memory, mobile hard disk, multimedia card, card-type memory (for example: SD or DX memory, etc.), magnetic memory, disk, optical disk, etc. The memory 12 can be an internal storage unit of the vehicle-mounted terminal 1 in some embodiments, such as a mobile hard disk of the vehicle-mounted terminal 1. The memory 12 can also be an external storage device of the vehicle-mounted terminal 1 in other embodiments, such as a plug-in mobile hard disk, a smart memory card (Smart Media Card, SMC), a secure digital (Secure Digital, SD) card, a flash card (Flash Card), etc. equipped on the vehicle-mounted terminal 1. Further, the memory 12 can also include both an internal storage unit of the vehicle-mounted terminal 1 and an external storage device. The memory 12 can not only be used to store application software and various types of data installed in the vehicle-mounted terminal 1, such as multimedia control codes, etc., but can also be used to temporarily store data that has been output or is to be output.

[0072] In some embodiments, the processor 13 may be composed of an integrated circuit, for example, a single packaged integrated circuit, or a plurality of packaged integrated circuits with the same or different functions, including one or more central processing units (CPUs), microprocessors, digital processing chips, graphics processors, and combinations of various control chips. The processor 13 is the control core (Control Unit) of the vehicle-mounted terminal 1, and uses various interfaces and lines to connect various components of the entire vehicle-mounted terminal 1, and executes or executes programs or modules (such as multimedia control programs, etc.) stored in the memory 12, and calls data stored in the memory 12 to execute various functions of the vehicle-mounted terminal 1 and process data.

[0073] The processor 13 executes the operating system and various installed applications of the vehicle terminal 1. The processor 13 executes the applications to implement the steps in the above-mentioned multimedia control method.

[0074] Exemplarily, the computer program may be divided into one or more modules, which are stored in the memory 12 and executed by the processor 13 to complete the present application. The one or more modules may be a series of computer program instruction segments capable of completing specific functions, which are used to describe the execution process of the computer program in the vehicle-mounted terminal 1. For example, the computer program may be divided into a data acquisition module 110, an audio extraction module 120, an instruction generation module 130, and a control module 140.

[0075] The above-mentioned integrated unit implemented in the form of a software function module can be stored in a computer-readable storage medium, and the computer-readable storage medium can be non-volatile or volatile. The above-mentioned software function module is stored in a storage medium, including several instructions for enabling a computer device (which can be a personal computer, a computer device, or a network device, etc.) or a processor to perform part of the functions of the multimedia control method described in each embodiment of the present application.

[0076] In summary, the multimedia control method, system, vehicle-mounted terminal and medium disclosed in the present invention acquire and analyze the blowing audio set of each input device within a preset time range, use sound source localization technology to accurately locate and filter out the target audio, input the target audio into the instruction generation model, and generate corresponding control instructions according to the number of blowing segments. No matter what posture the user is in, complex multimedia control can be achieved through simple blowing actions. The problem that the existing method cannot accurately identify the user's operation instructions is improved. This method of the present invention is applicable to multi-posture operations. Whether the user is lying or sitting, he can use this method to control the corresponding multimedia device without using more tiring body interactions, and this method will not generate additional voice interference, will not interfere with other users, and improve the user experience. Furthermore, this method does not need to be specially developed for language and has wide universality. Therefore, the present invention effectively overcomes the various shortcomings of the prior art and has a high industrial utilization value.

[0077] The above embodiments are merely illustrative of the principles and effects of the present invention, and are not intended to limit the present invention. Anyone familiar with the art may modify or alter the above embodiments without departing from the spirit and scope of the present invention. Therefore, all equivalent modifications or alterations made by a person of ordinary skill in the art without departing from the spirit and technical concept disclosed by the present invention shall still be covered by the claims of the present invention.

Claims

1. A multimedia control method, characterized in that: Applied to a multimedia system, the multimedia system corresponds to at least two groups of multimedia devices, each group of multimedia devices includes an input device and an output device, the method includes: Get the blowing audio collection of each group of input devices within the preset time range; Perform sound source localization on each acquired blowing audio set, filter and extract target audio; Input each extracted target audio into the instruction generation model respectively, extract the number of blowing segments or the duration of the blowing segments from the target audio, and generate a control instruction corresponding to each target audio according to the number of blowing segments or the duration of the blowing segments; Based on the control instruction, the corresponding output device is controlled to perform a preset corresponding operation.

2. The multimedia control method according to claim 1, characterized in that: The step of obtaining a blowing audio set of each group of input devices within a preset time range includes: Get the initial blowing audio set of each group of input devices within a preset time range; Filtering each blowing audio in the initial blowing audio set to remove noise in the blowing audio, and using the remaining blowing audio as the blowing audio after noise filtering; Echo cancellation is performed on each blowing audio after noise filtering to obtain a blowing audio set.

3. The multimedia control method according to claim 1, characterized in that: The method of locating the sound source of each acquired blowing audio set, screening and extracting the target audio includes: For each blowing audio set: Selecting a blowing audio as a candidate audio, and extracting audio features and time marks of the candidate audio; Selecting a blowing audio with the same audio features as a reference audio from other blowing audio sets, and obtaining a corresponding time mark; wherein the time mark is the time when the blowing audio arrives at the corresponding input device; Using the time difference positioning method, the sound source position of the reference audio is determined according to the time mark of the candidate audio and the time mark of all the reference audios, and it is determined whether the sound source position is within the preset valid area: If yes, extract the candidate audio as the target audio of the blowing audio set; Otherwise, continue to select another blowing audio as a candidate audio and repeat the confirmation of the target audio.

4. The multimedia control method according to claim 2, characterized in that: The method of using the time difference positioning method to determine the sound source position of the reference audio according to the time mark of the candidate audio and the time marks of all the reference audios, and to judge whether the sound source position is within a preset valid area, includes: Calculate the time difference between the candidate audio and each reference audio according to the time identifier of the candidate audio and the time identifiers of all reference audios; Determine the location of the sound source emitting the candidate audio based on each time difference and the preset location of the corresponding input device; Determine whether the sound source position is within the valid area preset by the input device corresponding to the candidate audio: If yes, the blowing audio is the target audio of the reference set; Otherwise, the blow audio is a non-target audio of the reference set.

5. The multimedia control method according to claim 1, characterized in that: The output device is a speaker, and based on the control instruction, controlling the corresponding output device to perform a corresponding operation includes: Parse the control instruction to determine the operation content and the speaker identification that executes the operation content: If the operation content is switching, switching to the next audio, and sending the switched audio to the corresponding speaker according to the speaker identifier, so that the speaker plays the corresponding audio within the preset sound zone; If the operation content is pause, the corresponding speaker is controlled to stop playing according to the speaker identifier.

6. The multimedia control method according to claim 1, characterized in that: The output device includes a speaker and a screen, and the controlling the corresponding output device to perform a corresponding operation based on the control instruction includes: Parsing the control instruction to determine the operation content and the speaker identifier and screen identifier for executing the operation content; Performing corresponding operations on the currently played audio content and the corresponding video content according to the operation content; According to the screen identifier and the speaker identifier, the video content after the operation is sent to the corresponding screen, and the audio content after the operation is sent to the corresponding speaker, so as to control the screen to present the video content after the operation, and control the speaker to play the audio content after the operation within a preset sound zone.

7. The multimedia control method according to claim 1, characterized in that: After the corresponding output device is controlled to perform a corresponding operation based on the control instruction, the method further includes: Determine whether a successful operation response from the corresponding output device is received within the preset receiving time: If yes, record the successful operation information and keep the current control status of the output device unchanged; Otherwise, the operation failure information is recorded and a prompt to check the device connection is issued.

8. A multimedia system, characterized in that: The multimedia system corresponds to at least two groups of multimedia devices, each group of multimedia devices includes an input device and an output device, and the system includes: A data acquisition module, used to acquire a set of blowing audios of each group of input devices within a preset time range; An audio extraction module is used to locate the sound source of each acquired blowing audio set, filter and extract the target audio; An instruction generation module, used to input each extracted target audio into the instruction generation model, extract the number of blowing segments or the duration of the blowing segments from the target audio, and generate a control instruction corresponding to each target audio according to the number of blowing segments or the duration of the blowing segments; The control module is used to control the corresponding output device to perform a preset corresponding operation based on the control instruction.

9. A vehicle-mounted terminal, characterized in that: The vehicle-mounted terminal comprises: one or more processors; A storage device for storing one or more programs, when the one or more programs are executed by the one or more processors, enables the vehicle terminal to implement the multimedia control method as described in any one of claims 1 to 7.

10. A computer-readable storage medium, characterized in that: A computer program is stored thereon, and when the computer program is executed by a processor of a computer, the computer is caused to execute the multimedia control method according to any one of claims 1 to 7.

Images