Multimedia file playing method and device and electronic equipment
By obtaining and analyzing voice data in the cabin in the vehicle terminal and automatically adjusting the playback parameters of the multimedia file, the cumbersome operation of users manually switching the accompaniment mode and the original singing mode is solved, and a more convenient user experience is achieved.
Patent Information
- Application Number
- CN202310518745.6
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2023-05-09
- Publication Date
- 2024-11-12
AI Technical Summary
When the on-board terminal plays multimedia files, users need to manually switch the accompaniment mode and the original singing mode, which is cumbersome.
By acquiring the voice data in the cabin, calculate the similarity between the actual dubbing data and the voice data in the cabin, decide whether to obtain the accompaniment parameters of the multimedia file according to the similarity threshold, and automatically adjust the volume and lighting parameters to achieve mode switching.
It simplifies user operations and automatically determines the playback mode based on the user's voice data in the cabin, reduces the need for manual switching and improves the user experience.
Smart Images

Figure CN118945413A_ABST
Abstract
Description
Technical Field
[0001] The present disclosure relates to the field of multimedia technology, and in particular to a method, device and electronic device for playing multimedia files. Background Art
[0002] Currently, when users play multimedia files on a car terminal, they can use two playback modes, one is the accompaniment mode and the other is the original singing mode; among them, the accompaniment mode is a mode in which only the accompaniment music of the multimedia file is played, while the original singing mode is a mode in which both the vocals and the accompaniment music of the multimedia file are played.
[0003] When the vehicle terminal plays multimedia files, if the user needs to switch between the accompaniment mode and the original singing mode, it is usually achieved through manual operation, that is, by clicking the switch control on the vehicle terminal to switch between the accompaniment mode and the original singing mode, which is cumbersome. Summary of the invention
[0004] In order to solve the above technical problems, the present disclosure provides a method, device and electronic device for playing multimedia files.
[0005] In order to achieve the above objectives, the present invention adopts the following technical solutions:
[0006] In a first aspect, the present disclosure provides a method for playing a multimedia file, comprising: when playing the multimedia file, obtaining cabin voice data in a cabin; wherein the multimedia file includes actual dubbing data and actual background sound data; performing similarity calculation on the actual dubbing data and the cabin voice data to obtain the similarity between the cabin voice data and the actual dubbing data; when the similarity is greater than or equal to a similarity threshold, obtaining accompaniment parameters of the multimedia file; wherein the accompaniment parameters include the volume value of the actual dubbing data, and the volume value of the actual background sound data; and playing the multimedia file according to the accompaniment parameters.
[0007] In some feasible examples, a similarity calculation is performed on the actual dubbing data and the in-cabin voice data to obtain the similarity between the in-cabin voice data and the actual dubbing data, including: performing a speech-to-text operation on the in-cabin voice data to obtain in-cabin text data corresponding to the in-cabin voice data; when the in-cabin text data contains characters, respectively extracting text features of the actual dubbing data and the actual text data corresponding to the in-cabin text data to obtain a first text feature vector corresponding to the in-cabin text data and a second text feature vector corresponding to the actual text data; calculating a first relationship between the first text feature vector and the second text feature vector, and using the first relationship as the similarity between the actual dubbing data and the in-cabin voice data; wherein the first relationship includes any one of cosine similarity and a preset distance, and the preset distance includes any one of Euclidean distance and Manhattan distance.
[0008] In some feasible examples, the multimedia file playback method provided by the embodiments of the present disclosure also includes: when the in-cabin text data does not contain characters, respectively extracting audio features of the actual dubbing data and the in-cabin voice data to obtain a first audio feature vector corresponding to the in-cabin voice data and a second audio feature vector corresponding to the actual dubbing data; calculating a second relationship between the first audio feature vector and the second audio feature vector, and using the second relationship as the similarity between the actual dubbing data and the in-cabin voice data; wherein the second relationship includes any one of cosine similarity and a preset distance, and the preset distance includes any one of Euclidean distance and Manhattan distance.
[0009] In some feasible examples, when playing a multimedia file, obtaining the cabin voice data in the cabin includes: when playing the multimedia file, performing sound source separation on the multimedia file to obtain actual dubbing data and actual background sound data corresponding to the multimedia file.
[0010] In some feasible examples, when the similarity is greater than or equal to a similarity threshold, the accompaniment parameters of the multimedia file are obtained, including: when the similarity is greater than or equal to the similarity threshold, the file type of the multimedia file is obtained; based on the file type, a preset relationship table is queried to obtain the accompaniment parameters corresponding to the file type; wherein different file types correspond to different accompaniment parameters.
[0011] In some feasible examples, after playing the multimedia file according to the accompaniment parameters, the multimedia file playing method provided by the embodiment of the present disclosure also includes: when the similarity is less than the similarity threshold, playing the multimedia file according to preset parameters; wherein the preset parameters are the playing parameters set when playing the multimedia file.
[0012] In some feasible examples, the cabin includes at least one functional area, a functional area includes at least one audio playback device and at least one lighting device, and the accompaniment parameters also include lighting parameters; playing the multimedia file according to the accompaniment parameters includes: performing a position analysis on the voice data in the cabin to obtain the functional area that emits the voice data in the cabin; playing the multimedia file according to the accompaniment parameters in the functional area.
[0013] In a second aspect, the present disclosure provides a multimedia file playback device, comprising: a processing unit, for controlling an acquisition unit to acquire in-cabin voice data in a cabin when playing the multimedia file; wherein the multimedia file includes actual dubbing data and actual background sound data; the processing unit is further used to calculate the similarity between the actual dubbing data and the in-cabin voice data acquired by the acquisition unit, to obtain the similarity between the in-cabin voice data and the actual dubbing data; the processing unit is further used to control the acquisition unit to acquire accompaniment parameters of the multimedia file when the similarity is greater than or equal to a similarity threshold; wherein the accompaniment parameters include the volume value of the actual dubbing data and the volume value of the actual background sound data; the processing unit is further used to play the multimedia file according to the accompaniment parameters acquired by the acquisition unit.
[0014] In some feasible examples, the processing unit is specifically used to perform a speech-to-text operation on the in-cabin voice data acquired by the acquisition unit to obtain in-cabin text data corresponding to the in-cabin voice data; the processing unit is specifically used to extract text features of the actual dubbing data and the actual text data corresponding to the in-cabin text data acquired by the acquisition unit, respectively, when the in-cabin text data contains characters, to obtain a first text feature vector corresponding to the in-cabin text data and a second text feature vector corresponding to the actual text data; the processing unit is specifically used to calculate a first relationship between the first text feature vector and the second text feature vector, and use the first relationship as the similarity between the actual dubbing data and the in-cabin voice data; wherein the first relationship includes any one of cosine similarity and a preset distance, and the preset distance includes any one of Euclidean distance and Manhattan distance.
[0015] In some feasible examples, the processing unit is further used to extract the audio features of the actual dubbing data and the in-cabin voice data acquired by the acquisition unit respectively when the in-cabin text data does not contain characters, and obtain a first audio feature vector corresponding to the in-cabin voice data and a second audio feature vector corresponding to the actual dubbing data; the processing unit is further used to calculate the second relationship between the first audio feature vector and the second audio feature vector, and use the second relationship as the similarity between the actual dubbing data and the in-cabin voice data; wherein the second relationship includes any one of cosine similarity and a preset distance, and the preset distance includes any one of Euclidean distance and Manhattan distance.
[0016] In some feasible examples, the processing unit is specifically used to separate the sound source of the multimedia file when playing the multimedia file, so as to obtain the actual dubbing data and the actual background sound data corresponding to the multimedia file.
[0017] In some feasible examples, the processing unit is specifically used to control the acquisition unit to obtain the file type of the multimedia file when the similarity is greater than or equal to the similarity threshold; the processing unit is specifically used to query the preset relationship table based on the file type obtained by the acquisition unit to obtain the accompaniment parameters corresponding to the file type; wherein different file types correspond to different accompaniment parameters.
[0018] In some feasible examples, the processing unit is further used to play the multimedia file according to preset parameters when the similarity is less than a similarity threshold; wherein the preset parameters are playback parameters set when playing the multimedia file.
[0019] In some feasible examples, the cabin includes at least one functional area, a functional area includes at least one audio playback device and at least one lighting device, and the accompaniment parameters also include lighting parameters; the processing unit is specifically used to perform position analysis on the cabin voice data acquired by the acquisition unit to obtain the functional area that emits the cabin voice data; the processing unit is specifically used to play multimedia files in the functional area according to the accompaniment parameters.
[0020] In a third aspect, the present disclosure provides an electronic device, comprising: a memory and a processor, the memory being used to store a computer program; the processor being used to enable the electronic device to implement the multimedia file playback method provided in the first aspect when executing the computer program.
[0021] In a fourth aspect, the present disclosure provides a computer-readable storage medium, comprising: a computer program is stored on the computer-readable storage medium, and when the computer program is executed by a computing device, the computing device implements the multimedia file playback method provided in the first aspect above.
[0022] In a fifth aspect, the present disclosure provides a vehicle, comprising a device for playing any multimedia file provided in the second aspect.
[0023] In the present disclosure, the name of the multimedia file playback device does not limit the device or functional module itself. In actual implementation, these devices or functional modules may appear with other names. As long as the functions of each device or functional module are similar to those of the present disclosure, they fall within the scope of the claims of the present disclosure and their equivalent technologies.
[0024] These and other aspects of the present disclosure will become more apparent from the following description.
[0025] Compared with the prior art, the technical solution provided by the present invention has the following advantages:
[0026] When the multimedia file playback method provided by the present disclosure is applied to a car, the car obtains the cabin voice data in the cabin when playing the multimedia file. Afterwards, the car calculates the similarity between the actual dubbing data and the cabin voice data to obtain the similarity between the cabin voice data and the actual dubbing data; when the similarity is greater than or equal to the similarity threshold, the accompaniment parameters of the multimedia file are obtained; and the multimedia file is played according to the accompaniment parameters. In this way, the user does not need to manually click the switching control on the vehicle terminal to switch the accompaniment mode and the original singing mode, but can judge whether to use the accompaniment parameters to play the multimedia file according to the cabin voice data emitted by the user in the cabin of the car, which simplifies the user's operation and solves the problem that in the prior art, when the vehicle terminal plays multimedia files, if the user needs to switch the mode between the accompaniment mode and the original singing mode, it is usually achieved by manual operation, that is, by clicking the switching control on the vehicle terminal to switch the accompaniment mode and the original singing mode, which is cumbersome. BRIEF DESCRIPTION OF THE DRAWINGS
[0027] The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments consistent with the present disclosure and, together with the description, serve to explain the principles of the present disclosure.
[0028] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, for ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative labor.
[0029] Figure 1 One of the flowcharts of the method for playing multimedia files provided in the embodiment of the present disclosure;
[0030] Figure 2 A second flowchart of a method for playing multimedia files provided in an embodiment of the present disclosure;
[0031] Figure 3 A third flowchart of a method for playing multimedia files provided in an embodiment of the present disclosure;
[0032] Figure 4 A fourth flowchart of a method for playing multimedia files provided in an embodiment of the present disclosure;
[0033] Figure 5 A fifth flowchart of a method for playing multimedia files provided in an embodiment of the present disclosure;
[0034] Figure 6 A sixth flowchart of a method for playing multimedia files provided in an embodiment of the present disclosure;
[0035] Figure 7 A schematic diagram of the structure of a multimedia file playback device provided in an embodiment of the present disclosure;
[0036] Figure 8 A schematic diagram of the structure of a vehicle-mounted terminal provided in an embodiment of the present disclosure;
[0037] Fig. 9 A schematic diagram of the structure of a computer program product provided in an embodiment of the present disclosure. DETAILED DESCRIPTION
[0038] In order to more clearly understand the above-mentioned objectives, features and advantages of the present disclosure, the scheme of the present disclosure will be further described below. It should be noted that the embodiments of the present disclosure and the features in the embodiments can be combined with each other without conflict.
[0039] In the following description, many specific details are set forth to facilitate a full understanding of the present disclosure, but the present disclosure may also be implemented in other ways different from those described herein; it is obvious that the embodiments in the specification are only part of the embodiments of the present disclosure, rather than all of the embodiments.
[0040] It should be noted that, in this article, relational terms such as "first" and "second" are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Moreover, the terms "include", "comprise" or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or device. In the absence of further restrictions, the elements defined by the sentence "comprise a ..." do not exclude the existence of other identical elements in the process, method, article or device including the elements.
[0041] The sound source separation in the embodiment of the present disclosure refers to separating different musical instruments and vocals in a multimedia file (such as music), which can easily achieve the removal of vocals from music and the extraction of music accompaniment.
[0042] The execution subject of the multimedia file playback method in the embodiment of the present application is the multimedia file playback device, which can be set in a server or other device, and this embodiment does not limit this. For example, when the multimedia file playback device is set in a car, the execution subject of the multimedia file playback method provided in the embodiment of the present disclosure is the car's processor.
[0043] The to-be-recognized voice data and historical voice data involved in the present disclosure may be data authorized by the user or fully authorized by all parties.
[0044] Exemplarily, the multimedia file playback method provided by the embodiment of the present disclosure is introduced by taking the execution subject of the multimedia file playback method provided by the embodiment of the present disclosure as an example of a car.
[0045] like Figure 1 As shown, the multimedia file playback method provided by the embodiment of the present disclosure includes the following steps S11-S14:
[0046] S11, when playing a multimedia file, obtaining cabin voice data in the cabin, wherein the multimedia file includes actual dubbing data and actual background sound data.
[0047] In some examples, the user can play multimedia files (the corresponding playback parameters are preset parameters) through the vehicle terminal while driving the car, such as music videos (MV), songs, or movies. When the user hums along with the multimedia files played by the vehicle terminal, it means that the user has the need for accompaniment. To this end, the multimedia file playback method provided by the embodiment of the present disclosure, when the vehicle terminal plays the multimedia file, by obtaining the cabin voice data in the cabin, it can determine whether the user is currently humming along with the multimedia file played by the vehicle terminal based on the cabin voice data. For example: when playing the multimedia file, the multimedia file is separated from the multimedia file by sound source, and the actual dubbing data and the actual background sound data corresponding to the multimedia file are obtained. Afterwards, the similarity between the actual dubbing data and the cabin voice data is calculated to obtain the similarity between the cabin voice data and the actual dubbing data. When the similarity is greater than or equal to the similarity threshold, the accompaniment parameters of the multimedia file are obtained, and the multimedia file is played according to the accompaniment parameters. In this way, the accompaniment function of the vehicle terminal is realized. Afterwards, when the user no longer hums along with the multimedia file played by the vehicle terminal, it indicates that the user has a need for singing along. At this time, since the user no longer hums along with the multimedia file played by the vehicle terminal, the similarity between the actual dubbing data and the in-cabin voice data is less than the similarity threshold. In this case, the multimedia file can continue to be played according to the preset parameters when the user plays the multimedia file, ensuring the user experience.
[0048] In some examples, when the multimedia file is a song, the multimedia file only contains the audio data of the song. At this time, the audio data can be separated from the sound source to obtain the actual dubbing data and the actual background sound data corresponding to the multimedia file. Afterwards, the vehicle terminal performs frame alignment on the actual dubbing data and the actual background sound data to ensure the user experience.
[0049] In some examples, when the multimedia file is an MV or a movie, the multimedia file contains audio data and video data. At this time, when performing sound source separation on the multimedia file, the audio data in the multimedia file is separated to obtain the actual dubbing data and actual background sound data corresponding to the multimedia file. Afterwards, the vehicle terminal performs frame alignment on the actual dubbing data, the actual background sound data and the video data to ensure the user experience.
[0050] In some examples, the cabin of the car contains at least one audio collection device, such as a microphone, and the vehicle-mounted terminal obtains the voice data in the cabin through the microphone.
[0051] S12, performing similarity calculation on the actual dubbing data and the in-cabin voice data to obtain the similarity between the in-cabin voice data and the actual dubbing data.
[0052] In some examples, due to different accents of users, when different users hum along with the same multimedia file played by the vehicle-mounted terminal, the corresponding in-cabin voice data are different. In order to ensure that the function of accompaniment can be provided to different users, the multimedia file playback method provided in the embodiment of the present disclosure performs a voice-to-text operation on the in-cabin voice data to obtain the in-cabin text data corresponding to the in-cabin voice data. Afterwards, it is determined whether the in-cabin text data contains characters, such as: when it is determined that the in-cabin text data contains characters, the text features of the actual dubbing data and the actual text data corresponding to the in-cabin text data can be extracted respectively, and the first text feature vector corresponding to the in-cabin text data and the second text feature vector corresponding to the actual text data are obtained. Afterwards, the first relationship between the first text feature vector and the second text feature vector is calculated, and the first relationship is used as the similarity between the actual dubbing data and the in-cabin voice data.
[0053] Alternatively, when it is determined that the in-cabin text data does not contain characters, it means that the system cannot recognize the characters contained in the in-cabin voice data. In order to ensure the user experience, the multimedia file playback method provided in the embodiment of the present disclosure obtains the first audio feature vector corresponding to the in-cabin voice data and the second audio feature vector corresponding to the actual dubbing data by respectively extracting the audio features of the actual dubbing data and the in-cabin voice data. Afterwards, the second relationship between the first audio feature vector and the second audio feature vector is calculated, and the second relationship is used as the similarity between the actual dubbing data and the in-cabin voice data. In this way, the vehicle-mounted terminal can provide the function of accompaniment for users with different accents to ensure the user experience.
[0054] In some examples, the second relationship includes any one of cosine similarity and a preset distance, and the preset distance includes any one of Euclidean distance and Manhattan distance.
[0055] S13, when the similarity is greater than or equal to the similarity threshold, obtaining the accompaniment parameters of the multimedia file, wherein the accompaniment parameters include the volume value of the actual dubbing data and the volume value of the actual background sound data.
[0056] In some examples, in order to enhance the user experience, the multimedia file playback method provided in the embodiments of the present disclosure configures different accompaniment parameters for different multimedia files. For example, when the multimedia file is an MV or a song, the accompaniment parameters for the media file can be determined according to the file type of the song or MV (such as classical music, pop music, etc.). Or, when the multimedia file is a movie, the accompaniment parameters for the media file can be determined according to the file type of the movie (such as action movie, fantasy movie, comedy movie, etc.).
[0057] In some examples, a preset relationship table of accompaniment parameters corresponding to different file types is pre-stored in the memory of the car. In this way, when the vehicle-mounted terminal needs to obtain the accompaniment parameters of a multimedia file, it can first determine the file type of the multimedia file. Afterwards, the vehicle-mounted terminal can obtain the accompaniment parameters of the multimedia file by querying the accompaniment parameters corresponding to the file type in the preset relationship table stored in the memory of the car.
[0058] S14, playing the multimedia file according to the accompaniment parameters.
[0059] In some examples, when the vehicle-mounted terminal plays a multimedia file according to the accompaniment parameters, the volume value of the actual dubbing data can be adjusted from the current volume value of the playback to the volume value of the actual dubbing data in the accompaniment parameters, and the volume value of the actual background sound data can be adjusted from the current volume value of the playback to the volume value of the actual background sound data in the accompaniment parameters. Among them, the volume value of the actual dubbing data currently played is greater than the volume value of the actual dubbing data in the accompaniment parameters, and the volume value of the actual background sound data previously played is less than the volume value of the actual background sound data in the accompaniment parameters, so that the volume value of the actual dubbing data can be ensured to be less than the user's voice, so as to achieve the virtualization of the original singing and ensure the effect of the accompaniment.
[0060] In some examples, the cabin includes at least one functional area, and one functional area includes at least one audio playback device (such as a speaker) and at least one lighting device (such as an atmosphere light), and the accompaniment parameters also include lighting parameters. In the multimedia file playback method provided in the embodiment of the present disclosure, in order to improve the user experience, when the vehicle-mounted terminal plays the multimedia file according to the accompaniment parameters, the multimedia file can be played only for the functional area that sends the cabin voice data according to the accompaniment parameters. For example: at least one audio acquisition device (such as a microphone) and a seat are provided in each functional area of the cabin of the car, and a pressure acquisition device (such as a pressure sensor) is provided on the seat. In this way, when the pressure value collected by the pressure sensor is greater than the pressure threshold, it means that there is someone in the functional area. Afterwards, the audio data collected by the microphone of the functional area with people is summarized to obtain the cabin voice data, such as: when the cabin of the car contains 3 areas, namely the driving area, the co-pilot area and the rear area, if there are people in both the driving area and the rear area, the microphone in the driving area will collect the audio data in the driving area, and the microphone in the rear area will collect the audio data in the rear area. Afterwards, when it is determined that the similarity between the audio data in the driving area and the actual dubbing data is greater than the similarity threshold, the vehicle-mounted control controls the speakers, atmosphere lights, etc. in the driving area to play the multimedia file according to the accompaniment parameters (such as turning on the atmosphere lights, adjusting the volume value of the actual dubbing data played in the speakers from the current playback volume value to the volume value of the actual dubbing data in the accompaniment parameters, and adjusting the volume value of the actual background sound data played in the speakers from the current playback volume value to the volume value of the actual background sound data in the accompaniment parameters); for the co-pilot area and the rear row area, the vehicle-mounted terminal controls the speakers, atmosphere lights, etc. in the co-pilot area and the rear row area to play the multimedia file according to the preset parameters (such as turning on and off the atmosphere lights, adjusting the volume value of the actual dubbing data played in the speakers from the current playback volume value to the volume value of the actual dubbing data in the preset parameters, and adjusting the volume value of the actual background sound data played in the speakers from the current playback volume value to the volume value of the actual background sound data in the preset parameters). Alternatively, when it is determined that the similarity between the audio data in the driving area and the actual dubbing data is greater than the similarity threshold, and the similarity between the audio data in the rear area and the actual dubbing data is greater than the similarity threshold, the vehicle-mounted control controls the speakers, atmosphere lights, etc. in both the driving area and the rear area to play the multimedia file according to the accompaniment parameters; for the co-pilot area, the vehicle-mounted terminal controls the speakers, atmosphere lights, etc. in the co-pilot area to play the multimedia file according to preset parameters.
[0061] In some examples, an audio acquisition device (such as a microphone) and at least one image acquisition device (such as a camera) are provided in the cabin of the car. When the similarity between the in-cabin voice data collected by the microphone and the actual dubbing data is greater than the similarity threshold, at this time, by performing target detection on the in-cabin image of the camera in the time period including the collection time of the in-cabin voice data collected by the microphone, it is determined whether each user has spoken in the time period (such as: determining the image corresponding to the mouth of each user through target detection, and determining whether the user has spoken in the time period by comparing the area changes of the image at different times (the area of the image corresponding to the mouth when the user is not speaking is smaller than the area of the image corresponding to the mouth when the user is speaking)). Afterwards, the in-vehicle terminal controls the speakers, atmosphere lights, etc. in the functional area where the user who has spoken in the time period is located to play the multimedia file according to the accompaniment parameters.
[0062] In some examples, the absolute value of the difference between the start time of the time period and the collection time of the voice data in the cabin collected by the microphone is less than a first threshold, and the absolute value of the difference between the end time of the time period and the collection time of the voice data in the cabin collected by the microphone is less than a second threshold. Exemplarily, both the first threshold and the second threshold are less than or equal to 5s.
[0063] In some possible implementation examples, combined with Figure 1 ,like Figure 2 As shown, the above S12 can be specifically implemented through the following S120-S122.
[0064] S120: Perform a speech-to-text operation on the in-cabin voice data to obtain in-cabin text data corresponding to the in-cabin voice data.
[0065] S121. When the in-cabin text data contains characters, extract text features of the actual dubbing data and the actual text data corresponding to the in-cabin text data to obtain a first text feature vector corresponding to the in-cabin text data and a second text feature vector corresponding to the actual text data.
[0066] S122, calculating a first relationship between the first text feature vector and the second text feature vector, and using the first relationship as the similarity between the actual dubbing data and the in-cabin voice data, wherein the first relationship includes any one of cosine similarity and a preset distance, and the preset distance includes any one of Euclidean distance and Manhattan distance.
[0067] In some possible implementation examples, combined with Figure 1 ,like Figure 2 As shown, the above S12 can be specifically implemented through the following S123 and S124.
[0068] S123. When the in-cabin text data does not contain any characters, extract audio features of the actual dubbing data and the in-cabin voice data respectively to obtain a first audio feature vector corresponding to the in-cabin voice data and a second audio feature vector corresponding to the actual dubbing data.
[0069] S124. Calculate a second relationship between the first audio feature vector and the second audio feature vector, and use the second relationship as the similarity between the actual dubbing data and the in-cabin voice data; wherein the second relationship includes any one of cosine similarity and a preset distance, and the preset distance includes any one of Euclidean distance and Manhattan distance.
[0070] In some possible implementation examples, combined with Figure 1 ,like Figure 3 As shown, the above S11 can be specifically implemented through the following S110.
[0071] S110: When playing a multimedia file, perform sound source separation on the multimedia file to obtain actual dubbing data and actual background sound data corresponding to the multimedia file.
[0072] In some possible implementation examples, combined with Figure 1 ,like Figure 4 As shown, the above S13 can be specifically implemented through the following S130 and S131.
[0073] S130: When the similarity is greater than or equal to the similarity threshold, obtain the file type of the multimedia file.
[0074] S131, querying a preset relationship table based on the file type to obtain accompaniment parameters corresponding to the file type, wherein different file types correspond to different accompaniment parameters.
[0075] In some possible implementation examples, combined with Figure 1 ,like Figure 5 As shown, the multimedia file playing method provided by the embodiment of the present disclosure also includes S15.
[0076] S15. When the similarity is less than the similarity threshold, play the multimedia file according to preset parameters; wherein the preset parameters are playing parameters set when playing the multimedia file.
[0077] In some feasible examples, the cabin includes at least one functional area, one functional area includes at least one audio playback device and at least one lighting device, and the accompaniment parameters also include lighting parameters; Figure 1 ,like Figure 6 As shown, the above S14 can be specifically implemented through the following S140 and S141.
[0078] S140: Perform position analysis on the in-cabin voice data to obtain a functional area that emits the in-cabin voice data.
[0079] S141, playing the multimedia file in the function area according to the accompaniment parameters.
[0080] The above mainly introduces the solution provided by the embodiment of the present disclosure from the perspective of the method. In order to achieve the above functions, it includes hardware structures and / or software modules corresponding to the execution of each function. Those skilled in the art should easily realize that, in combination with the units and algorithm steps of each example described in the embodiments disclosed herein, the present disclosure can be implemented in the form of hardware or a combination of hardware and computer software. Whether a function is executed in the form of hardware or computer software driving hardware depends on the specific application and design constraints of the technical solution. Professional and technical personnel can use different methods to implement the described functions for each specific application, but such implementation should not be considered to exceed the scope of the present disclosure.
[0081] The embodiments of the present disclosure can divide the function modules of the multimedia file playback device according to the above method examples. For example, each function module can be divided according to each function, or two or more functions can be integrated into one processing module. The above integrated modules can be implemented in the form of hardware or in the form of software function modules. It should be noted that the division of modules in the embodiments of the present disclosure is schematic and is only a logical function division. There may be other division methods in actual implementation.
[0082] like Figure 7 FIG. 1 is a schematic diagram of a multimedia file playback device 10 provided in an embodiment of the present disclosure, comprising a processing unit 101 and an acquisition unit 102 .
[0083] The processing unit 101 is used to control the acquisition unit 102 to acquire the in-cabin voice data in the cabin when playing the multimedia file; wherein the multimedia file includes actual dubbing data and actual background sound data; the processing unit 101 is also used to calculate the similarity between the actual dubbing data and the in-cabin voice data acquired by the acquisition unit 102 to obtain the similarity between the in-cabin voice data and the actual dubbing data; the processing unit 101 is also used to control the acquisition unit 102 to acquire the accompaniment parameters of the multimedia file when the similarity is greater than or equal to the similarity threshold; wherein the accompaniment parameters include the volume value of the actual dubbing data and the volume value of the actual background sound data; the processing unit 101 is also used to play the multimedia file according to the accompaniment parameters acquired by the acquisition unit 102.
[0084] In some feasible examples, the processing unit 101 is specifically used to perform a speech-to-text operation on the in-cabin voice data acquired by the acquisition unit 102 to obtain in-cabin text data corresponding to the in-cabin voice data; the processing unit 101 is specifically used to extract text features of the actual dubbing data and the actual text data corresponding to the in-cabin text data acquired by the acquisition unit 102, respectively, when the in-cabin text data contains characters, to obtain a first text feature vector corresponding to the in-cabin text data and a second text feature vector corresponding to the actual text data; the processing unit 101 is specifically used to calculate a first relationship between the first text feature vector and the second text feature vector, and use the first relationship as the similarity between the actual dubbing data and the in-cabin voice data; wherein the first relationship includes any one of cosine similarity and a preset distance, and the preset distance includes any one of Euclidean distance and Manhattan distance.
[0085] In some feasible examples, the processing unit 101 is also used to extract the audio features of the actual dubbing data and the in-cabin voice data acquired by the acquisition unit 102 respectively when the in-cabin text data does not contain characters, and obtain a first audio feature vector corresponding to the in-cabin voice data and a second audio feature vector corresponding to the actual dubbing data; the processing unit 101 is also used to calculate the second relationship between the first audio feature vector and the second audio feature vector, and use the second relationship as the similarity between the actual dubbing data and the in-cabin voice data; wherein the second relationship includes any one of cosine similarity and a preset distance, and the preset distance includes any one of Euclidean distance and Manhattan distance.
[0086] In some feasible examples, the processing unit 101 is specifically used to perform sound source separation on the multimedia file when playing the multimedia file, so as to obtain actual dubbing data and actual background sound data corresponding to the multimedia file.
[0087] In some feasible examples, the processing unit 101 is specifically used to control the acquisition unit 102 to obtain the file type of the multimedia file when the similarity is greater than or equal to the similarity threshold; the processing unit 101 is specifically used to query the preset relationship table based on the file type obtained by the acquisition unit 102 to obtain the accompaniment parameters corresponding to the file type; wherein different file types correspond to different accompaniment parameters.
[0088] In some feasible examples, the processing unit 101 is further configured to play the multimedia file according to preset parameters when the similarity is less than a similarity threshold; wherein the preset parameters are playback parameters set when playing the multimedia file.
[0089] In some feasible examples, the cabin includes at least one functional area, a functional area includes at least one audio playback device and at least one lighting device, and the accompaniment parameters also include lighting parameters; the processing unit 101 is specifically used to perform position analysis on the cabin voice data acquired by the acquisition unit 102 to obtain the functional area that emits the cabin voice data; the processing unit 101 is specifically used to play multimedia files according to the accompaniment parameters in the functional area.
[0090] Among them, all relevant contents of each step involved in the above method embodiment can be referred to the functional description of the corresponding functional module, and its role will not be repeated here.
[0091] Of course, the multimedia file playback device 10 provided in the embodiment of the present disclosure includes but is not limited to the above modules, for example, the multimedia file playback device 10 may also include a storage unit 103. The storage unit 103 may be used to store the program code of the multimedia file playback device 10, and may also be used to store data generated by the multimedia file playback device 10 during operation, such as data in a write request, etc.
[0092] Figure 8 A schematic diagram of the structure of a vehicle-mounted terminal provided in an embodiment of the present disclosure is shown in FIG. Figure 8 As shown, the vehicle-mounted terminal may include: at least one processor 51 , a memory 52 , a communication interface 53 and a communication bus 54 .
[0093] Combine the following Figure 8 The following is a detailed introduction to the various components of the vehicle terminal:
[0094] The processor 51 is the control center of the electronic device 10, and may be a processor or a general term for multiple processing elements. For example, the processor 51 is a central processing unit (CPU), or an application specific integrated circuit (ASIC), or one or more integrated circuits configured to implement the embodiments of the present disclosure, such as one or more DSPs, or one or more field programmable gate arrays (FPGAs).
[0095] In a specific implementation, as an embodiment, the processor 51 may include one or more CPUs, such as Figure 8 Also, as an embodiment, the electronic device may include multiple processors, such as Figure 851 and 55 are shown in FIG. Each of these processors may be a single-core processor (Single-CPU) or a multi-core processor (Multi-CPU). The processor here may refer to one or more devices, circuits, and / or processing cores for processing data (e.g., computer program instructions).
[0096] The memory 52 may be a read-only memory (ROM) or other types of static storage devices that can store static information and instructions, a random access memory (RAM) or other types of dynamic storage devices that can store information and instructions, or an electrically erasable programmable read-only memory (EEPROM), a compact disc read-only memory (CD-ROM) or other optical disc storage, optical disc storage (including compressed optical disc, laser disc, optical disc, digital versatile disc, Blu-ray disc, etc.), a magnetic disk storage medium or other magnetic storage device, or any other medium that can be used to carry or store the desired program code in the form of instructions or data structures and can be accessed by a computer, but is not limited thereto. The memory 52 may exist independently and be connected to the processor 51 via a communication bus 54. The memory 52 may also be integrated with the processor 51.
[0097] In a specific implementation, the memory 52 is used to store the data in the present disclosure and execute the software program in the present disclosure. The processor 51 can execute various functions of the air conditioner by running or executing the software program stored in the memory 52 and calling the data stored in the memory 52.
[0098] The communication interface 53 uses any transceiver-like device to communicate with other devices or communication networks, such as Radio Access Network (RAN), Wireless Local Area Networks (WLAN), terminals, the cloud, etc. The communication interface 53 may include an acquisition unit to implement the acquisition function.
[0099] The communication bus 54 may be an Industry Standard Architecture (ISA) bus, a Peripheral Component Interconnect (PCI) bus, or an Extended Industry Standard Architecture (EISA) bus. The bus may be divided into an address bus, a data bus, a control bus, etc. For ease of representation, Figure 8 Only one thick line is used in the diagram, but this does not mean that there is only one bus or only one type of bus.
[0100] As an example, combine Figure 7 The function implemented by the acquisition unit 102 in the multimedia file playback device 10 is similar to Figure 8 The function of the communication interface 53 in the multimedia file playback device 10 is the same as that of the processing unit 101 in the multimedia file playback device 10. Figure 8 The function of the processor 51 in the multimedia file playback device 10 is the same as that of the storage unit 103 in the multimedia file playback device 10. Figure 8 The function of the memory 52 in is the same.
[0101] Another embodiment of the present disclosure further provides a computer-readable storage medium having a computer program stored thereon. When the computer program is executed by a computing device, the computing device executes the intent recognition method shown in the above method embodiment.
[0102] In some embodiments, the disclosed methods may be implemented as computer program instructions encoded in a machine-readable format on a computer-readable storage medium or on other non-transitory media or articles of manufacture.
[0103] Fig. 9 A conceptual partial view of a computer program product provided by an embodiment of the present disclosure is schematically shown, wherein the computer program product includes a computer program for executing a computer process on a computing device.
[0104] In one embodiment, the computer program product is provided using a signal bearing medium 410. The signal bearing medium 410 may include one or more program instructions, which when executed by one or more processors may provide the above-mentioned Figure 1 Thus, for example, reference to Figure 1 In the embodiment shown in , one or more features of S11-S14 can be undertaken by one or more instructions associated with the signal bearing medium 410. In addition, Fig. 9 The program instructions in also describe example instructions.
[0105] In some examples, signal bearing medium 410 may include computer readable medium 411, such as, but not limited to, a hard drive, a compact disk (CD), a digital video disk (DVD), a digital tape, a memory, a read-only memory (ROM), or a random access memory (RAM), and the like.
[0106] In some implementations, signal bearing medium 410 may include computer recordable medium 412 such as, but not limited to, memory, read / write (R / W) CD, R / W DVD, and the like.
[0107] In some implementations, signal bearing medium 410 may include communication medium 413 such as, but not limited to, digital and / or analog communication media (eg, fiber optic cables, waveguides, wired communication links, wireless communication links, etc.).
[0108] Signal bearing medium 410 may be communicated by a wireless form of communication medium 413 (eg, wireless communication medium complying with IEEE 802.41 standard or other transmission protocol). The one or more program instructions may be, for example, computing device executable instructions or logic implementing instructions.
[0109] In some examples, such as for Figure 7 The described multimedia file playback device 10 may be configured to provide various operations, functions, or actions in response to one or more program instructions via the computer-readable medium 411 , the computer-recordable medium 412 , and / or the communication medium 413 .
[0110] Through the description of the above implementation methods, technical personnel in the relevant field can clearly understand that for the convenience and simplicity of description, only the division of the above-mentioned functional modules is used as an example. In actual applications, the above-mentioned functions can be assigned to different functional modules as needed, that is, the internal structure of the device can be divided into different functional modules to complete all or part of the functions described above.
[0111] In the several embodiments provided in the present disclosure, it should be understood that the disclosed devices and methods can be implemented in other ways. For example, the device embodiments described above are only schematic. For example, the division of the modules or units is only a logical function division. There may be other division methods in actual implementation, such as multiple units or components can be combined or integrated into another device, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, indirect coupling or communication connection of devices or units, which can be electrical, mechanical or other forms.
[0112] The units described as separate components may or may not be physically separated, and the components shown as units may be one physical unit or multiple physical units, that is, they may be located in one place or distributed in multiple different places. Some or all of the units may be selected according to actual needs to achieve the purpose of the present embodiment.
[0113] In addition, each functional unit in each embodiment of the present disclosure may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit. The above-mentioned integrated unit may be implemented in the form of hardware or in the form of software functional units.
[0114] If the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a readable storage medium. Based on this understanding, the technical solution of the embodiment of the present disclosure is essentially or the part that contributes to the prior art or all or part of the technical solution can be embodied in the form of a software product, which is stored in a storage medium and includes several instructions for a device (which can be a single-chip microcomputer, chip, etc.) or a processor (processor) to perform all or part of the steps of the method described in each embodiment of the present disclosure. The aforementioned storage medium includes: various media that can store program codes, such as USB flash drives, mobile hard drives, ROM, RAM, magnetic disks or optical disks.
[0115] The above description is only a specific embodiment of the present disclosure, so that those skilled in the art can understand or implement the present disclosure. Various modifications to these embodiments will be apparent to those skilled in the art, and the general principles defined herein may be implemented in other embodiments without departing from the spirit or scope of the present disclosure. Therefore, the present disclosure will not be limited to the embodiments described herein, but will conform to the widest scope consistent with the principles and novel features disclosed herein.
Claims
1. A method for playing a multimedia file, characterized in that: include: In the case of playing a multimedia file, obtaining cabin voice data in the cabin; wherein the multimedia file includes actual dubbing data and actual background sound data; Calculating the similarity between the actual dubbing data and the in-cabin voice data to obtain the similarity between the in-cabin voice data and the actual dubbing data; In the case where it is determined that the similarity is greater than or equal to the similarity threshold, obtaining the accompaniment parameters of the multimedia file; wherein the accompaniment parameters include the volume value of the actual dubbing data and the volume value of the actual background sound data; The multimedia file is played according to the accompaniment parameters.
2. The method for playing multimedia files according to claim 1, characterized in that: The calculating the similarity between the actual dubbing data and the in-cabin voice data to obtain the similarity between the in-cabin voice data and the actual dubbing data includes: Performing a speech-to-text operation on the in-cabin voice data to obtain in-cabin text data corresponding to the in-cabin voice data; In the case where the in-cabin text data contains characters, respectively extracting text features of the actual dubbing data and the actual text data corresponding to the in-cabin text data to obtain a first text feature vector corresponding to the in-cabin text data and a second text feature vector corresponding to the actual text data; Calculate a first relationship between the first text feature vector and the second text feature vector, and use the first relationship as the similarity between the actual dubbing data and the in-cabin voice data; wherein the first relationship includes any one of cosine similarity and a preset distance, and the preset distance includes any one of Euclidean distance and Manhattan distance.
3. The method for playing multimedia files according to claim 2, characterized in that: The method further comprises: When the in-cabin text data does not contain any characters, respectively extracting audio features of the actual dubbing data and the in-cabin voice data to obtain a first audio feature vector corresponding to the in-cabin voice data and a second audio feature vector corresponding to the actual dubbing data; Calculate a second relationship between the first audio feature vector and the second audio feature vector, and use the second relationship as the similarity between the actual dubbing data and the in-cabin voice data; wherein the second relationship includes any one of cosine similarity and a preset distance, and the preset distance includes any one of Euclidean distance and Manhattan distance.
4. The method for playing multimedia files according to claim 1, characterized in that: In the case of playing the multimedia file, obtaining the cabin voice data in the cabin includes: When playing a multimedia file, the sound source of the multimedia file is separated to obtain actual dubbing data and actual background sound data corresponding to the multimedia file.
5. The method for playing multimedia files according to claim 1, characterized in that: When the similarity is greater than or equal to a similarity threshold, obtaining the accompaniment parameter of the multimedia file includes: When the similarity is greater than or equal to a similarity threshold, obtaining a file type of the multimedia file; A preset relationship table is queried based on the file type to obtain the accompaniment parameters corresponding to the file type; wherein different file types correspond to different accompaniment parameters.
6. The method for playing multimedia files according to claim 1, characterized in that: After playing the multimedia file according to the accompaniment parameters, the method further includes: When the similarity is less than the similarity threshold, the multimedia file is played according to preset parameters; wherein the preset parameters are playing parameters set when playing the multimedia file.
7. The method for playing multimedia files according to claim 1, characterized in that: The cabin includes at least one functional area, one of the functional areas includes at least one audio playback device and at least one lighting device, and the accompaniment parameters also include lighting parameters; The step of playing the multimedia file according to the accompaniment parameter comprises: Performing position analysis on the in-cabin voice data to obtain a functional area where the in-cabin voice data is emitted; The multimedia file is played in the functional area according to the accompaniment parameters.
8. A multimedia file playback device, characterized in that: include: The processing unit is used to control the acquisition unit to acquire the in-cabin voice data in the cabin when playing the multimedia file; wherein the multimedia file includes actual dubbing data and actual background sound data; The processing unit is further used to calculate the similarity between the actual dubbing data and the in-cabin voice data acquired by the acquisition unit, so as to obtain the similarity between the in-cabin voice data and the actual dubbing data; The processing unit is further configured to control the acquisition unit to acquire the accompaniment parameters of the multimedia file when the similarity is greater than or equal to the similarity threshold; wherein the accompaniment parameters include the volume value of the actual dubbing data and the volume value of the actual background sound data; The processing unit is further used to play the multimedia file according to the accompaniment parameters obtained by the obtaining unit.
9. An electronic device, characterized in that: include: A memory and a processor, wherein the memory is used to store a computer program; and the processor is used to enable the electronic device to implement the multimedia file playing method described in any one of claims 1 to 7 when executing the computer program.
10. A computer-readable storage medium, characterized in that: include: The computer-readable storage medium stores a computer program, and when the computer program is executed by a computing device, the computing device implements the method for playing a multimedia file according to any one of claims 1 to 7.
11. A vehicle, comprising the multimedia file playing device according to claim 8.
