A method, device, equipment and storage medium for controlling the playback of in-vehicle music

Abstract

The embodiment of the present invention discloses a method, device, equipment and storage medium for controlling the playback of in-vehicle music. The method comprises: obtaining the currently played music of the target vehicle, determining the emotion detection reference point corresponding to the currently played music; when the currently played music is played to the emotion detection reference point, obtaining the object image information and object sound information of the target object in the target vehicle; determining the target emotion of the target object according to the object image information and the object sound information; determining the playback mode of the currently played music according to the target emotion, wherein the playback mode includes continuing to play or stopping to play. Thus, intelligent real-time adjustment of the in-vehicle music is realized, and the user experience is improved.

Description

Technical Field

[0001] The present invention relates to the technical field of in-vehicle music, and in particular to a method, device, equipment and storage medium for controlling the playback of in-vehicle music. Background Art

[0002] In-car music playback is currently the main form of entertainment while driving. When users are driving, they need to manually switch or select music, which will distract users from driving, affecting not only driving safety but also a poor entertainment experience for users.

[0003] In the prior art, music is often recommended and played while the user is driving based on the big data of the user's daily use. However, the current user's experience of the current music is not taken into consideration during the music playback process. There is a lack of user perception of the preferences of the currently playing music, and it is difficult to meet the demand for real-time adjustment of the played music according to the user's emotions. Summary of the invention

[0004] The present invention provides a method, device, equipment and storage medium for controlling the playback of in-vehicle music, which can realize intelligent real-time adjustment of in-vehicle music and improve user experience.

[0005] According to one aspect of the present invention, a method for controlling the playback of in-vehicle music is provided, comprising:

[0006] Obtaining the currently playing music of the target vehicle, and determining an emotion detection reference point corresponding to the currently playing music;

[0007] When the currently playing music pair is played to the emotion detection reference point, acquiring object image information and object sound information of the target object in the target vehicle;

[0008] Determining a target emotion of the target object according to the object image information and the object sound information;

[0009] The playing mode of the currently playing music is determined according to the target emotion, wherein the playing mode includes continuing to play or stopping to play.

[0010] According to another aspect of the present invention, there is provided a vehicle-mounted music playback control device, comprising:

[0011] A music acquisition module, used to acquire the currently played music of the target vehicle and determine the emotion detection reference point corresponding to the currently played music;

[0012] A target object detection module, configured to obtain object image information and object sound information of a target object in the target vehicle when the currently playing music pair is played to the emotion detection reference point;

[0013] A target emotion determination module, used to determine the target emotion of the target object according to the object image information and the object sound information;

[0014] The music control module is used to determine the playing mode of the currently playing music according to the target emotion, wherein the playing mode includes continuing to play or stopping playing.

[0015] According to another aspect of the present invention, an electronic device is provided, the electronic device comprising:

[0016] at least one processor; and

[0017] a memory communicatively connected to the at least one processor; wherein,

[0018] The memory stores a computer program that can be executed by the at least one processor, and the computer program is executed by the at least one processor so that the at least one processor can execute the method for controlling the playback of in-vehicle music described in any embodiment of the present invention.

[0019] According to another aspect of the present invention, a computer-readable storage medium is provided, wherein the computer-readable storage medium stores computer instructions, and the computer instructions are used to enable a processor to implement the method for controlling the playback of in-vehicle music described in any embodiment of the present invention when executed.

[0020] The technical solution of the embodiment of the present invention obtains the currently playing music of the target vehicle and determines the emotion detection reference point corresponding to the currently playing music. The emotional state of the user can be accurately identified through the emotion detection reference point, thereby improving the accuracy of emotion detection. When the currently playing music is played to the emotion detection reference point, the object image information and object sound information of the target object in the target vehicle are obtained, and the target emotion of the target object is determined according to the object image information and the object sound information. The playing mode of the currently playing music is determined according to the target emotion. The user's emotion is jointly judged using the user's sound information and image information. The user's real-time emotion can be accurately perceived, and the playing mode of the in-vehicle music can be intelligently adjusted in real time, thereby improving the user's usage experience.

[0021] It should be understood that the contents described in this section are not intended to identify the key or important features of the embodiments of the present invention, nor are they intended to limit the scope of the present invention. Other features of the present invention will become easily understood through the following description. BRIEF DESCRIPTION OF THE DRAWINGS

[0022] In order to more clearly illustrate the technical solutions in the embodiments of the present invention, the following briefly introduces the drawings required for use in the description of the embodiments. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without creative work.

[0023] Figure 1 is a flow chart of a method for controlling the playback of in-vehicle music provided in Embodiment 1 of the present invention;

[0024] Figure 2 is a flow chart of another method for controlling the playback of in-vehicle music provided by Embodiment 2 of the present invention;

[0025] Figure 3 is a flow chart of another method for controlling the playback of in-vehicle music provided in Embodiment 3 of the present invention;

[0026] Figure 4 is a structural diagram of a car music playback control device provided by a fourth embodiment of the present invention;

[0027] Figure 5 The present invention is a schematic diagram of the structure of an electronic device for implementing the in-vehicle music playback control method according to an embodiment of the present invention. DETAILED DESCRIPTION

[0028] In order to enable those skilled in the art to better understand the scheme of the present invention, the technical scheme in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work should fall within the scope of protection of the present invention.

[0029] It should be noted that the terms "first", "second", etc. in the specification and claims of the present invention and the above-mentioned drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence. It should be understood that the data used in this way can be interchanged where appropriate, so that the embodiments of the present invention described herein can be implemented in an order other than those illustrated or described herein. In addition, the terms "including" and "having" and any variations thereof are intended to cover non-exclusive inclusions, for example, a process, method, system, product or device that includes a series of steps or units is not necessarily limited to those steps or units that are clearly listed, but may include other steps or units that are not clearly listed or inherent to these processes, methods, products or devices.

[0030] Embodiment 1

[0031] Figure 1 A flowchart of a method for controlling the playback of in-vehicle music is provided for the first embodiment of the present invention. This embodiment is applicable to the case where the music playback is controlled while driving a vehicle. The method can be executed by controlling the playback of in-vehicle music. The in-vehicle music playback control device can be implemented in the form of hardware and / or software. The in-vehicle music playback control device can be configured in a vehicle. Figure 1 As shown, the method includes:

[0032] S110: Acquire the currently playing music of the target vehicle, and determine the emotion detection reference point corresponding to the currently playing music.

[0033] The currently playing music may be music played by a car music player. The emotion detection reference point may be a detection time reference point in the currently playing music for detecting the user's emotion. Optionally, the number of emotion detection reference points is at least greater than 1. In order to accurately perceive the user's preference for music, exemplarily, the currently playing music includes at least a plurality of emotion detection reference points.

[0034] Optionally, when the user is driving the target vehicle, when the user turns on the target vehicle's in-vehicle music player to play music, the music played by the in-vehicle music player is obtained, and the emotion detection reference point of the music is queried based on the music played by the in-vehicle music player. The user can connect the in-vehicle music player to play music through a smart device, or directly use the in-vehicle music player to select music to play, and when the in-vehicle music player receives the song to be played, the emotion detection reference point of the song to be played is obtained.

[0035] Optionally, in one embodiment of the present invention, determining the emotion detection reference point corresponding to the currently playing music includes: using a preset speech endpoint detection algorithm to obtain the speech starting point and the speech ending point in the currently playing music, and using the music starting point and the music ending point of the currently playing music and the acquired speech starting point and the speech ending point as the emotion detection reference points corresponding to the currently playing music.

[0036] Specifically, after obtaining the currently playing music, the preset voice endpoint detection algorithm can be sampled to obtain the voice starting point and voice ending point of the currently playing music singing segment, and the music starting point and music ending point of the currently playing music singing segment and the obtained voice starting point and voice ending point are used as the emotion detection reference points corresponding to the currently playing music. Among them, the preset voice endpoint detection algorithm can include but is not limited to an endpoint detection algorithm based on volume, an endpoint detection algorithm combined with a high-order difference of volume, and an endpoint detection based on frequency domain.

[0037] S120. When the currently playing music pair is played to the emotion detection reference point, the object image information and the object sound information of the target object in the target vehicle are obtained.

[0038] The target image information may be the image information of the user in the target vehicle, including video information and photo information. The target sound information may be the voice information of the user in the target vehicle, or other sound information of the user, such as laughter, crying or humming.

[0039] Specifically, when the user plays music in the target vehicle, when the currently played music reaches the emotion detection reference point of the music smart cabinet, the image and voice information of the target object are collected by the image acquisition device and the sound acquisition device in the target vehicle. The target object can be all users within the image acquisition range, or users within the image acquisition range and matching the pre-stored whitelist users, etc.

[0040] In an embodiment of the present invention, when the currently playing music is played to the emotion detection reference point, the object image information and object sound information of the target object in the target vehicle are obtained. Compared with obtaining the object image information and object sound information of the target object in real time, the memory consumption caused by data processing is reduced, and the emotion-related information can be obtained in a targeted manner in combination with the characteristics of the currently playing music.

[0041] S130: Determine a target emotion of the target object according to the object image information and the object sound information.

[0042] The target emotion may be the emotion of the user at the emotion detection reference point. Optionally, the emotion may be any one of happiness, sadness, surprise, regret, boredom, anxiety or calmness.

[0043] Specifically, after acquiring the user's object image information and object sound information, the target vehicle processes the user's object image information and object sound information respectively, and determines the target emotion of the target object according to the processing results of the user's object image information and object sound information.

[0044] Optionally, the object image information includes a facial image and a body image, and the first emotion information of the target object can be determined based on the object image information. Specifically, the first emotion information of the target object can be determined based on the facial image and the body image in the object image information. Furthermore, the second emotion information of the target object can be determined based on the object sound information. Specifically, the second emotion information of the target object can be determined by performing speech recognition on the object sound information based on the object sound information, determining the speech text corresponding to the object sound information, and determining the second emotion information of the target object by identifying keywords in the speech text. Then, the target emotion of the target object is determined based on the first emotion information and the second emotion information.

[0045] S140. Determine a playing mode of the currently playing music according to the target emotion, wherein the playing mode includes continuing to play or stopping to play.

[0046] In an embodiment of the present invention, after obtaining the user's target emotion, it is determined whether the user's target emotion is consistent with the playback mode of the currently playing music. If the user's target emotion meets the conditions for continuing to play, the currently playing music continues to be played; if the user's target emotion does not meet the conditions for continuing to play, the currently playing music is stopped.

[0047] Optionally, in an embodiment of the present invention, determining the playing mode of the currently playing music according to the target emotion includes: obtaining label data of the currently playing music, wherein the label data at least includes a genre label; determining the playing mode of the currently playing music according to the target emotion and the label data.

[0048] The genre label may reflect the mood of the music, for example, the genre label may be lyrical, melancholy, cheerful, lonely, etc.

[0049] Specifically, in the car music player, each piece of music has a corresponding label. The car music player obtains the label data of the currently playing music and matches the playing mode of the current music according to the label data of the currently playing music and the target emotion.

[0050] Optionally, when the currently playing music is stopped, the next piece of music is recommended according to the target emotion of the user, and the recommended music is played in the in-vehicle music player of the target vehicle.

[0051] The technical solution of the embodiment of the present invention obtains the currently playing music of the target vehicle and determines the emotion detection reference point corresponding to the currently playing music. The emotional state of the user can be accurately identified through the emotion detection reference point, thereby improving the accuracy of emotion detection. When the currently playing music is played to the emotion detection reference point, the object image information and object sound information of the target object in the target vehicle are obtained, and the target emotion of the target object is determined according to the object image information and the object sound information. The playing mode of the currently playing music is determined according to the target emotion. The user's emotion is jointly judged using the user's sound information and image information. The user's real-time emotion can be accurately perceived, and the playing mode of the in-vehicle music can be intelligently adjusted in real time, thereby improving the user's usage experience.

[0052] Embodiment 2

[0053] Figure 2 A flowchart of another method for controlling the playback of in-vehicle music is provided for the second embodiment of the present invention. This embodiment mainly further explains the method for determining the reference point for emotion detection. Figure 2 As shown, the method includes:

[0054] S210. Obtain a heat curve of the currently playing music, wherein the heat curve is constructed based on each playing time point of the currently playing music and the playing heat values ​​corresponding to the playing time points.

[0055] Specifically, the playback time point may be the time point of the currently playing music in the music playback progress, and the playback heat value may be a comprehensive reflection of the popularity of the current playback time point, or may be a reflection of the probability of not interrupting the music playback at the current playback time point. The target vehicle queries and obtains the heat curve of the currently playing music based on the in-vehicle music player. Optionally, the in-vehicle music player obtains a list of music to be played for the currently playing music, and queries and obtains the heat curves of a preset number of music to be played in the list of music to be played. The preset number may be the number of all the music to be played by the in-vehicle music player, or the number of music to be played may be set by the user.

[0056] S220: Determine an emotion detection reference point corresponding to the currently playing music according to the playing heat value corresponding to each playing time point.

[0057] Specifically, obtain the playback heat value corresponding to each playback time point in the heat curve of the currently playing music, extract each playback time point whose playback heat value meets the emotion detection reference point, and use each playback time point that meets the emotion detection reference point as the emotion detection reference point corresponding to the currently playing music.

[0058] Optionally, in another embodiment of the present invention, determining the emotion detection reference points corresponding to the currently playing music based on the playback heat values ​​corresponding to each playback time point includes: determining the playback time points corresponding to the peaks and troughs of the heat curve based on a preset peak and trough detection algorithm, and using the playback time points corresponding to the peaks and troughs as the emotion detection reference points corresponding to the currently playing music.

[0059] Specifically, obtain the heat curve of the currently playing music, detect the heat curve of the currently playing music according to a preset peak and trough detection algorithm, detect the peaks and troughs of the heat curve of the currently playing music, and then determine the peak time points corresponding to the peaks and troughs of the heat curve of the currently playing music, and use the playing time points corresponding to the peaks and troughs as emotion detection reference points corresponding to the currently playing music.

[0060] Among them, the preset peak and trough detection algorithm may include but is not limited to any peak and trough detection algorithm in signal processing, such as comparison judgment method, first-order difference combined with comparison judgment method or second-order difference judgment method and zero-crossing point detection combined with maximum value detection judgment method.

[0061] S230. When the currently playing music pair is played to the emotion detection reference point, the object image information and the object sound information of the target object in the target vehicle are obtained.

[0062] S240: Determine a target emotion of the target object according to the object image information and the object sound information.

[0063] S250. Determine a playing mode of the currently playing music according to the target emotion, wherein the playing mode includes continuing to play or stopping to play.

[0064] The technical solution of the embodiment of the present invention obtains the heat curve of the currently playing music, and determines the emotion detection reference point corresponding to the currently playing music according to the playback heat values ​​corresponding to each playback time point in the heat curve. The heat curve can be used to accurately know the changes in the user's emotional state when the music is currently playing, and accurately analyze the emotion detection reference point, thereby further improving the accuracy of emotion detection. When the currently playing music is played to the emotion detection reference point, the object image information and object sound information of the target object in the target vehicle are obtained, and the target emotion of the target object is determined according to the object image information and the object sound information. The current music playback mode is matched with the target emotion according to the genre label of the currently playing music, and the user's emotion and the music itself are accurately matched. While accurately perceiving the user's real-time emotions, music suitable for the current emotions is selected, thereby realizing intelligent and real-time adjustment of the playback mode of the in-vehicle music, thereby improving the user's usage experience.

[0065] Embodiment 3

[0066] Figure 2 A flowchart of another method for controlling the playback of in-vehicle music is provided for the third embodiment of the present invention. This embodiment mainly further explains the method for determining the reference point for emotion detection. Figure 2 As shown, the method includes:

[0067] S310: Acquire the currently playing music of the target vehicle, and determine the emotion detection reference point corresponding to the currently playing music.

[0068] S320: When the currently playing music pair is played to the emotion detection reference point, the object image information and the object sound information of the target object in the target vehicle are obtained.

[0069] Pre-collect the image data and sound data of the target object, establish the image data set and sound data set of the object, split the image data set and sound data set of the object, divide the image data set into an image data training set and an image data test set, divide the sound data set into a sound data training set and a sound data test set, train the neural network according to the image data and sound data training set and the image data test set, and train to generate the first deep learning model

[0070] S330. Determine the first emotion information of the target object according to the object image information and a pre-trained first emotion recognition model, wherein the object image information includes facial image information and / or body image information, and the first emotion recognition model is trained based on a pre-established first deep learning model.

[0071] The first emotion recognition model is used to recognize an emotion recognition model of image information. The first emotion information is emotion information recognized based on image information of the object. The first deep learning model may include but is not limited to at least one of a convolutional neural network model, a recurrent neural network, and a structured recursive neural network.

[0072] Specifically, before determining the first emotion information of the target object according to the object image information and the pre-trained first emotion recognition model, the method further includes:

[0073] Sample image information is obtained, and an image training set of the sample image information and an image test set of the image information are established according to the sample image information.

[0074] The sample image information in the image training set is input into the first deep learning model, and the first deep learning model is trained to generate a first emotion recognition basic model. Specifically, the sample image information in the image training set is input into the first deep learning model to generate training emotion information; the training error is determined based on the generated training emotion information and the expected emotion information corresponding to the sample image information, and then the parameters of the first deep learning model are adjusted based on the training error to obtain the first emotion recognition basic model.

[0075] The test image information in the image test set is input into the first emotion recognition basic model to generate test emotion information, and it is determined whether the accuracy of the current test emotion information and the expected emotion information meets the preset accuracy. The preset accuracy can be the minimum accuracy of the output of the first emotion recognition basic model that meets the emotion recognition. If the accuracy of the current test emotion information and the expected emotion information does not meet the preset accuracy, training is continued based on the sample image information in the image training set so that the accuracy of the current test emotion information and the expected emotion information meets the preset accuracy. If the accuracy of the current test emotion information and the expected emotion information meets the preset accuracy, the first emotion recognition basic model is determined as the first emotion recognition model.

[0076] Furthermore, the object image information is input into a first emotion recognition model to generate first emotion information of the target object.

[0077] S340. Determine the second emotion information of the target object according to the object sound information and a pre-trained second emotion recognition model, wherein the second emotion recognition model is trained based on a pre-established second deep learning model.

[0078] The second emotion recognition model is an emotion recognition model for recognizing sound information. The second emotion information is emotion information recognized based on the sound information of the object. The second deep learning model may include but is not limited to at least one of a convolutional neural network model, a recurrent neural network, and a structured recursive neural network.

[0079] Specifically, before determining the second emotion information of the target object according to the object sound information and the pre-trained second emotion recognition model, the method further includes:

[0080] Sample sound information is obtained, and a sound training set of the sample sound information and a sound test set of the sample sound information are established according to the sample sound information.

[0081] The sample sound information in the sound training set is input into the second deep learning model, and the second deep learning model is trained to generate a second emotion recognition basic model. Specifically, the sample sound information in the sound training set is input into the second deep learning model to generate training emotion information; the training error is determined based on the generated training emotion information and the expected emotion information corresponding to the sample sound information, and then the parameters of the second deep learning model are adjusted based on the training error to obtain the second emotion recognition basic model.

[0082] The test sound information in the sound test set is input into the second emotion recognition basic model to generate test emotion information, and it is determined whether the accuracy of the current test emotion information and the expected emotion information meets the preset accuracy. The preset accuracy can be the minimum accuracy of the output of the second emotion recognition basic model that meets the emotion recognition. If the accuracy of the current test emotion information and the expected emotion information does not meet the preset accuracy, training is continued based on the sample sound information in the sound training set so that if the accuracy of the current test emotion information and the expected emotion information meets the preset accuracy. If the accuracy of the current test emotion information and the expected emotion information meets the preset accuracy, the second emotion recognition basic model is determined as the second emotion recognition model.

[0083] Furthermore, the object sound information is input into a second emotion recognition model to generate second emotion information of the target object.

[0084] S350: Determine a target emotion of the target object according to the first emotion information and the second emotion information.

[0085] Optionally, in another embodiment of the present invention, determining the target emotion of the target object based on the first emotion information and the second emotion information includes: for the same emotion in the first emotion information and the second emotion information, performing a weighted summation of the emotion probability value of the emotion in the first emotion information and the emotion probability value in the second emotion information to obtain a target emotion value corresponding to the emotion; and determining the emotion corresponding to the largest target emotion value among the target emotion values ​​corresponding to various emotions as the target emotion of the target object.

[0086] The emotion information probability value may be a probability value of the first emotion information being the target emotion, and the second emotion probability value may be a probability value of the second emotion information being the target emotion.

[0087] Specifically, after the target vehicle obtains the first emotion information and the second emotion information, for the same kind of first emotion information and second emotion information, the emotion probability value of the target emotion in the first emotion information and the emotion probability value of the second emotion information are weightedly summed according to the weight value of the emotion information, and the target emotion value is calculated, and the emotion corresponding to the largest target emotion value among the target emotion values ​​corresponding to various emotions is determined as the target emotion of the target object.

[0088] The calculation formula for weighted sum of the emotion probability value of the first emotion information and the emotion probability value of the second emotion information is as follows:

[0089] V=(1-Wk)*Vt+Wk*Vy

[0090] Among them, V is the target emotion value; Vt is the emotion probability value of the first emotion information; Vy is the emotion probability value of the second emotion information; Wk is the weight value of the emotion information.

[0091] Exemplarily, the weighted sum calculation based on the emotion probability value of the first emotion information and the emotion probability value of the second emotion information is shown in the following table:

[0092]

[0093] According to the weighted sum calculation formula, the emotional value of happiness is 0.025375, the emotional value of sadness is 0.84925, the emotional value of surprise is 0.025375, and the emotional values ​​of regret, boredom, anxiety and calmness are 0.025. Therefore, the target emotional value is determined to be 0.84925, and the target emotion is determined to be sadness.

[0094] S360: Determine a playing mode of the currently playing music according to the target emotion, wherein the playing mode includes continuing to play or stopping to play.

[0095] The technical solution of the embodiment of the present invention obtains the currently played music of the target vehicle and determines the emotion detection reference point corresponding to the currently played music. The emotion detection reference point can accurately identify the emotional state of the user, thereby improving the accuracy of emotion detection. When the currently played music is played to the emotion detection reference point, the object image information and the object sound information of the target object in the target vehicle are obtained. The object image information and the object sound information of the target object are respectively trained to generate a first emotion recognition model and a second emotion recognition model through a pre-established second deep learning model, thereby determining the first emotion information and the second emotion information of the object. The emotion recognition model can make the result of emotion recognition closer to the actual emotion of the user, reduce the error of emotion recognition, and improve the accuracy and efficiency of identifying the target emotion. The weighted summation is used to determine the target emotion of the user for the first emotion information and the second emotion information, and the fusion method is used to reduce the single emotion estimation deviation, thereby further improving the accuracy of identifying the target emotion. The playing mode of the currently played music is determined according to the target emotion, and the user's sound information and image information are used to jointly judge the user's emotion. The user's real-time emotion can be accurately perceived, and the playing mode of the in-vehicle music can be intelligently adjusted in real time, thereby improving the user's usage experience.

[0096] Embodiment 4

[0097] Figure 4 This is a schematic diagram of the structure of a car music playback control device provided by the fourth embodiment of the present invention. Figure 4 As shown, the device specifically includes: a music acquisition module 410, a target object detection module 420, a target emotion determination module 430 and a music control module 440. Among them:

[0098] The music acquisition module 410 is used to acquire the currently played music of the target vehicle and determine the emotion detection reference point corresponding to the currently played music;

[0099] A target object detection module 420, configured to obtain object image information and object sound information of a target object in the target vehicle when the currently playing music pair is played to the emotion detection reference point;

[0100] A target emotion determination module 430, configured to determine a target emotion of the target object according to the object image information and the object sound information;

[0101] The music control module 440 is used to determine the playing mode of the currently playing music according to the target emotion, wherein the playing mode includes continuing to play or stopping playing.

[0102] The technical solution of the embodiment of the present invention obtains the currently playing music of the target vehicle and determines the emotion detection reference point corresponding to the currently playing music. The emotional state of the user can be accurately identified through the emotion detection reference point, thereby improving the accuracy of emotion detection. When the currently playing music is played to the emotion detection reference point, the object image information and object sound information of the target object in the target vehicle are obtained, and the target emotion of the target object is determined according to the object image information and the object sound information. The playing mode of the currently playing music is determined according to the target emotion. The user's emotion is jointly judged using the user's sound information and image information. The user's real-time emotion can be accurately perceived, and the playing mode of the in-vehicle music can be intelligently adjusted in real time, thereby improving the user's usage experience.

[0103] Optionally, the music acquisition module 410 is used to:

[0104] Obtaining a heat curve of the currently playing music, wherein the heat curve is constructed based on each playing time point of the currently playing music and the playing heat values ​​corresponding to the playing time points;

[0105] The emotion detection reference point corresponding to the currently playing music is determined according to the playing heat value corresponding to each playing time point.

[0106] Optionally, the music acquisition module 410 is further configured to:

[0107] The playback time points corresponding to the peaks and troughs of the heat curve are determined based on a preset peak and trough detection algorithm, and the playback time points corresponding to the peaks and troughs are used as emotion detection reference points corresponding to the currently playing music.

[0108] Optionally, the music acquisition module 410 is further configured to:

[0109] A preset speech endpoint detection algorithm is used to obtain the speech starting point and speech ending point in the currently playing music, and the music starting point and music ending point of the currently playing music and the acquired speech starting point and speech ending point are used as emotion detection reference points corresponding to the currently playing music.

[0110] Optionally, the target emotion determination module 430 is specifically used for:

[0111] Determining first emotion information of the target object according to the object image information and a pre-trained first emotion recognition model, wherein the object image information includes facial image information and / or body image information, and the first emotion recognition model is trained based on a pre-established first deep learning model;

[0112] Determining the second emotion information of the target object according to the object sound information and a pre-trained second emotion recognition model, wherein the second emotion recognition model is trained based on a pre-established second deep learning model;

[0113] A target emotion of the target object is determined according to the first emotion information and the second emotion information.

[0114] Optionally, the target emotion determination module 430 is specifically used to

[0115] For the same emotion in the first emotion information and the second emotion information, weighted summing the emotion probability value of the emotion in the first emotion information and the emotion probability value in the second emotion information to obtain a target emotion value corresponding to the emotion;

[0116] The emotion corresponding to the largest target emotion value among the target emotion values ​​corresponding to various emotions is determined as the target emotion of the target object.

[0117] Optionally, the music control module 440 is specifically used for:

[0118] Acquire tag data of the currently playing music, wherein the tag data at least includes a genre tag;

[0119] The playing mode of the currently playing music is determined according to the target emotion and the tag data.

[0120] The in-vehicle music playback control device provided in the embodiment of the present invention can execute the in-vehicle music playback control method provided in any embodiment of the present invention, and has the corresponding functional modules and beneficial effects for executing the in-vehicle music playback control method.

[0121] It is worth noting that in the embodiment of the above-mentioned in-vehicle music playback control device, the various units and modules included are only divided according to functional logic, but are not limited to the above-mentioned division, as long as the corresponding functions can be achieved; in addition, the specific names of the functional units are only for the convenience of distinguishing each other, and are not used to limit the scope of protection of the present invention.

[0122] Embodiment 5

[0123] Figure 5A schematic diagram of the structure of an electronic device 10 that can be used to implement an embodiment of the present invention is shown. The electronic device is intended to represent various forms of digital computers, such as laptop computers, desktop computers, workstations, personal digital assistants, servers, blade servers, mainframe computers, and other suitable computers. The electronic device can also represent various forms of mobile devices, such as personal digital processing, cellular phones, smart phones, wearable devices (such as helmets, glasses, watches, etc.) and other similar computing devices. The components shown herein, their connections and relationships, and their functions are merely examples and are not intended to limit the implementation of the present invention described and / or required herein.

[0124] like Figure 5 As shown, the electronic device 10 includes at least one processor 11, and a memory connected to the at least one processor 11, such as a read-only memory (ROM) 12, a random access memory (RAM) 13, etc., wherein the memory stores a computer program that can be executed by at least one processor, and the processor 11 can perform various appropriate actions and processes according to the computer program stored in the read-only memory (ROM) 12 or the computer program loaded from the storage unit 18 to the random access memory (RAM) 13. In the RAM 13, various programs and data required for the operation of the electronic device 10 can also be stored. The processor 11, the ROM 12, and the RAM 13 are connected to each other through a bus 14. An input / output (I / O) interface 15 is also connected to the bus 14.

[0125] A number of components in the electronic device 10 are connected to the I / O interface 15, including: an input unit 16, such as a keyboard, a mouse, etc.; an output unit 17, such as various types of displays, speakers, etc.; a storage unit 18, such as a disk, an optical disk, etc.; and a communication unit 19, such as a network card, a modem, a wireless communication transceiver, etc. The communication unit 19 allows the electronic device 10 to exchange information / data with other devices through a computer network such as the Internet and / or various telecommunication networks.

[0126] The processor 11 may be a variety of general and / or special processing components with processing and computing capabilities. Some examples of the processor 11 include, but are not limited to, a central processing unit (CPU), a graphics processing unit (GPU), various dedicated artificial intelligence (AI) computing chips, various processors running machine learning model algorithms, a digital signal processor (DSP), and any appropriate processor, controller, microcontroller, etc. The processor 11 executes the various methods and processes described above, such as a method for controlling the playback of in-car music.

[0127] In some embodiments, the method for controlling the playback of in-car music may be implemented as a computer program, which is tangibly contained in a computer-readable storage medium, such as a storage unit 18. In some embodiments, part or all of the computer program may be loaded and / or installed on the electronic device 10 via the ROM 12 and / or the communication unit 19. When the computer program is loaded into the RAM 13 and executed by the processor 11, one or more steps of the method for controlling the playback of in-car music described above may be performed. Alternatively, in other embodiments, the processor 11 may be configured to execute the method for controlling the playback of in-car music in any other appropriate manner (e.g., by means of firmware).

[0128] Various implementations of the systems and techniques described above herein can be implemented in digital electronic circuit systems, integrated circuit systems, field programmable gate arrays (FPGAs), application specific integrated circuits (ASICs), application specific standard products (ASSPs), systems on chips (SOCs), load programmable logic devices (CPLDs), computer hardware, firmware, software, and / or combinations thereof. These various implementations can include: being implemented in one or more computer programs that can be executed and / or interpreted on a programmable system including at least one programmable processor, which can be a special purpose or general purpose programmable processor that can receive data and instructions from a storage system, at least one input device, and at least one output device, and transmit data and instructions to the storage system, the at least one input device, and the at least one output device.

[0129] Computer programs for implementing the methods of the present invention may be written in any combination of one or more programming languages. These computer programs may be provided to a processor of a general-purpose computer, a special-purpose computer, or other programmable data processing device, so that when the computer program is executed by the processor, the functions / operations specified in the flow chart and / or block diagram are implemented. The computer program may be executed entirely on the machine, partially on the machine, partially on the machine and partially on a remote machine as a stand-alone software package, or entirely on a remote machine or server.

[0130] In the context of the present invention, a computer-readable storage medium may be a tangible medium that may contain or store a computer program for use by or in combination with an instruction execution system, device or equipment. A computer-readable storage medium may include, but is not limited to, electronic, magnetic, optical, electromagnetic, infrared, or semiconductor systems, devices or equipment, or any suitable combination of the foregoing. Alternatively, a computer-readable storage medium may be a machine-readable signal medium. A more specific example of a machine-readable storage medium may include an electrical connection based on one or more lines, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing.

[0131] To provide interaction with a user, the systems and techniques described herein may be implemented on an electronic device having: a display device (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor) for displaying information to the user; and a keyboard and a pointing device (e.g., a mouse or trackball) through which the user can provide input to the electronic device. Other types of devices may also be used to provide interaction with the user; for example, the feedback provided to the user may be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user may be received in any form (including acoustic input, voice input, or tactile input).

[0132] The systems and techniques described herein may be implemented in a computing system that includes backend components (e.g., as a data server), or a computing system that includes middleware components (e.g., an application server), or a computing system that includes frontend components (e.g., a user computer with a graphical user interface or a web browser through which a user can interact with implementations of the systems and techniques described herein), or a computing system that includes any combination of such backend components, middleware components, or frontend components. The components of the system may be interconnected by any form or medium of digital data communication (e.g., a communication network). Examples of communication networks include: a local area network (LAN), a wide area network (WAN), a blockchain network, and the Internet.

[0133] A computing system may include a client and a server. The client and the server are generally remote from each other and usually interact through a communication network. The client and server relationship is generated by computer programs running on the corresponding computers and having a client-server relationship with each other. The server may be a cloud server, also known as a cloud computing server or cloud host, which is a host product in the cloud computing service system to solve the defects of difficult management and weak business scalability in traditional physical hosts and VPS services.

[0134] It should be understood that the various forms of processes shown above can be used to reorder, add or delete steps. For example, the steps described in the present invention can be executed in parallel, sequentially or in different orders, as long as the desired results of the technical solution of the present invention can be achieved, and this document does not limit this.

[0135] Embodiment 5

[0136] This embodiment provides a computer-readable storage medium, on which a computer program is stored. When the program is executed by a processor, the steps of the in-vehicle music playback control method provided by any embodiment of the present invention are implemented. The method includes:

[0137] Obtaining the currently playing music of the target vehicle, and determining an emotion detection reference point corresponding to the currently playing music;

[0138] When the currently playing music pair is played to the emotion detection reference point, acquiring object image information and object sound information of the target object in the target vehicle;

[0139] Determining a target emotion of the target object according to the object image information and the object sound information;

[0140] The playing mode of the currently playing music is determined according to the target emotion, wherein the playing mode includes continuing to play or stopping to play.

[0141] The computer storage medium of the embodiment of the present invention can adopt any combination of one or more computer-readable media. The computer-readable medium can be a computer-readable signal medium or a computer-readable storage medium. The computer-readable storage medium can be, for example, but not limited to: an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, device or device, or any combination of the above. More specific examples (non-exhaustive list) of computer-readable storage media include: an electrical connection with one or more wires, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the above. In this document, a computer-readable storage medium can be any tangible medium containing or storing a program, which can be used by an instruction execution system, device or device or used in combination with it.

[0142] Computer-readable signal media may include data signals propagated in baseband or as part of a carrier wave, which carry computer-readable program code. Such propagated data signals may take a variety of forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination of the above. Computer-readable signal media may also be any computer-readable medium other than a computer-readable storage medium, which may send, propagate, or transmit a program for use by or in conjunction with an instruction execution system, apparatus, or device.

[0143] The program code embodied on the computer readable medium may be transmitted using any appropriate medium, including but not limited to wireless, wireline, optical fiber cable, RF, etc., or any suitable combination of the foregoing.

[0144] Computer program code for performing the operations of the present invention may be written in one or more programming languages ​​or a combination thereof, including object-oriented programming languages ​​such as Java, Smalltalk, C++, and conventional procedural programming languages ​​such as "C" or similar programming languages. The program code may be executed entirely on the user's computer, partially on the user's computer, as a separate software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In the case of a remote computer, the remote computer may be connected to the user's computer through any type of network, including a local area network (LAN) or a wide area network (WAN), or may be connected to an external computer (e.g., via the Internet using an Internet service provider).

[0145] It should be understood by those skilled in the art that the modules or steps of the present invention described above can be implemented by a general-purpose computing device, they can be concentrated on a single computing device, or distributed on a network composed of multiple computing devices, optionally, they can be implemented by a program code executable by a computer device, so that they can be stored in a storage device and executed by the computing device, or they can be made into individual integrated circuit modules, or multiple modules or steps therein can be made into a single integrated circuit module for implementation. Thus, the present invention is not limited to any specific combination of hardware and software.

[0146] It should be understood that the various forms of processes shown above can be used to reorder, add or delete steps. For example, the steps described in the present invention can be executed in parallel, sequentially or in different orders, as long as the desired results of the technical solution of the present invention can be achieved, and this document does not limit this.

[0147] The above specific implementations do not constitute a limitation on the protection scope of the present invention. It should be understood by those skilled in the art that various modifications, combinations, sub-combinations and substitutions can be made according to design requirements and other factors. Any modification, equivalent substitution and improvement made within the spirit and principle of the present invention should be included in the protection scope of the present invention.

Claims

1. A method for controlling the playback of in-vehicle music, characterized in that: include: Obtaining the currently playing music of the target vehicle, and determining an emotion detection reference point corresponding to the currently playing music; When the currently playing music pair is played to the emotion detection reference point, acquiring object image information and object sound information of the target object in the target vehicle; Determining a target emotion of the target object according to the object image information and the object sound information; Determining a playing mode of the currently playing music according to the target emotion, wherein the playing mode includes continuing playing or stopping playing; The determining of the emotion detection reference point corresponding to the currently playing music includes: Obtaining a heat curve of the currently playing music, wherein the heat curve is constructed based on each playing time point of the currently playing music and the playing heat value corresponding to the playing time point; the playing heat value is a comprehensive reflection of the popularity of the current playing time point or the probability of not interrupting the playing of the music at the current playing time point; Determine the emotion detection reference point corresponding to the currently playing music according to the playing heat value corresponding to each playing time point; The step of determining the emotion detection reference point corresponding to the currently playing music according to the playing heat values ​​corresponding to each playing time point includes: The playback time points corresponding to the peaks and troughs of the heat curve are determined based on a preset peak and trough detection algorithm, and the playback time points corresponding to the peaks and troughs are used as emotion detection reference points corresponding to the currently playing music.

2. The method according to claim 1, characterized in that The determining the target emotion of the target object according to the object image information and the object sound information comprises: Determining first emotion information of the target object according to the object image information and a pre-trained first emotion recognition model, wherein the object image information includes facial image information and / or body image information, and the first emotion recognition model is trained based on a pre-established first deep learning model; Determining the second emotion information of the target object according to the object sound information and a pre-trained second emotion recognition model, wherein the second emotion recognition model is trained based on a pre-established second deep learning model; A target emotion of the target object is determined according to the first emotion information and the second emotion information.

3. The method according to claim 2, characterized in that The determining the target emotion of the target object according to the first emotion information and the second emotion information includes: For the same emotion in the first emotion information and the second emotion information, weighted summing the emotion probability value of the emotion in the first emotion information and the emotion probability value in the second emotion information to obtain a target emotion value corresponding to the emotion; The emotion corresponding to the largest target emotion value among the target emotion values ​​corresponding to various emotions is determined as the target emotion of the target object.

4. The method according to claim 1, characterized in that: The determining the playing mode of the currently playing music according to the target emotion includes: Acquire tag data of the currently playing music, wherein the tag data at least includes a genre tag; The playing mode of the currently playing music is determined according to the target emotion and the tag data.

5. A car music playback control device, characterized in that: include: A music acquisition module, used to acquire the currently played music of the target vehicle and determine the emotion detection reference point corresponding to the currently played music; A target object detection module, configured to obtain object image information and object sound information of a target object in the target vehicle when the currently playing music pair is played to the emotion detection reference point; A target emotion determination module, used to determine the target emotion of the target object according to the object image information and the object sound information; A music control module, used to determine a playing mode of the currently playing music according to the target emotion, wherein the playing mode includes continuing to play or stopping playing; The music acquisition module is specifically used for: Obtaining a heat curve of the currently playing music, wherein the heat curve is constructed based on each playing time point of the currently playing music and the playing heat value corresponding to the playing time point; the playing heat value is a comprehensive reflection of the popularity of the current playing time point or the probability of not interrupting the playing of the music at the current playing time point; Determine the emotion detection reference point corresponding to the currently playing music according to the playing heat value corresponding to each playing time point; The music acquisition module is also specifically used for: The playback time points corresponding to the peaks and troughs of the heat curve are determined based on a preset peak and trough detection algorithm, and the playback time points corresponding to the peaks and troughs are used as emotion detection reference points corresponding to the currently playing music.

6. An electronic device, characterized in that: The electronic device comprises: at least one processor; and a memory communicatively connected to the at least one processor; wherein, The memory stores a computer program that can be executed by the at least one processor, and the computer program is executed by the at least one processor so that the at least one processor can execute the in-vehicle music playback control method according to any one of claims 1 to 4.

7. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores computer instructions, and the computer instructions are used to enable a processor to implement the in-vehicle music playback control method according to any one of claims 1 to 4 when executed.

Images