System and method for controlling light with music, and car audio system

Abstract

The present application relates to a system and method for controlling light with music, and a car system. The system comprises a music analysis unit configured to analyze rhythm and content of music, and a light control unit configured to dynamically control at least one of color, brightness, and rhythm of change of color and brightness of light emitted by a light emitting component based on the analysis of the rhythm and content of the music. Embodiments of the present application can achieve better integration between light, music and scene, and provide users with improved audio-visual experience.

Description

Technical Field

[0001] This invention relates to a system and method for controlling lights using music, as well as a vehicle infotainment system. Background Technology

[0002] When people listen to music outdoors, indoors, or in their vehicles, they often desire lighting that changes in sync with the rhythm of the music for a better audiovisual experience. For example, during vehicle use, to provide users with a more comfortable and enjoyable driving experience, ambient lighting is often combined with music, such as linking changes in ambient lighting to the rhythm of the music, making the ambient lighting change with the music. In musical fountains, concerts, or outdoor plazas, the lighting changes with the rhythm of the music, creating a pleasing visual experience.

[0003] A common problem with existing technologies is that they typically only allow lighting changes to rhythmically match the music, failing to achieve a truly harmonious integration of light, music, and scene. For example, the music might not match the ambient lighting, or the ambient lighting might not match the real-world scene, resulting in overly bright or dull lighting. While music can significantly improve the user experience, music that doesn't match the real-world scene or the user's emotions can worsen the experience and render the ambient lighting merely a response to the music's rhythm, rather than creating a pleasant atmosphere.

[0004] On the other hand, while driving, the screen of the in-vehicle unit (HU) is within the driver's line of sight. However, when ambient lighting and music are displayed in the vehicle, the HU screen does not participate in the lighting display or only displays the background color, thus failing to provide a better user experience using the HU's display. Summary of the Invention

[0005] The purpose of this invention is to provide a system and method for controlling lights using music, as well as a vehicle infotainment system, to overcome at least one of the aforementioned problems in the prior art.

[0006] According to one aspect of the present invention, a system for controlling light using music is provided, the system comprising: a music analysis unit configured to analyze the rhythm and content of music; and a light control unit configured to dynamically control at least one of the color, brightness, and the rhythm of color and brightness changes of light emitted by a light-emitting component based on the analysis of the rhythm and content of the music; wherein the light control unit includes a color encoder configured to generate a color code based on at least one of the rhythm and content of the music; and the light control unit controls the color of the light emitted by the light-emitting component based on the color code generated by the color encoder.

[0007] According to another aspect of the present invention, a method for controlling light using music is provided, comprising: analyzing the rhythm and content of the music; and dynamically controlling at least one of the color, brightness, and the rhythm of color and brightness changes of light emitted by a light-emitting component based on the analysis of the rhythm and content of the music; wherein the control step comprises: generating a color code based on at least one of the rhythm and content of the music; and controlling the color of the light emitted by the light-emitting component based on the color code.

[0008] According to another aspect of the present invention, there is provided an apparatus for providing scene music and ambient lighting, the apparatus comprising: a plurality of light-emitting components; a music playback device; and a system for controlling lighting using music as described above.

[0009] According to another aspect of the invention, a vehicle is provided that includes the device for providing ambient music and ambient lighting as described above.

[0010] According to another aspect of the present invention, a vehicle infotainment system is provided, the vehicle infotainment system comprising: a memory storing computer-executable instructions; and at least one processor; wherein the computer-executable instructions, when executed by the processor, cause the vehicle infotainment system to perform the method of controlling lights using music as described above, to dynamically control at least one of the color, brightness, and rhythm of color and brightness changes of a plurality of light-emitting components arranged in the interior space of the vehicle and the display screen of the vehicle infotainment system.

[0011] Embodiments of the present invention can achieve better integration between lighting, music and scenes to improve the user experience. Attached Figure Description

[0012] Figure 1 A block diagram illustrating the structure of a system for controlling lights using music, according to an embodiment of the present invention, is shown.

[0013] Figure 2 A flowchart illustrating a method for controlling lights using music according to an embodiment of the present invention is shown.

[0014] Figure 3 A structural block diagram of a device for providing background music and ambient lighting according to an embodiment of the present invention is shown.

[0015] Figure 4 The diagram illustrates the overall process flow for processing music signals and controlling lights in an embodiment.

[0016] Figure 5 A schematic diagram showing the light distribution of the light-emitting components and display screen in the embodiment is shown.

[0017] Figure 6A schematic diagram of a vehicle according to an embodiment of the present invention is shown. Detailed Implementation

[0018] The embodiments of the present invention are further described below with reference to the accompanying drawings.

[0019] Figure 1 This diagram illustrates a structural block diagram of a system 100 for controlling lighting using music, according to an embodiment of the present invention. The system 100 includes: a music analysis unit 120 configured to analyze the rhythm and content of music; and a lighting control unit 130 configured to dynamically control at least one of the following: the color, brightness, and the rhythm of color and brightness changes of the light emitted by a light-emitting component 140, based on the analysis of the rhythm and content of the music; the lighting control unit 130 includes a color encoder 131 configured to generate a color code based on at least one of the medium and content of the music; and the lighting control unit 130 controls the color of the light emitted by the light-emitting component 140 based on the color code generated by the color encoder 131. In this embodiment, the lighting control unit 130 can generate dynamically changing lighting control signals based on the analysis of the rhythm and content of the music, and correspondingly control at least one of the following: the color, brightness, and the rhythm of color and brightness changes, to achieve deep integration of lighting and music. The system 100 uses color encoding generated by a color encoder to control the light emission of the light-emitting components, and can generate dynamic changes and rich color schemes that are fully integrated with the rhythm and content of the music, thereby providing users with an improved audio-visual experience. In some embodiments, the system 100 also includes a music selection unit 110, configured to select music to be played based on a scenario, so that the music, lighting, and scenario are coordinated; wherein the scenario includes at least one of user preferences, weather, time, user status, user's geographical location, and music volume. The music selection unit 110 can be an input unit that allows the user to select music to be played based on subjective factors such as their preferences or status, so that the music and corresponding lighting meet the user's preferences or are coordinated with the user's current status. For example, the music selection unit 110 can play music that the user likes, such as soothing music, energetic music, love music, movie music, or children's music; or, when the user is fatigued but needs to be refreshed (e.g., while driving), it can play music that can uplift their spirits; or when the user (e.g., a child) is fatigued and wants to sleep, it can play soothing and calming music. The music selection unit 110 can also automatically select the music to play based on objective factors such as weather, time, and location, so that the music and corresponding lighting are coordinated with these factors. For example, it can play music about spring in springtime, music about rain when it's raining, music about holidays (e.g., Chinese New Year), music about the sea when the user is at the beach, and music about the forest when the user is in a forest.

[0020] In some embodiments, the user state may include at least one of the user's physical fatigue state, psychological state, and movement speed. The music selection unit 110 may automatically select music based on at least one of these user states, for example, selecting fast-paced, energetic music when the user is excited or moving at high speed (e.g., in a vehicle traveling at high speed). The user state may be automatically detected using appropriate sensors, or it may be set by the user themselves via an input device.

[0021] In some embodiments, the music selection unit 110 includes a music mode selection unit 111, which is configured for a user to select a music mode that matches their preferences. The music mode indicates the style of the music to be played. The music selection unit 111 automatically selects the music to be played based on the selected music mode. In one example, the music modes selectable on the music mode selection unit 111 include romantic mode, conversation mode, pregnant woman mode, children's mode, blessing mode, and smart mode. Romantic mode, conversation mode, blessing mode, pregnant woman mode, and children's mode respectively represent selected music themes that are romantic (e.g., lyrical and imaginative), conversational (wordless light music), blessing (joyful and festive), and suitable for pregnant women (e.g., gentle and warm, melodious and beautiful) and children (e.g., light and lively). Smart mode can automatically select the music to be played based on objective factors or historical data. The music modes selectable on the music mode selection unit 111 may also include other modes, and the present invention is not limited to these modes. The music selection unit 110 may also include units that select music in other ways, and is not limited to the music mode selection unit 111 of the above embodiments.

[0022] In some embodiments, the music analysis unit 120 includes: a music rhythm analysis module 121 configured to analyze the rhythm of the music; and a music content analysis module 122 configured to analyze the content of the music. During music playback, the music rhythm changes, and the music continuously presents rich content (e.g., expressing multiple different themes or emotions, depicting multiple different scenes, lyrics accompanying the music). Accordingly, the lighting control unit 130 can dynamically control at least one of the color, brightness, and rhythmic changes of the light emitted by the light-emitting component 140 based on the current rhythm of the music and the currently presented content. In one embodiment, the lighting control unit 130 dynamically controls the rhythmic changes of the light emitted by the light-emitting component 140 based on the music rhythm analysis by the music rhythm analysis module 121, so that the rhythmic changes of the light's color and brightness are consistent with the music rhythm; the lighting control unit 130 also dynamically controls at least one of the color and brightness of the light emitted by the light-emitting component 140 based on the music content analysis by the music content analysis module 122, so that the color and / or brightness of the light are coordinated with the music content.

[0023] Musical content may include at least one of the following: musical style, setting, and lyrics. However, this invention is not limited to these; musical content may also include other aspects of the musical meaning. Musical content can reflect the emotions and moods expressed in the music. Similarly, according to color psychology theory, the colors and brightness displayed by light can also express emotions and moods. For example, red expresses warmth, passion, sensuality, praise, and excitement; orange expresses friendliness, positivity, ambition, and vitality; yellow expresses youth, optimism, and warmth; green expresses nature, relaxation, peace, and life; blue expresses sadness, comfort, and tranquility; purple expresses romance, fantasy, and elegance; pink expresses gentleness, sweetness, romance, and ease; and white expresses purity, holiness, kindness, trust, and openness. In embodiments of this invention, ensuring that the color and / or brightness of the light are consistent with the musical content means ensuring that the emotions and moods expressed by the light are consistent with those expressed by the music.

[0024] On the other hand, the rhythm of music itself can also express emotions and moods. For example, a soothing rhythm can express calmness, dreaminess, relaxation, and nature, corresponding to the emotions and moods expressed by colors such as blue, purple, and green. Fast-paced music can express joy, optimism, ease, positivity, and enthusiasm, corresponding to the emotions and moods expressed by colors such as yellow, pink, orange, and red. In another embodiment, the lighting control unit 130 can dynamically control the color and brightness of the light emitted by the light-emitting component 140 based on the analysis of music rhythm by the music rhythm analysis module 121 and the analysis of music content by the music content analysis module 122, and can also dynamically control the rhythm of color and brightness changes based on the analysis of music rhythm by the music rhythm analysis module 121.

[0025] In some embodiments, system 100 may further include a music theme dictionary containing various themes expressed by musical rhythms and content, and their corresponding colors. For example, the music theme dictionary may include a mood color palette based on color psychology. The music analysis unit 120 analyzes the rhythm and content of the music to determine the emotions or moods expressed by the rhythm and content; the lighting control unit 130 can select colors corresponding to these emotions or moods from the mood color palette, and simultaneously control the light-emitting component 140 to present the corresponding colors in the emitted light. The mood color palette may include various emotions, moods, and their corresponding colors as exemplified above.

[0026] In some embodiments, the music content analysis module 122 is further configured to analyze at least one of the music's style, scene, and lyrics, extract features related to at least one of the music's style, scene, and lyrics, and classify at least one of the music's style, scene, and lyrics using machine learning algorithms; the lighting control unit 130 is further configured to dynamically control the color and brightness of the light based at least partially on the classification of at least one of the music's style, scene, and lyrics, and to control the rhythm of color and brightness changes of the light based at least partially on the analysis of the scene and music rhythm analysis module 121; wherein, the scene includes at least one of user preferences, weather, time, user status, user's geographical location, and music volume. In embodiments, the music's style, scene, and lyrics can be classified into corresponding emotions or moods, so that the lighting control unit 130 can control the light-emitting component 140 to emit light with colors and brightness corresponding to the classified emotions or moods based on the current music style, scene, or lyrics. The scene can be set by the user or automatically. A sensor for detecting user status can be provided in the system 100 to detect user status.

[0027] In the process of classifying music content, machine learning algorithms can be used to build models and perform feature extraction and training to obtain a continuously improving classifier model, which can be used to classify music content according to its expressed emotions and artistic conception. For example, when playing the Carpenters' song "Yesterday Once More," the background music and lyrics are presented: "When I was young, I listened to the radio. Waiting for my favorite songs. When they played, I sang along. It made me happy...". In the process of lyric classification, the lyrics in the song music are segmented (e.g., for the lyrics in "Yesterday Once More," they can be divided into four sections, each expressing a different emotion and artistic conception), contextual information association analysis is performed, keywords in the lyrics are extracted, and features are extracted and analyzed to classify the emotions or artistic conceptions expressed by the segmented lyrics and add corresponding classification labels. The lighting control unit 130 can control the light-emitting component 140 to emit light with colors and brightness corresponding to the classified emotions or artistic conceptions based on the current classification labels of the lyric segments.

[0028] Musical style refers to the representative and unique characteristics of a musical work as a whole, i.e., the genre of music. Different musical styles can express different emotions or moods, and the theme or name of a piece of music can also reflect the emotions or moods it expresses. For example, light music has a simple structure, brisk rhythm, and beautiful melody, creating a warm and romantic atmosphere with a relaxing quality; classical music is profound, classic, and elegant; country music has simple melodies, a steady rhythm, and a narrative and rustic feel; rock music can express various emotions and moods such as fervor, passion, and rebellion. The music content analysis module 122 can analyze fragments of music, extract features related to the music's style (including the theme or name), determine the type of music and the emotions and moods it expresses, and then use machine learning algorithms to classify the music's style; the lighting control unit 130 can further dynamically control the color and brightness of the lights, as well as the rhythm of color and brightness changes, based on the classification of music style. In another example, the music rhythm analysis module 121 and the music content analysis module 122 can be combined to analyze the style of music in order to classify the music or music fragments.

[0029] A musical scene refers to the scene described by music, such as natural scenery (e.g., the sea, rivers, grasslands), festival celebrations, weddings, bars, war, etc. A piece of music can describe several different scenes. The music content analysis module 122 analyzes the musical scenes (e.g., based on the theme or name of the music, elements in the musical fragments), extracts features related to the musical scenes, compares them with classic musical scene elements, and classifies the musical scenes using machine learning algorithms. The lighting control unit 130 further dynamically controls the color and brightness of the lights based on the classification of musical scenes, and controls the rhythm of color and brightness changes based at least partially on the analysis of the scene and music rhythm analysis module 121, so that users can experience the scene described by the music through music and lighting. For example, when playing a musical fragment related to the sea, the lights are blue; when playing a musical fragment related to grasslands, the lights are green; when playing a musical fragment related to festival celebrations, the lights are red and flash in rhythm with the music.

[0030] In some embodiments, the light-emitting component 140 includes a plurality of light-emitting components; the lighting control unit 130 further includes a color mixer 132; the color encoder 131 is configured to generate a color code based on the emotion or mood expressed by at least one of the rhythm, style, scene and lyrics of the music analyzed by the music analysis unit 120; the color mixer 132 is configured to generate a mixed color scheme expressing multiple emotions or moods based on the generated color code; and the lighting control unit 130 is configured to control the color of the light emitted by the plurality of light-emitting components 140 based on the generated mixed color scheme.

[0031] Multiple light-emitting components 140 can be arranged in a specific pattern. For example, multiple light tubes can be arranged along the two side doors, front windows, and rear windows inside the vehicle to form a rectangular ambient lighting structure. In this ambient lighting structure, the lighting control unit 130 can control each light-emitting component 140 to present a mixed color scheme that expresses various emotions or moods in the music by combining the light emission colors of each light-emitting component 140 in a temporal and spatial manner.

[0032] In some embodiments, the color mixer 132 generates a mixed color scheme that includes at least one of the following: a combination scheme of multiple colors in different spaces, a switching scheme of multiple colors at different time periods, and a color mixing scheme of multiple colors. The combination scheme of multiple colors in different spaces can involve simultaneously presenting multiple colors within the overall space of the ambient lighting structure to express multiple emotions or moods; for example, different light colors can be used to represent the emotions or moods expressed by the main melody and secondary melody appearing simultaneously in music. The color representing the main melody can occupy a larger luminous space (i.e., a large luminous color block) in the ambient lighting structure, while the color representing the secondary melody can occupy a smaller luminous space (i.e., a small luminous color block). These color blocks presented by multiple light-emitting components 140 can form the lighting composition of the mixed color scheme. Through the control of the lighting control unit 130, a lighting effect can also be presented where small luminous color blocks flow or flicker within large luminous color blocks. A multi-color switching scheme at different times can be implemented within the overall space of the ambient lighting structure, presenting different colors at different times to express various emotions or moods. For example, different light colors can be used to represent the emotions or moods expressed by the main melody, secondary melody, lyrics, etc., appearing in chronological order in music. As the main melody, secondary melody, and lyrics appear in turn at different times, the light colors are switched at the corresponding times to present the light colors expressing the corresponding emotions or moods. A multi-color mixing scheme can be implemented within the overall space of the ambient lighting structure, presenting mixed colors that express a combination of multiple emotions or moods; for example, blue and red mix to form purple, red and yellow mix to form orange, and yellow and blue mix to form green. The above-mentioned color mixing schemes can be used in combination. However, this invention is not limited to the above three color mixing schemes, and other color mixing schemes can also be used.

[0033] In some embodiments, the color encoder 131 includes a first color encoder configured to generate a corresponding first color code based on at least one of the following classifications: music style, scene, and lyrics; wherein the first color code indicates the luminous color of each of the luminous components 140, and / or the overall pattern formed by the luminous colors of multiple luminous components 140; the lighting control unit 130 is further configured to generate a control signal based on the first color code for controlling the luminous colors of the multiple luminous components 140. Ambient lighting is typically provided by multiple luminous components arranged in a specific configuration. The first color code generated by the first color encoder can not only indicate the luminous color of each luminous component, matching the luminous color of the luminous components, but also indicate the overall pattern formed by the light emitted by each of the multiple luminous components 140 at any given moment, thereby allowing the emotion or mood expressed by the overall luminous pattern of the multiple luminous components 140 to blend and harmonize with the rhythm and / or content of the music. For example, the lighting control unit 130 can control the multiple luminous components 140 to produce the effect of blue blocks flowing among the multiple luminous components 140 to create a seascape scene; or to produce large areas of green to create a forest scene.

[0034] In another embodiment, the color mixer 132 is configured to generate a multi-color scheme-based mixed scene effect based on multiple color codes generated by the first color encoder and the arrangement scheme of the set light-emitting components, to obtain a multi-light strip mixing and multi-color soft overall light emission color scheme that blends with music and scene, so as to achieve the corresponding overall sound and light experience effect.

[0035] In some embodiments, the lighting control unit 130 controls the emotion or mood expressed by the light emission color of the light-emitting component 140 to be coordinated with the emotion or mood expressed by at least one of the music's style, scene, and lyrics. This enables the fusion of music and lighting.

[0036] In other embodiments, the color encoder 131 includes a second color encoder; the second color encoder includes a tag color encoder and a pattern color encoder; the tag color encoder is configured to generate a second color code based on a classification tag of the music content (such as at least one of music style, scene, and lyrics), and the pattern color encoder is configured to generate a third color code based on the scene and the selected music pattern; the second color encoder is further configured to generate a combined color code based on the second and third color codes; the combined color code indicates the emission color of each of the plurality of light-emitting components 140, and / or the overall color scheme composed of the emission colors of the plurality of light-emitting components 140; wherein, the classification tag of the music content indicates the classification of the music content; the scene includes at least one of user preference, weather, time, user status, user's geographical location, and music volume; the music pattern indicates the style of music selected by the user; the lighting control unit 130 is further configured to generate a control signal for controlling the emission colors of the plurality of light-emitting components 140 based on the overall color scheme. In this embodiment, the music to be played can be automatically selected based on the scene and / or the selected music pattern. This allows for the generation of comprehensive color codes based on environmental characteristics (such as weather, time, and the user's geographical location), music features (such as music content), and user factors (such as user preferences, user's psychological or physical state, and the music mode selected by the user). Furthermore, it generates corresponding overall color schemes to control the lighting, thereby integrating environmental characteristics and user factors into the music and lighting, providing users with a more perfect and comfortable audio-visual experience.

[0037] The overall color scheme can integrate the emotions and moods that are appropriate to the environmental characteristics, the emotions and moods expressed in the music, and the emotions and moods required by the user, and present them in a colorful manner across multiple luminous components. The overall color scheme can be dynamic and change with the rhythm and cadence of the music to present a rhythmic sense of color change.

[0038] In the lighting control unit 130, the color encoder 131 may include at least one of a first color encoder and a second color encoder, as needed. The first color encoder and the second color encoder may be implemented by hardware or software, or a combination thereof.

[0039] In embodiments of the present invention, by utilizing a color encoder 131 and a color mixer 132, and controlling multiple corresponding light-emitting components (including a display screen) to present integrated ambient lighting, intelligent dynamic lighting based on multi-dimensional factors (such as music, weather, user preferences, user status, etc.) can be realized, providing a lighting presentation scheme with richer colors, more varied and harmonious changes with music, and more flexible lighting, achieving deep integration of music, scene, lighting and other factors, and providing users with a better audio-visual experience.

[0040] In some embodiments, system 100 is included in a vehicle infotainment system; the light-emitting components include multiple light-emitting components arranged in the interior space of the vehicle, and the display screen of the infotainment system. The lighting control unit 130 can control the color and brightness of the light emitted by the multiple light-emitting components and the display screen of the infotainment system. The display screen of the infotainment system can also display any of the following in sync with music: patterns, images, or videos, to provide an immersive experience for users in the vehicle.

[0041] This invention is not limited to applying System 100 in vehicles, but can also be applied in other situations, such as music and lighting control at various art performances such as musical fountains, concerts, and cinemas.

[0042] Figure 2 A flowchart illustrating a method for controlling lights using music, according to an embodiment of the present invention, is shown. Figure 2 As shown, method 200 includes: analyzing the rhythm and content of music (step 220); and dynamically controlling at least one of the color, brightness, and rhythm of color and brightness changes of the light emitted by the light-emitting component based on the analysis of the rhythm and content of the music (step 230); wherein, the control step 230 includes: generating a color code based on at least one of the rhythm and content of the music; and controlling the color of the light emitted by the light-emitting component based on the color code.

[0043] In some embodiments, method 200 further includes: selecting music to be played based on a context so that the music, lighting, and context are coordinated (step 210); wherein the context includes at least one of user preferences, weather, time, user status, user's geographical location, and music volume. User status may include at least one of user's physical fatigue, user's psychological state, and user's movement speed.

[0044] In some embodiments, the selection step 210 includes: selecting a music mode that matches the user's preferences, the music mode indicating the style of music to be played; and automatically selecting music to be played based on the selected music mode.

[0045] In some embodiments, analysis step 220 includes: analyzing at least one of the music style, scene, and lyrics; extracting features related to at least one of the music style, scene, and lyrics; and classifying at least one of the music style, scene, and lyrics using a machine learning algorithm; control step 230 includes: dynamically controlling the color and brightness of the lights based at least in part on the classification of at least one of the music style, scene, and lyrics; and dynamically controlling the rhythm of the color and brightness changes of the lights based at least in part on the scene and the rhythm of the music; wherein the scene includes at least one of user preferences, weather, time, user status, user's geographical location, and music volume.

[0046] In some embodiments, the light-emitting components include multiple light-emitting components; the control step 230 further includes: generating a color code based on the emotion or mood expressed by at least one of the rhythm, style, scene, and lyrics of the music analyzed in the analysis step 220; generating a mixed color scheme expressing multiple emotions or moods based on the generated color code; and controlling the color of the light emitted by the multiple light-emitting components based on the mixed color scheme. The mixed color scheme may include at least one of the following: a combination scheme of multiple colors in different spaces, a switching scheme of multiple colors in different time periods, and a color mixing scheme of multiple colors.

[0047] In some embodiments, the control step 230 further includes: generating a corresponding first color code based on at least one of the music's style, scene, and lyrics; wherein the first color code indicates the luminous color of each luminous component and / or the overall pattern formed by the luminous colors of multiple luminous components; and generating a control signal based on the first color code for controlling the luminous colors of the multiple luminous components.

[0048] In some embodiments, during control step 230, the emotion expressed by the luminous color of the luminous component is controlled to be consistent with the emotion expressed by at least one of the music's style, scene, and lyrics.

[0049] In some embodiments, control step 230 includes: generating a second color code based on the category tag of the music content; generating a third color code based on the context and the selected music mode; generating a combined color code based on the second and third color codes; the combined color code indicating the emission color of each of the plurality of light-emitting components, and / or the overall color scheme composed of the emission colors of the plurality of light-emitting components; and generating a control signal for controlling the emission colors of the plurality of light-emitting components based on the overall color scheme; the category tag of the music content indicates the category of the music content; the context includes at least one of user preferences, weather, time, user status, user's geographical location, and music volume; the music mode indicates the style of music selected by the user.

[0050] Those skilled in the art will understand that Figure 1 The system 100 shown and its various embodiments can all be used for implementation. Figure 2 The method 200 shown can also be implemented using method 200 and various signal processing and control systems. Figure 1 The system 100 shown and its various implementations are illustrated.

[0051] Figure 3 This diagram illustrates a structural block diagram of a device for providing ambient music and lighting according to an embodiment of the present invention. As shown, the device 300 includes: a music playback device 310; a system 320 for controlling lighting using music as described in the above embodiments; and multiple light-emitting components 330. The system 320 can employ the system 100 described in the above embodiments. The device 300 can use music played by the music playback device 310 to control the light emitted by the light-emitting components 330, achieving a fusion of music and light to improve the user experience. In other embodiments, the device 300 may further include one or more displays for displaying images and / or videos that conform to the rhythm and content of the music. Furthermore, by integrating the displays with ambient lighting, intelligent dynamic lighting based on multi-dimensional scenarios (such as music, weather, user preferences, etc.) can be further realized. For example, two large displays could be placed at the left and right front of a large concert venue, or a single display could be placed directly in front of the venue. The display screen can also form an interactive mode with the ambient lighting composed of light-emitting components 330. The colored blocks can flow and pass, diffuse or converge in multiple directions between multiple light-emitting components 330 and the display screen to create a dynamic lighting effect that matches the rhythm of the music and the scene.

[0052] Figure 4 A schematic diagram illustrating the overall method flow for processing music signals and controlling lights in an embodiment is shown. Figure 4As shown, method 400 includes steps such as music rhythm analysis, music content analysis, music mode selection, color encoding, spatial link encoding of rhythm and melody, and lighting control. Specifically, in the music rhythm analysis step, the audio source signal waveform file of the music signal is first received and preprocessed (sub-step 410), for example, a Hamming window is added to the 23ms music signal, and the data in the Hamming window is subjected to Fast Fourier Transform (FFT) to obtain the power spectrum; fast audio source segmentation is performed (sub-step 412); audio editing is performed (sub-step 414); short-time Fourier transform (STFT) / fast Fourier transform (FFT) is performed on the edited audio signal to obtain the spectrogram (sub-step 416); music signal periodicity analysis is performed (sub-step 418) to obtain the periodic rhythm and melody information of the music signal. In addition, a scene-based rhythm enhancement module can be set up to further combine scene information (such as weather information, user status, etc.) with the obtained periodic rhythm and melody information of the music signal to achieve further integration of rhythm and melody with the scene. In the process of analyzing music rhythm, this invention is not limited to the above sub-steps, and other methods can also be used to analyze music rhythm.

[0053] In the music content analysis step, lyrics are extracted from the song (sub-step 420); Natural Language Processing (NLP) technology is used to classify the lyrics and add classification labels (sub-step 422), including extracting the theme words of the lyrics, semantic environment analysis and sentiment analysis of the lyrics and their classification, and training the lyrics classifier (e.g., feature extraction, machine learning algorithm processing, prediction processing, classifier model processing, adding lyrics classification labels, etc.). The music mode selection step includes: automatically selecting the music mode based on factors such as weather, vehicle speed, or music volume, selecting the music mode based on user preferences, and intelligently selecting the mode (sub-step 430); the user can choose any mode from the available music modes (including romantic mode, conversation mode, blessing mode, pregnant woman mode, children's mode, etc.) to start music playback, and the music playback device will automatically play music that matches the selected music mode (sub-step 432).

[0054] Based on the lyric classification tags obtained in sub-step 422 and the music played in sub-step 432, a two-factor color encoder generates color codes corresponding to the emotions and moods expressed by the lyrics and music signals (step 440). The two-factor color encoder can be an implementation of the second color encoder as described above, which can include a tag color encoder and a pattern color encoder; the tag color encoder can perform tag color encoding on the lyrics according to the lyric classification tags, and the pattern color encoder can perform pattern color encoding on the music according to the weather, user movement speed, music volume, and the selected music mode; then, the music content tags are classified and merged through the combined color encoding of the tag color encoder and the pattern color encoder (step 450). Further, based on the periodic rhythm information of the music signal obtained in sub-step 418 and the music content tags after classification and merging in step 450, the two are combined and linked to realize the spatial link encoding of the rhythm of the music signal, thereby generating the overall luminous color and brightness distribution and its light change rhythm information, and correspondingly generating control signals to control the luminous emission of various luminous components and the display screen (such as the display screen of the vehicle system or the display screen in device 300). These generated control signals will be sent to the controller to control the light emission of each light-emitting component and the display of the screen (step 470).

[0055] In one example, the music content may simultaneously express multiple emotions and moods, such as primary and secondary emotions. In this case, multiple music content tags are generated, and correspondingly, the color encoder generates multiple color codes (e.g., corresponding to the emotions and moods expressed by the lyrics and musical signals). When the system 100 controls multiple light-emitting components to emit light, the light color (primary color) corresponding to the primary emotion is more than the light color corresponding to the secondary emotion, or the light color corresponding to the secondary emotion appears to flow or fluctuate within the primary color.

[0056] Embodiments of the present invention may also provide a vehicle infotainment system, comprising: a memory storing computer-executable instructions; and at least one processor; the computer-executable instructions, when executed by the processor, cause the vehicle infotainment system to perform the method of controlling lights using music as described in the above embodiments, so as to dynamically control at least one of the color, brightness, and rhythm of color and brightness changes of a plurality of light-emitting components arranged in the interior space of the vehicle and the display screen of the vehicle infotainment system.

[0057] Figure 5 A schematic diagram showing the light distribution of the light-emitting components and display screen in the embodiment is shown. Figure 3 The light-emitting components and display screen in the device 300 shown can be arranged according to... Figure 5The arrangement is as shown. In this embodiment, light-emitting components 510 and 520 can be positioned above the display screen 530, and light-emitting components 540 and 550 can be positioned on both sides of the display screen 530 to form a semi-surround effect on the user. The light-emitting components 510, 520, 540, and 550 can be located close to the front speaker, left channel speaker, and right channel speaker of the music playback device 310, so that the user can obtain an immersive audio-visual experience through the integration of music, light, and scene.

[0058] The present invention also provides a vehicle that includes the device for providing ambient music and ambient lighting as described in the above embodiments. Figure 6 A schematic diagram of a vehicle 600 according to an embodiment of the present invention is shown. Inside the vehicle 600, front light-emitting tubes 610 and 620, side light-emitting tubes 640 and 650 are respectively provided, and a display screen 630 of a vehicle infotainment system (such as a Head Unit) is provided near the steering wheel. Under the control of system 100, the light-emitting tubes 610, 620, 640, and 650, as well as the display screen 630, can emit lights that are synchronized with the rhythm and content of the music played by the vehicle infotainment system; the display screen 630 can also further display changing colors and patterns in a video format. Figure 6 The arrows indicate the direction in which color blocks (such as yellow blocks) flow within the LEDs 610, 620, 640, and 650. Through the individual LEDs and displays, and in conjunction with the stereo multi-channel audio system inside the vehicle 600, an immersive audio-visual experience can be provided to the users inside the vehicle 600 as a whole.

[0059] The above description is merely a preferred embodiment of the present invention and does not constitute any limitation on the present invention. Those skilled in the art, after understanding the content and principles of the present invention, can make various modifications and variations in form and detail without departing from the principles of the present invention, and such modifications and variations based on the principles of the present invention are still within the protection scope defined by the claims of this patent.

Claims

1. A system for controlling lights using music, comprising: The music selection unit is configured to select music to play based on a context, so that the music, the lighting, and the context are coordinated, wherein the context includes at least one of user preferences, weather, time, user status, user's geographical location, and music volume; wherein the user status includes at least one of user's physical fatigue state, user's psychological state, and user's movement speed. A music analysis unit is configured to analyze the rhythm and content of music, wherein the content of the music includes at least one of the music's style, setting, and lyrics; and A lighting control unit is configured to dynamically control at least one of the following: the color, brightness, and the rhythm of color and brightness changes of the light emitted by the light-emitting component, based on an analysis of the rhythm and content of the music. The lighting control unit includes a color encoder configured to generate a color code based on at least one of the rhythm and content of the music; the lighting control unit controls the color of the light emitted by the light-emitting component based on the color code generated by the color encoder. The color encoder includes a second color encoder, which includes a label color encoder and a pattern color encoder. The label color encoder is configured to generate a second color code based on the rhythm and / or content classification labels of the music; the pattern color encoder is configured to generate a third color code based on the scenario and the selected music pattern; the second color encoder is further configured to generate a combined color code based on the second color code and the third color code; wherein the combined color code indicates the emission color of each of the plurality of emission components, and / or the overall color scheme composed of the emission colors of the plurality of emission components; The category tags for the music content indicate the category of the music content; the music mode indicates the style of music selected by the user. The lighting control unit is further configured to generate a control signal for controlling the luminous color of the plurality of light-emitting components based on the overall color scheme.

2. The system according to claim 1, wherein, The music selection unit includes a music mode selection unit, which is configured for a user to select a music mode that matches the user's preferences, wherein the music mode indicates the style of the music to be played. The music selection unit automatically selects the music to play based on the selected music mode.

3. The system according to claim 1, wherein, The music analysis unit includes: A music rhythm analysis module is configured to analyze the rhythm of the music; and The music content analysis module is configured to analyze the content of the music.

4. The system according to claim 3, wherein, The music content analysis module is further configured to analyze at least one of the music's style, scene, and lyrics, extract features related to at least one of the music's style, scene, and lyrics, and classify at least one of the music's style, scene, and lyrics using machine learning algorithms. The lighting control unit is further configured to dynamically control the color and brightness of the lights based at least in part on the classification of at least one of the music's style, scene, and lyrics, and to dynamically control the rhythm of the light's color and brightness changes based at least in part on the scene and the analysis of the rhythm analysis module.

5. The system according to claim 1, wherein, The light-emitting component includes multiple light-emitting components; the light control unit also includes a color mixer; The color encoder is configured to generate the color code based on the emotion or mood expressed by at least one of the rhythm, style, scene, and lyrics of the music analyzed by the music analysis unit; the color mixer is configured to generate a mixed color scheme expressing multiple emotions or moods based on the generated color code; and the lighting control unit is configured to control the color of the light emitted by the multiple light-emitting components based on the mixed color scheme.

6. The system according to claim 5, wherein, The color mixing scheme includes at least one of the following: a combination scheme of multiple colors in different spaces, a switching scheme of multiple colors in different time periods, and a color mixing scheme of multiple colors.

7. The system according to claim 1, wherein, The color encoder includes a first color encoder configured to generate a corresponding first color code based on a classification of at least one of the rhythm, style, scene, and lyrics of the music; wherein the first color code indicates the luminous color of each of the luminous components and / or the overall pattern formed by the luminous colors of multiple luminous components. The lighting control unit is further configured to generate a control signal based on the first color code for controlling the luminous color of the plurality of luminous components.

8. The system according to claim 7, wherein, The lighting control unit controls the emotion or mood expressed by the light-emitting color of the light-emitting component to be in harmony with the emotion or mood expressed by at least one of the style, scene and lyrics of the music.

9. The system according to any one of claims 1 to 8, wherein, The system is included in the vehicle's infotainment system; wherein the components emitting the light include multiple light-emitting components arranged in the interior space of the vehicle, and the display screen of the infotainment system.

10. A method for controlling lights using music, comprising: The music to be played is selected based on the context so that the music, the lighting and the context are coordinated. The context includes at least one of user preferences, weather, time, user status, user's geographical location and music volume. The user status includes at least one of user's physical fatigue, user's psychological state and user's movement speed. Analyze the rhythm and content of the music, wherein the content of the music includes at least one of the following: style, setting, and lyrics; and Based on the analysis of the rhythm and content of the music, at least one of the following is dynamically controlled: the color, brightness, and the rhythm of color and brightness changes of the light emitted by the light-emitting component. The control steps include: generating a color code based on at least one of the rhythm and content of the music; and controlling the color of the light emitted by the light-emitting component based on the color code. The control steps include: A second color code is generated based on the rhythm and / or content classification tags of the music; A third color code is generated based on the scenario and the selected music mode; A combined color code is generated based on the second color code and the third color code; wherein the combined color code indicates the emission color of each of the plurality of light-emitting components, and / or the overall color scheme composed of the emission colors of the plurality of light-emitting components; and Based on the overall color scheme, a control signal is generated to control the luminous color of the multiple luminous components; The category tags of the music content indicate the category of the music content; the music mode indicates the style of music selected by the user.

11. The method of claim 10, wherein, The selection step includes: selecting a music mode that matches the user's preference, wherein the music mode indicates the style of music to be played; and automatically selecting music to be played based on the selected music mode.

12. The method of claim 10, wherein, The analysis steps include: analyzing at least one of the music's style, scene, and lyrics; extracting features related to at least one of the music's style, scene, and lyrics; and classifying at least one of the music's style, scene, and lyrics using a machine learning algorithm. The control steps include: dynamically controlling the color and brightness of the light based at least in part on the classification of at least one of the music's style, scene, and lyrics; and dynamically controlling the rhythm of the color and brightness changes of the light based at least in part on the scene and the rhythm of the music.

13. The method of claim 10, wherein, The light-emitting component includes multiple light-emitting components; the control steps further include: The color code is generated based on the emotion or mood expressed by at least one of the rhythm, style, scene, and lyrics of the music analyzed in the analysis steps. Based on the generated color codes, a mixed color scheme expressing various emotions or moods is generated; and The color of the light emitted by the multiple light-emitting components is controlled based on the aforementioned color mixing scheme.

14. The method of claim 13, wherein, The color mixing scheme includes at least one of the following: a combination scheme of multiple colors in different spaces, a switching scheme of multiple colors in different time periods, and a color mixing scheme of multiple colors.

15. The method according to any one of claims 10 to 14, wherein, The control steps further include: A corresponding first color code is generated based on at least one of the following classifications: rhythm, style, scene, and lyrics of the music; wherein the first color code indicates the luminous color of each luminous component and / or the overall pattern formed by the luminous colors of multiple luminous components; and Based on the first color encoding, a control signal is generated to control the luminous color of the plurality of luminous components.

16. The method of claim 13, wherein, In the control step, the emotion expressed by the luminous color of the luminous component is controlled to be consistent with the emotion expressed by at least one of the style, scene, and lyrics of the music.

17. A device for providing ambient music and ambient lighting, comprising: Multiple light-emitting components; Music playback device; as well as The system as described in any one of claims 1 to 9.

18. A vehicle infotainment system, comprising: Memory, which stores computer-executable instructions; as well as At least one processor; Wherein, the computer-executable instructions, when executed by the processor, cause the vehicle system to perform the method as described in any one of claims 10 to 16, to dynamically control at least one of the color, brightness, and color and brightness variation rhythm of light emitted by a plurality of light-emitting components arranged in the interior space of the vehicle and the display screen of the vehicle system.

19. A vehicle comprising the device as claimed in claim 17.

Images