Atmosphere lamp rhythm method, system and equipment and storage medium

Abstract

The invention discloses an atmosphere lamp rhythm method, system and device and a storage medium, and relates to the technical field of vehicle-mounted atmosphere lamps, and the method comprises the steps: obtaining an audio file; performing audio rhythm detection on the audio file to obtain a rhythm point weight; performing video color mixing degree detection on the audio file to obtain a picture mixing degree weight; and adjusting the rhythm effect of the vehicle-mounted atmosphere lamp according to the comparison result of the rhythm point weight and the image mixing degree weight. According to the scheme, by detecting the audio rhythm and the video color mixing degree and carrying out weight calculation and comparison, the rhythm of the atmosphere lamp is switched along with the audio and video rhythm, the cabin cinema mode experience can be more real, and the user experience is better.

Description

Technical Field

[0001] The present application relates to the technical field of vehicle-mounted ambient light technology, and in particular to an ambient light rhythm method, system, device and storage medium. Background Art

[0002] Currently, the in-car ambient lighting effects on the market are mainly based on audio rhythm, and the lighting can only be designed according to the rhythm of the music. However, in video playback scenarios such as cinema mode, this lighting effect cannot meet the needs and lacks the video rhythm function synchronized with the video content. Summary of the invention

[0003] The main purpose of this application is to provide an ambient light rhythm method, system, device and storage medium, aiming to solve the technical problem of how to enable the rhythm of the vehicle-mounted ambient light to switch freely between audio rhythm and video rhythm according to the video content being played.

[0004] To achieve the above purpose, the present application proposes an atmosphere light rhythm method, the method comprising:

[0005] Get the audio file;

[0006] Performing audio rhythm detection on the audio file to obtain rhythm point weights; and

[0007] Performing video color clutter detection on the audio file to obtain a picture clutter weight;

[0008] According to the comparison result of the rhythm point weight and the picture confusion weight, the rhythm effect of the vehicle atmosphere light is adjusted.

[0009] In one embodiment, the step of performing audio rhythm detection on the audio file and obtaining rhythm point weights includes:

[0010] Extracting a time slice of fixed length from the audio file according to a preset number of time slice sampling points and a sampling rate;

[0011] Calculate the amplitude difference and local mean between adjacent time slices;

[0012] generating a rhythm extreme point based on the amplitude difference and the local mean;

[0013] The rhythm point weight is calculated based on the total number of the rhythm extreme value points and the total number of the time slices.

[0014] In one embodiment, the step of calculating the amplitude difference and the local mean between adjacent time slices comprises:

[0015] Performing Fourier transform on the time slice to generate a corresponding frequency domain graph;

[0016] Acquire the amplitude corresponding to the time slice according to the frequency domain diagram;

[0017] According to the amplitude of the current time slice and the amplitude of the previous time slice, the amplitude difference is calculated;

[0018] Extracting amplitude differences that meet a preset value range, and using the amplitude differences as potential rhythm points;

[0019] According to the potential rhythm points, a local mean is calculated.

[0020] In one embodiment, the step of generating a rhythm extreme point based on the amplitude difference and the local mean value comprises:

[0021] comparing the amplitude difference with the local mean;

[0022] When the amplitude difference is greater than or equal to the local mean, the amplitude difference is determined to be a rhythm extreme point.

[0023] In one embodiment, the step of performing video color clutter detection on the audio file to obtain a picture clutter weight includes:

[0024] Extracting picture data corresponding to a preset duration from the audio file;

[0025] Performing cluster analysis on the picture data to generate color distribution corresponding to the picture data;

[0026] According to the color distribution, the picture confusion weight is calculated.

[0027] In one embodiment, the step of calculating the picture confusion weight according to the color distribution includes:

[0028] Determining whether a main color exists in the picture data by cluster analysis;

[0029] If yes, determine the color proportion of the main color in the color distribution;

[0030] If the color proportion exceeds a preset threshold, the total number of picture data whose picture purity exceeds the preset threshold within a preset time length is accumulated and counted;

[0031] The picture complexity weight is calculated according to the total number of picture data and the total number of picture frames.

[0032] In one embodiment, the step of adjusting the rhythmic effect of the vehicle ambient light according to the comparison result of the rhythm point weight and the picture clutter weight includes:

[0033] Calculating an actual difference between the rhythm point weight and the picture confusion weight;

[0034] If the rhythm point weight is less than or equal to the picture confusion weight, and the actual difference is less than or equal to the preset tolerance, the vehicle ambient light is kept at the audio rhythm;

[0035] If the rhythm point weight is less than or equal to the picture complexity weight, and the actual difference is greater than a preset tolerance, the vehicle atmosphere light is switched from the audio rhythm to the video rhythm.

[0036] In addition, to achieve the above purpose, the present application also proposes an atmosphere light rhythm system, the atmosphere light rhythm system comprising:

[0037] An acquisition module is used to acquire audio files;

[0038] A detection module, used to perform audio rhythm detection on the audio file to obtain rhythm point weights; and

[0039] The detection module is further used to perform video color clutter detection on the audio file to obtain a picture clutter weight;

[0040] The adjustment module is used to adjust the rhythmic effect of the vehicle atmosphere light according to the comparison result of the rhythm point weight and the picture confusion weight.

[0041] In addition, to achieve the above-mentioned purpose, the present application also proposes an atmosphere light rhythm device, which includes: a memory, a processor, and a computer program stored in the memory and executable on the processor, and the computer program is configured to implement the steps of the atmosphere light rhythm method described above.

[0042] In addition, to achieve the above-mentioned purpose, the present application also proposes a storage medium, which is a computer-readable storage medium, and a computer program is stored on the storage medium. When the computer program is executed by a processor, the steps of the atmosphere light rhythm method described above are implemented.

[0043] In addition, to achieve the above-mentioned purpose, the present application also provides a computer program product, which includes a computer program, and when the computer program is executed by a processor, the steps of the atmosphere light rhythm method described above are implemented.

[0044] The embodiment of the present application proposes an ambient light rhythm method, system, device and storage medium, the method comprising obtaining an audio file; performing audio rhythm detection on the audio file to obtain a rhythm point weight; and performing video color clutter detection on the audio file to obtain a picture clutter weight; and adjusting the rhythm effect of the vehicle-mounted ambient light according to the comparison result of the rhythm point weight and the picture clutter weight. The solution detects the audio rhythm and video color clutter, and performs weight calculation and comparison, so that the ambient light rhythm switches with the audio and video rhythm, which can make the cabin theater mode experience more realistic and the user experience better. BRIEF DESCRIPTION OF THE DRAWINGS

[0045] The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments consistent with the present application and, together with the description, serve to explain the principles of the present application.

[0046] In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, for ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative labor.

[0047] Figure 1 A schematic diagram of a flow chart provided for Embodiment 1 of the ambient light rhythm method of the present application;

[0048] Figure 2 A schematic diagram of a curve of converting a time domain signal provided in Example 1 of the present application into a frequency domain signal;

[0049] Figure 3 A schematic diagram of the system architecture of the ambient light rhythm method provided in an embodiment of the present application;

[0050] Figure 4 A brief flowchart of the atmosphere light rhythm method provided in an embodiment of the present application;

[0051] Figure 5 This is a schematic diagram of the module structure of the atmosphere light rhythm system of the embodiment of the present application;

[0052] Figure 6 Schematic diagram of the device structure of the hardware operating environment involved in the ambient light rhythmic method in the embodiment of the present application.

[0053] The purpose, features and advantages of this application will be further described in conjunction with the embodiments and with reference to the accompanying drawings. DETAILED DESCRIPTION

[0054] It should be understood that the specific embodiments described herein are only used to explain the technical solutions of the present application and are not used to limit the present application.

[0055] In order to better understand the technical solution of the present application, a detailed description will be given below in conjunction with the accompanying drawings and specific implementation methods.

[0056] The main solution of the embodiment of the present application is: obtain an audio file; perform audio rhythm detection on the audio file to obtain rhythm point weights; and perform video color clutter detection on the audio file to obtain picture clutter weights; and adjust the rhythmic effect of the vehicle ambient light based on the comparison result of the rhythm point weights and the picture clutter weights.

[0057] The current in-vehicle ambient lighting systems on the market mainly change the lighting effects by analyzing audio signals, which is the so-called "audio rhythm" mode. This mode can automatically adjust the light color, brightness and even flashing frequency according to the rhythm of the music, creating a visual enjoyment that matches the music for passengers.

[0058] However, current in-vehicle ambient lighting solutions are somewhat limited when faced with more complex usage scenarios, especially in cinema mode that supports video playback. Traditional lighting control methods based on audio signals cannot effectively capture the emotional color or key events in the video content, resulting in a disconnect between the lighting effects and the video content, thus affecting the overall viewing experience. In addition, videos usually contain a richer level of information than pure audio, including but not limited to changes in picture color, emotional changes, and the appearance of specific scenes (such as action scenes, romantic moments, etc.), which are difficult to accurately identify and reflect with existing technologies.

[0059] This application provides a solution that extracts the rhythm points in the music by analyzing the audio signal and calculates their weights to provide a basis for the subsequent lighting effect design. At the same time, by analyzing the color distribution of the video screen, the complexity weight of the screen is calculated to determine whether the screen is suitable for video rhythm effects. According to the comparison results of the rhythm point weight and the picture complexity weight, the rhythm effect of the vehicle atmosphere light is intelligently adjusted to ensure that the lighting effect matches the currently playing content, which not only provides a richer and more personalized user experience, but also improves the user experience.

[0060] It should be noted that the execution subject of this embodiment can be a computing service device with data processing, network communication and program running functions, such as a vehicle-mounted atmosphere light, a personal computer, a mobile phone, etc., or an electronic device capable of realizing the above functions, a vehicle-mounted infotainment system, etc. The following takes a vehicle-mounted atmosphere light as an example to illustrate this embodiment and the following embodiments.

[0061] Based on this, the embodiment of the present application provides an atmosphere light rhythm method, referring to Figure 1 , Figure 1 This is a flow chart of the first embodiment of the ambient light rhythmic method of the present application.

[0062] In this embodiment, the atmosphere light rhythmic method includes steps S10 to S40:

[0063] Step S10, obtaining an audio file;

[0064] Specifically, this embodiment provides a multi-mode rhythmic solution for cabin atmosphere lights. When the host is in normal working state, the user enters the atmosphere light selection interface and selects the atmosphere light rhythmic mode. Among them, there are three types of atmosphere light rhythmic modes: audio rhythm, intelligent rhythm, and video rhythm. This embodiment mainly uses intelligent rhythm as an example.

[0065] When the user selects the intelligent rhythm solution through the atmosphere light selection interface, the system needs to read the audio and video files from the cache, decode these audio and video files, and convert the video (RGB888 format) + audio (PCM format) format.

[0066] Step S20, performing audio rhythm detection on the audio file to obtain rhythm point weights;

[0067] It should be noted that an audio rhythm point refers to a specific time point in an audio signal at which the beat of the music or the sound of a percussion instrument is most obvious. The rhythm point weight indicates the importance of each identified rhythm point to the overall music rhythm.

[0068] It is worth noting that this embodiment mainly performs audio rhythm and tempo detection. The audio rhythm algorithm can be completed through OEM customization or integration of a third party. This example mainly illustrates the rhythm detection and subsequent weight calculation. The audio rhythm detection processing step can also include steps S21 to S24:

[0069] Step S21, extracting a time slice of a fixed length from the audio file according to a preset number of time slice sampling points and a sampling rate;

[0070] It should be noted that a time slice refers to a fixed-length audio data segment extracted from a continuous audio signal.

[0071] Specifically, before extracting a fixed-length time slice from an audio file, the sampling rate fs required for time slice processing needs to be set in advance, where the sampling rate fs is set to 48KHz (the commonly used CD is 44.1KHz, and the maximum in the cockpit environment is generally 48KH), that is, 48,000 sample points are collected per second. To ensure consistent frequency resolution, the number of sampling points per time slice is fixed to N=2048 (N can be configured and adjusted, and in order to facilitate FFT processing, the value of this embodiment is 211), then the frequency resolution △f=fs / N≈23.4Hz (the sampling rate is calculated at 48KHz).

[0072] Then, use t=N / fs to calculate the actual duration of each time slice, which is about 42.6ms here. Select a suitable T value according to actual needs, such as 3 seconds, 5 seconds or longer. Then, from the beginning of the audio file, extract a time slice every t time length until the entire T time length data is covered, and the total number of time slices is M=T / t.

[0073] Finally, all the extracted time slices are put into the sequence in order to obtain the audio slice digital signal time domain sequence F t ={F t (1), F t (2), F t (3), ..., F t (M)}.

[0074] Through the above steps, the continuous audio stream can be effectively converted into discrete time slices, which can be used to perform detailed rhythm detection later.

[0075] Step S22, calculating the amplitude difference and local mean between adjacent time slices;

[0076] It should be noted that the amplitude difference between time slices refers to the degree of change in volume (or energy) between two consecutive time slices. The local mean refers to the average amplitude of multiple time slices within a time window. This "window" can be composed of several consecutive time slices.

[0077] It is worth noting that in order to more accurately capture the key features in the audio signal, step S22 is performed to provide a solid foundation for achieving high-quality intelligent rhythmic effects.

[0078] In another feasible embodiment, step S22 further includes steps A1 to A5:

[0079] Step A1, performing Fourier transform on the time slice to generate a corresponding frequency domain graph;

[0080] Specifically, after obtaining the time domain sequence F of the audio slice digital signal t Then, the time domain signal F is transformed by Fourier transform (FFT, FastFourier Transform) t Convert to frequency domain signal F a , that is, perform FFT transformation on each time slice to generate a frequency domain sequence F a ={F a (1), F a (2), F a (3), ..., F a (M)}, where 1≤p≤M, M is the total number of time slices.

[0081] Step A2, obtaining the amplitude corresponding to the time slice according to the frequency domain diagram;

[0082] For details, please refer to Figure 2 , Figure 2 This is a curve diagram of the conversion of time domain signals into frequency domain signals. a Frequency domain sequence, any F a (p) is composed of N (N is the number of sampling points) imaginary array elements, each element is represented by z = a + bi (i is an imaginary number), then the frequency corresponding to the element in the spectrum is p*△f, and the amplitude is F a (p) Total energy (total amplitude) in the time slice The total energy sequence corresponding to all time slices can be obtained as A = {A(1), A(2), A(3), A(p) ... A(M)}, 1 ≤ P ≤ M. Where M is the total number of time slices, and A(p) represents the total energy (or total amplitude) of the pth time slice in the frequency domain.

[0083] Step A3, calculating the amplitude difference according to the amplitude of the current time slice and the amplitude of the previous time slice;

[0084] Specifically, from the amplitude sequence A={A(1), A(2), A(3), A(p)...A(M)} corresponding to all time slices obtained from step A2, the capacity value of the current time slice is set, that is, the amplitude of the current time slice is A(p-1), the amplitude of the previous time slice is A(P), and the amplitude difference between the current time slice and the previous time slice is set to D(j)=A(P)-A(p-1), where j is actually the index of p-1, that is, j=p-1.

[0085] Step A4, extracting amplitude differences that meet a preset value range, and using the amplitude differences as potential rhythm points;

[0086] It should be noted that a potential rhythm point refers to a key time point in an audio signal that is identified by analyzing the amplitude difference and may correspond to a music rhythm structure.

[0087] Specifically, when D(j)>0, it means that the energy in the time period from p-1 to p has increased, which is considered to be a potential rhythm point. All D(j) values ​​that satisfy D(j)>0 are extracted to generate a potential rhythm point sequence D, D={D(1), D(2), D(3), D(j)...D(X)}, 0<X≤M / 2, where X is the number of D(j) that meet the conditions, and X does not exceed M / 2, because D(j) is based on the difference of time slices, so there are at most M-1 differences.

[0088] Step A5: Calculate the local mean value according to the potential rhythm points.

[0089] Specifically, since the frequency of human voice reception is generally in the range of 20 to 20 kHz, most frequencies in daily life are mainly concentrated below 6000 Hz, and it is impossible to use the complete mean method for screening. Therefore, this embodiment uses local mean filtering.

[0090] The local mean is calculated as follows:

[0091]

[0092] Where y refers to the amplitude or energy value of the time slice, and the y value can be adjusted dynamically. When j-5<0, it starts from 0 by default.

[0093] Step S23, generating a rhythm extreme point based on the amplitude difference and the local mean;

[0094] It should be noted that the rhythm extreme point refers to an energy change point in the audio signal that is significantly higher or lower than its surrounding environment.

[0095] Specifically, for each potential rhythm point, check whether its amplitude D(j) is significantly higher or lower than the local mean V(j). If the amplitude difference is not only greater than zero but also significantly higher than the local mean, it is considered to be a rhythm extreme point, and an extreme point sequence E = {D(1), D(2), D(3), D(j) ... D(y)} (y < x) is generated.

[0096] Step S24: Calculate the rhythm point weight based on the total number of the rhythm extreme value points and the total number of the time slices.

[0097] Specifically, according to the total number of time slices M obtained from step S21 and the total number of rhythm points y obtained from step S23, the weight corresponding to the audio rhythm point in the duration T is Percent Audio =(y / M)*100%, the higher the weight value, the more suitable the judgment is for the audio rhythm.

[0098] Through the above steps, a weight can be assigned to each rhythm extreme point, which helps the system to more accurately identify and respond to important rhythm points in the music, thereby achieving a more natural and dynamic ambient light rhythm effect.

[0099] Step S30, performing video color clutter detection on the audio file to obtain a picture clutter weight;

[0100] It should be noted that video color clutter refers to the diversity and complexity of colors in the video image. It measures the uniformity and diversity of color distribution within a frame or a series of frames.

[0101] The image clutter weight is a value assigned based on the video color clutter detection result, which is used to characterize the complexity of the image relative to other images. High color clutter means that the image contains many different colors, and these colors are more dispersed; while low color clutter means that the colors in the image are relatively single or concentrated.

[0102] It is worth noting that in intelligent rhythm, in addition to detecting the audio rhythm and rhythm points in the audio file, the video content in the audio file also needs to be detected. Therefore, executing step S30 can effectively extract relevant video data from the audio file, and calculate the picture confusion weight by analyzing the picture color, thereby guiding the design and implementation of the video rhythm effect.

[0103] In a feasible embodiment, step S30 may further include steps S31 to S33:

[0104] Step S31, extracting picture data corresponding to a preset duration from the audio file;

[0105] Specifically, first, a preset time length T is set, and it is ensured that the time point of the selected video clip corresponds to a time point in the audio file to achieve audio and video synchronization.

[0106] Then, all frames within the duration T are read from the buffer in the audio file. By default, 30 frames per second are assumed, so for a video of T seconds, there will be a total number of frames of R = 30*T / 1s frames.

[0107] In addition, if the original video data is not in RGB888 format, it needs to be converted to RGB888 format for subsequent processing.

[0108] Step S32, performing cluster analysis on the picture data to generate color distribution corresponding to the picture data;

[0109] In this step, a clustering algorithm (such as K-Means algorithm) is used to perform clustering analysis on each frame of each picture data to obtain the color distribution of each frame.

[0110] Specifically, first set the number of color clusters K (for example, K = 7), which represents the number of color categories to be identified. Then perform cluster analysis on each frame to obtain the center point of each color category, that is, the main color, and the number of pixels in each color category. Then, count the proportion of the number of pixels in each color category to the total number of pixels to obtain the color band distribution. Finally, based on the color band distribution, record the color distribution of each frame.

[0111] Based on the above steps, cluster analysis is performed to identify the main color components and their proportions of each frame.

[0112] Step S33, calculating the image confusion weight according to the color distribution.

[0113] Specifically, first, a threshold of a single color proportion is set, such as 30%, and the threshold can be dynamically adjusted according to actual needs.

[0114] Then, check whether the color distribution of each frame exceeds the set threshold (such as 30%). If yes, the frame data is considered to be "pure color" or "low mixed". For all the frames judged as "pure color" within the T time length, the cumulative count is obtained to obtain the total count SUM.

[0115] Finally, according to the formula Percent video =(SUM / R)*100% calculates the picture complexity weight, where R is the total number of frames and SUM is the cumulative number of pure color pictures.

[0116] Among them, Percent video The higher the value, the more suitable the picture is for video rhythm effect during this period. video Close to 100%, it means that most of the pictures are monochrome or simple in color, and such pictures are more suitable for lighting rhythm; on the contrary, if Percent video If it is lower, it means that the picture is colorful and not suitable for strong rhythmic effects.

[0117] Through the above steps, relevant video data can be effectively extracted from the audio file, and the picture confusion weight can be calculated by analyzing the picture color, thereby guiding the design and implementation of the video rhythm effect.

[0118] Step S40: adjusting the rhythmic effect of the vehicle ambient light according to the comparison result of the rhythm point weight and the picture clutter weight.

[0119] Specifically, the video rhythm point proportion weight and the picture clutter proportion weight in the cache T time period obtained in the above steps S20 and S30 are compared, and the actual difference between the two is calculated as ΔP=|Percent Audio -Percent video |.

[0120] In order to avoid frequent switching of audio / video and generate jitter, this embodiment pre-sets a tolerance φ and does not perform switching. The specific strategy is as follows:

[0121] If the current car ambient light rhythm effect performs audio rhythm, when Percent Audio ≤Percent video , and ΔP=|Percent Audio -Percent video |≤φ(φ is 5% by default and can be adjusted dynamically), the rhythmic effect of the vehicle ambient light will remain the audio rhythm unchanged.

[0122] If the current car ambient light rhythm effect performs audio rhythm, when Percent Audio ≤Percent video , and ΔP=|Percent Audio -Percent video |>φ(φ is 5% by default and can be adjusted dynamically), the rhythmic effect of the vehicle ambient light is switched from audio rhythm to video rhythm.

[0123] Through the above steps, the rhythmic mode of the ambient light can be dynamically adjusted according to the characteristics of the audio and video content, providing a richer and more coordinated user experience.

[0124] Through the above-mentioned embodiment method, the audio rhythm of the audio file is detected to obtain the rhythm point weight; and the video color confusion of the audio file is detected to obtain the picture confusion weight; according to the comparison result of the rhythm point weight and the picture confusion weight, the rhythm effect of the vehicle ambient light is adjusted. This solution detects the audio rhythm and video color confusion, and performs weight calculation and comparison, so that the ambient light rhythm switches with the audio and video rhythm, which can make the cabin theater mode experience more realistic and the user experience better.

[0125] For example, in order to help understand the implementation process of the atmosphere light rhythm method obtained by combining this embodiment with the first embodiment, please refer to Figure 3 , Figure 3 A system architecture diagram of an atmosphere light rhythm method is provided, specifically:

[0126] exist Figure 3 In the middle, the B1 part is the central control display screen. Through the central control display screen, users can control the turning on and off of the ambient light, and can set related rhythm modes, such as: audio rhythm, video rhythm, and intelligent rhythm.

[0127] The B2 part is the host SOC system, which is mainly composed of APP, FWK, and BSP. When the user uses a video APP, the OS layer obtains the audio and video files from the background server, and can output the video or audio stream through decoding. After being processed by the audio rhythm or video rhythm algorithm, the atmosphere light driving information is generated.

[0128] Part B3 is the communication module MCU, which obtains the ambient light driving information output by the SPI and other interfaces on the SOC side, and converts it into CAN information and outputs it to the ambient light module.

[0129] Part B4 is the atmosphere light module, which converts the CAN signal on the MCU side into the control information of each specific lamp bead through the integrated microprocessor unit, so as to control the color and brightness of the lamp bead.

[0130] Through the above system architecture, the audio rhythm detection of the audio file is realized to obtain the rhythm point weight; the video color confusion detection of the audio file is realized to obtain the picture confusion weight; according to the comparison result of the rhythm point weight and the picture confusion weight, the rhythm effect of the vehicle ambient light is adjusted. Letting the ambient light rhythm switch with the audio and video rhythm can make the cockpit theater mode experience more realistic and the user experience better.

[0131] For example, in order to help understand the implementation process of the atmosphere light rhythm method obtained by combining this embodiment with the above-mentioned embodiment 1, please refer to Figure 4 , Figure 4 A brief flow chart of an atmosphere light rhythm method is provided, specifically:

[0132] When the host is working normally, the user enters the atmosphere light selection interface and selects the atmosphere light rhythm mode. There are three atmosphere light rhythm modes: audio rhythm, intelligent rhythm, and video rhythm.

[0133] If the user selects music rhythm, when the user plays the audio file, the media APP will pass the obtained pre-loaded cache audio data (such as MP4, etc.) file to the FWK system layer. The system layer will output the PCM audio stream data to the algorithm processing module after software decoding or hardware decoding.

[0134] If the user selects video rhythm, when playing the media file, only the pre-loaded cached video data is output to the algorithm module.

[0135] If the user selects intelligent rhythm, before playing the video media file, the system layer decodes the cached audio and video data in advance and converts the video (RGB888 format) + audio (PCM format) format. The embodiment of the present application mainly describes the solution with intelligent rhythm.

[0136] The algorithm processing module is mainly divided into audio rhythm algorithm processing and video rhythm algorithm processing (mainly deployed in the FWK layer or HAL layer on the SOC side).

[0137] The audio rhythm algorithm processing mainly performs audio rhythm + rhythm detection. Specifically, first, the audio file is Fourier transformed to generate a spectrum diagram. In order to ensure consistent frequency resolution, the number of sampling points and frequency resolution of each time slice are fixed.

[0138] Then, the total number of time slices generated by calculating the T-length buffer is taken from the audio file, and a time domain sequence of audio slice digital signals is generated.

[0139] Subsequently, the time domain sequence of the audio slice digital signal is Fourier transformed to generate a frequency domain sequence, and the total energy sequence corresponding to all time slices, that is, the amplitude sequence, is obtained. According to the difference between the amplitude of the current time slice and the previous time slice, the amplitude difference between the two is obtained. When the amplitude difference is greater than 0, it can be considered that the amplitude is increased at the next time point, which means that the energy is enhanced and can be considered as a potential rhythm point. All data with an amplitude difference greater than 0 are extracted to generate a potential rhythm point sequence.

[0140] Next, the local mean is calculated based on the amplitude difference. When the amplitude difference is greater than or equal to ≥ the local mean, the amplitude difference is determined to be a rhythm extreme point, and an extreme point sequence is generated.

[0141] Finally, based on the total number of time slices and the total number of corresponding rhythm points, the corresponding weights of the audio rhythm points in the duration T are calculated.

[0142] The video rhythm algorithm is executed to first obtain the picture data corresponding to the duration T from the audio file, and then count the total number of picture frames.

[0143] Then, cluster analysis is performed on each frame to generate a color ratio distribution band. According to the color ratio, the main color RGB value is obtained. According to the color band distribution ratio obtained by the color value analysis of each picture, statistics are made on whether the main color of the current picture cluster analysis exceeds the threshold of 30%. If it exceeds 30%, it is determined that the picture is low in impurity, and cumulative counting is performed to obtain the total count of the picture purity exceeding 30% within the T time length.

[0144] Finally, the picture complexity weight is calculated according to the total count of pictures with purity exceeding 30% and the total number of pictures within the time length T.

[0145] After obtaining the audio rhythm point weight and the picture clutter weight, calculate the difference between the two and compare the difference with the preset tolerance. If the current vehicle atmosphere light is performing audio rhythm, the audio rhythm point weight is less than or equal to the picture clutter weight, and the difference between the two is less than or equal to the preset tolerance, the vehicle atmosphere light continues to perform audio rhythm. If the current vehicle atmosphere light is performing audio rhythm, the audio rhythm point weight is less than or equal to the picture clutter weight, and the difference between the two is greater than the preset tolerance, the current vehicle atmosphere light switches from audio rhythm to video rhythm. By analogy, in actual application, the media files can be processed in a loop according to the above steps.

[0146] Through the above-mentioned embodiment method, the system can dynamically adjust the rhythmic mode of the atmosphere light according to the characteristics of the audio and video content, providing a richer and more coordinated user experience.

[0147] It should be noted that the above examples are only used to understand the present application and do not constitute a limitation on the atmosphere light rhythm method of the present application. More simple transformations based on this technical concept are all within the scope of protection of the present application.

[0148] This application also provides an atmosphere light rhythm system, please refer to Figure 5 , the atmosphere light rhythm system includes:

[0149] An acquisition module 10 is used to acquire an audio file;

[0150] A detection module 20, configured to perform audio rhythm detection on the audio file to obtain rhythm point weights; and

[0151] The detection module 20 is further used to perform video color clutter detection on the audio file to obtain a picture clutter weight;

[0152] The adjustment module 30 is used to adjust the rhythmic effect of the vehicle ambient light according to the comparison result of the rhythm point weight and the picture confusion weight.

[0153] The ambient light rhythm system provided by the present application adopts the ambient light rhythm method in the above embodiment, which can solve the technical problem of how to make the rhythm of the vehicle-mounted ambient light freely switch between audio rhythm and video rhythm according to the video content played. Compared with the prior art, the beneficial effects of the ambient light rhythm system provided by the present application are the same as the beneficial effects of the ambient light rhythm method provided by the above embodiment, and other technical features of the ambient light rhythm system are the same as the features disclosed in the above embodiment method, which will not be repeated here.

[0154] The present application provides an atmosphere light rhythm device, which includes: at least one processor; and a memory communicatively connected to the at least one processor; wherein the memory stores instructions executable by the at least one processor, and the instructions are executed by the at least one processor so that the at least one processor can execute the atmosphere light rhythm method in the above-mentioned embodiment one.

[0155] Reference below Figure 6, which shows a schematic diagram of the structure of an atmosphere light rhythm device suitable for implementing the embodiment of the present application. The atmosphere light rhythm device in the embodiment of the present application may include but is not limited to mobile terminals such as mobile phones, laptop computers, digital broadcast receivers, PDAs (Personal Digital Assistants), PADs (Portable Application Descriptions), PMPs (Portable Media Players), vehicle-mounted terminals (such as vehicle-mounted navigation terminals), etc., and fixed terminals such as digital TVs, desktop computers, etc. Figure 6 The illustrated ambient light rhythm device is merely an example and should not impose any limitation on the functions and scope of use of the embodiments of the present application.

[0156] like Figure 6 As shown, the ambient light rhythm device may include a processing system 1001 (e.g., a central processing unit, a graphics processor, etc.), which can perform various appropriate actions and processes according to a program stored in a read-only memory (ROM: Read Only Memory) 1002 or a program loaded from a storage system 1003 to a random access memory (RAM: Random Access Memory) 1004. In RAM1004, various programs and data required for the operation of the ambient light rhythm device are also stored. The processing system 1001, ROM1002, and RAM1004 are connected to each other through a bus 1005. An input / output (I / O) interface 1006 is also connected to the bus. Generally, the following systems can be connected to the I / O interface 1006: an input system 1007 including, for example, a touch screen, a touchpad, a keyboard, a mouse, an image sensor, a microphone, an accelerometer, a gyroscope, etc.; an output system 1008 including, for example, a liquid crystal display (LCD: Liquid Crystal Display), a speaker, a vibrator, etc.; a storage system 1003 including, for example, a magnetic tape, a hard disk, etc.; and a communication system 1009. The communication system 1009 can allow the ambient light rhythm device to communicate with other devices wirelessly or wired to exchange data. Although the figure shows an ambient light rhythm device with various systems, it should be understood that it is not required to implement or have all the systems shown. More or fewer systems can be implemented or provided alternatively.

[0157] In particular, according to the embodiments disclosed in the present application, the process described above with reference to the flowchart can be implemented as a computer software program. For example, the embodiments disclosed in the present application include a computer program product, which includes a computer program carried on a computer-readable medium, and the computer program includes a program code for executing the method shown in the flowchart. In such an embodiment, the computer program can be downloaded and installed from a network through a communication system, or installed from a storage system 1003, or installed from a ROM 1002. When the computer program is executed by the processing system 1001, the above-mentioned functions defined in the method of the embodiment disclosed in the present application are executed.

[0158] The ambient light rhythm device provided by the present application adopts the ambient light rhythm method in the above embodiment, which can solve the technical problem of how to make the rhythm of the vehicle-mounted ambient light be able to freely switch between audio rhythm and video rhythm according to the video content being played. Compared with the prior art, the beneficial effects of the ambient light rhythm device provided by the present application are the same as the beneficial effects of the ambient light rhythm method provided by the above embodiment, and the other technical features of the ambient light rhythm device are the same as the features disclosed in the method of the previous embodiment, which will not be repeated here.

[0159] It should be understood that the various parts disclosed in this application can be implemented by hardware, software, firmware or a combination thereof. In the description of the above embodiments, specific features, structures, materials or characteristics can be combined in any one or more embodiments or examples in a suitable manner.

[0160] The above is only a specific implementation of the present application, but the protection scope of the present application is not limited thereto. Any person skilled in the art who is familiar with the present technical field can easily think of changes or substitutions within the technical scope disclosed in the present application, which should be included in the protection scope of the present application. Therefore, the protection scope of the present application should be based on the protection scope of the claims.

[0161] The present application provides a computer-readable storage medium having computer-readable program instructions (ie, computer programs) stored thereon, and the computer-readable program instructions are used to execute the atmosphere light rhythm method in the above-mentioned embodiment.

[0162] The computer-readable storage medium provided in the present application may be, for example, a USB flash drive, but is not limited to electrical, magnetic, optical, electromagnetic, infrared, or semiconductor systems, systems or devices, or any combination of the above. More specific examples of computer-readable storage media may include, but are not limited to: an electrical connection with one or more wires, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the above. In this embodiment, the computer-readable storage medium may be any tangible medium containing or storing a program that can be used by or in combination with an instruction execution system, system or device. The program code contained on the computer-readable storage medium may be transmitted using any appropriate medium, including but not limited to: wires, optical cables, RF (Radio Frequency), etc., or any suitable combination of the above.

[0163] The computer-readable storage medium may be included in the atmosphere light rhythm device; or may exist independently without being assembled into the atmosphere light rhythm device.

[0164] The above-mentioned computer-readable storage medium carries one or more programs. When the above-mentioned one or more programs are executed by the atmosphere light rhythm device, the atmosphere light rhythm device: performs audio rhythm detection on the audio file to obtain the rhythm point weight; and performs video color confusion detection on the audio file to obtain the picture confusion weight; and adjusts the rhythm effect of the vehicle-mounted atmosphere light according to the comparison result of the rhythm point weight and the picture confusion weight.

[0165] Computer program code for performing the operations of the present application may be written in one or more programming languages ​​or a combination thereof, including object-oriented programming languages ​​such as Java, Smalltalk, C++, and conventional procedural programming languages ​​such as "C" or similar programming languages. The program code may be executed entirely on the user's computer, partially on the user's computer, as a separate software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In the case of a remote computer, the remote computer may be connected to the user's computer through any type of network, including a local area network (LAN) or a wide area network (WAN), or may be connected to an external computer (e.g., via the Internet using an Internet service provider).

[0166] The flow chart and block diagram in the accompanying drawings illustrate the possible architecture, function and operation of the system, method and computer program product according to various embodiments of the present application. In this regard, each box in the flow chart or block diagram can represent a module, a program segment or a part of a code, and the module, the program segment or a part of the code contains one or more executable instructions for realizing the specified logical function. It should also be noted that in some alternative implementations, the functions marked in the box can also occur in a sequence different from that marked in the accompanying drawings. For example, two boxes represented in succession can actually be executed substantially in parallel, and they can sometimes be executed in the opposite order, depending on the functions involved. It should also be noted that each box in the block diagram and / or flow chart, and the combination of the boxes in the block diagram and / or flow chart can be implemented with a dedicated hardware-based system that performs a specified function or operation, or can be implemented with a combination of dedicated hardware and computer instructions.

[0167] The modules involved in the embodiments described in this application may be implemented by software or hardware, wherein the name of the module does not constitute a limitation on the unit itself in some cases.

[0168] The readable storage medium provided by the present application is a computer-readable storage medium, which stores computer-readable program instructions (i.e., computer programs) for executing the above-mentioned atmosphere light rhythm method, and can solve the technical problem of how to make the rhythm of the vehicle-mounted atmosphere light freely switch between audio rhythm and video rhythm according to the played video content. Compared with the prior art, the beneficial effects of the computer-readable storage medium provided by the present application are the same as the beneficial effects of the atmosphere light rhythm method provided by the above-mentioned embodiment, and will not be repeated here.

[0169] The present application also provides a computer program product, including a computer program, which implements the steps of the above-mentioned atmosphere light rhythm method when executed by a processor.

[0170] The computer program product provided by the present application can solve the technical problem of how to make the rhythm of the vehicle-mounted ambient light switch freely between audio rhythm and video rhythm according to the video content being played. Compared with the prior art, the beneficial effects of the computer program product provided by the present application are the same as the beneficial effects of the ambient light rhythm method provided by the above-mentioned embodiment, which will not be elaborated here.

[0171] The above descriptions are only some embodiments of the present application, and are not intended to limit the patent scope of the present application. All equivalent structural changes made using the contents of the present application specification and drawings under the technical concept of the present application, or direct / indirect applications in other related technical fields are included in the patent protection scope of the present application.

Claims

1. A method for rhythmic movement of an atmosphere light, characterized in that: The method includes: Get the audio file; Performing audio rhythm detection on the audio file to obtain rhythm point weights; and Performing video color clutter detection on the audio file to obtain a picture clutter weight; According to the comparison result of the rhythm point weight and the picture confusion weight, the rhythm effect of the vehicle atmosphere light is adjusted.

2. The method according to claim 1, characterized in that The step of performing audio rhythm detection on the audio file to obtain rhythm point weights comprises: Extracting a time slice of fixed length from the audio file according to a preset number of time slice sampling points and a sampling rate; Calculate the amplitude difference and local mean between adjacent time slices; generating a rhythm extreme point based on the amplitude difference and the local mean; The rhythm point weight is calculated based on the total number of the rhythm extreme value points and the total number of the time slices.

3. The method according to claim 2, characterized in that The step of calculating the amplitude difference and the local mean between adjacent time slices comprises: Performing Fourier transform on the time slice to generate a corresponding frequency domain graph; Acquire the amplitude corresponding to the time slice according to the frequency domain diagram; According to the amplitude of the current time slice and the amplitude of the previous time slice, the amplitude difference is calculated; Extracting amplitude differences that meet a preset value range, and using the amplitude differences as potential rhythm points; According to the potential rhythm points, a local mean is calculated.

4. The method according to claim 2, characterized in that The step of generating a rhythm extreme point based on the amplitude difference and the local mean value comprises: comparing the amplitude difference with the local mean; When the amplitude difference is greater than or equal to the local mean, the amplitude difference is determined to be a rhythm extreme point.

5. The method according to claim 1, characterized in that The step of performing video color clutter detection on the audio file to obtain the picture clutter weight comprises: Extracting picture data corresponding to a preset duration from the audio file; Performing cluster analysis on the picture data to generate color distribution corresponding to the picture data; According to the color distribution, the picture confusion weight is calculated.

6. The method according to claim 5, characterized in that The step of calculating the picture confusion weight according to the color distribution comprises: Determining whether a main color exists in the picture data by cluster analysis; If yes, determine the color proportion of the main color in the color distribution; If the color proportion exceeds a preset threshold, the total number of picture data whose picture purity exceeds the preset threshold within a preset time length is accumulated and counted; The picture complexity weight is calculated according to the total number of picture data and the total number of picture frames.

7. The method according to claim 1, characterized in that The step of adjusting the rhythm effect of the vehicle ambient light according to the comparison result of the rhythm point weight and the picture confusion weight comprises: Calculating an actual difference between the rhythm point weight and the picture confusion weight; If the rhythm point weight is less than or equal to the picture confusion weight, and the actual difference is less than or equal to the preset tolerance, the vehicle ambient light is kept at the audio rhythm; If the rhythm point weight is less than or equal to the picture complexity weight, and the actual difference is greater than a preset tolerance, the vehicle atmosphere light is switched from the audio rhythm to the video rhythm.

8. An ambient light rhythm system, characterized in that: The system comprises: An acquisition module is used to acquire audio files; A detection module, used to perform audio rhythm detection on the audio file to obtain rhythm point weights; and The detection module is further used to perform video color clutter detection on the audio file to obtain a picture clutter weight; The adjustment module is used to adjust the rhythmic effect of the vehicle ambient light according to the comparison result of the rhythm point weight and the picture confusion weight.

9. An ambient light rhythm device, characterized in that: The device comprises: a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the computer program is configured to implement the steps of the ambient light rhythmic method according to any one of claims 1 to 7.

10. A storage medium, characterized in that: The storage medium is a computer-readable storage medium, and a computer program is stored on the storage medium. When the computer program is executed by a processor, the steps of the atmosphere light rhythm method according to any one of claims 1 to 7 are implemented.

Images