Audio visualization method, vehicle, storage medium and computer program product

Abstract

The invention discloses an audio visualization method, a vehicle, a storage medium and a computer program product, and relates to the technical field of vehicles, the audio visualization method is applied to the vehicle, the vehicle comprises a first display screen, and the method specifically comprises the following steps: obtaining a plurality of real-time driving parameters of the vehicle; generating a plurality of first audio signals based on the plurality of real-time driving parameters, and determining first animation effects respectively matched with the plurality of first audio signals; first display areas, matched with the first animation effects, on the first display screen are determined; and controlling the loudspeaker to play the first audio signal, and controlling each first display area to play the corresponding first animation effect. By adopting the method and the device, a driver can more intuitively and comprehensively know the change condition generated in the driving process of the vehicle through visual and auditory information, so that the driver has more immersive driving experience.

Description

Technical Field

[0001] The present application relates to the field of vehicle technology, and in particular to an audio visualization method, a vehicle, a storage medium, and a computer program product. Background Art

[0002] With the continuous development of the automobile industry, new energy vehicles have become the preferred means of transportation for more and more users in their daily travel.

[0003] In related technologies, in order to meet the driver's needs for audio-visual collaborative feedback during driving, the vehicle can detect hardware signals such as the steering wheel angle, accelerator pedal opening and closing degree, gear light, etc. generated by the driver during driving, and process the music file being played according to each hardware signal, so that the sound effects of the music can be dynamically adjusted as the driving behavior changes, so that the driver can perceive the changes in the vehicle's driving process through auditory information, giving the driver a more immersive driving experience.

[0004] However, although this method can modify the sound effects, it cannot allow the driver to intuitively feel the changes that occur during the vehicle's driving process through visual information. Summary of the invention

[0005] The main purpose of this application is to provide an audio visualization method, a vehicle, a storage medium and a computer program product, aiming to solve the technical problem in the related art that the driver cannot intuitively feel the changes in the vehicle's driving state.

[0006] To achieve the above object, the present application proposes an audio visualization method, which is applied to a vehicle, wherein the vehicle includes a first display screen and a speaker, and the method includes: Acquiring multiple real-time driving parameters of the vehicle; Generate a plurality of first audio signals based on a plurality of the real-time driving parameters, and determine a first animation effect that matches each of the plurality of the first audio signals; Determine a first display area on the first display screen to which each of the plurality of first animation effects matches; The speaker is controlled to play the first audio signal, and each of the first display areas is controlled to play a first animation effect corresponding to the first display area.

[0007] In one embodiment, the step of generating a plurality of first audio signals based on a plurality of the real-time driving parameters comprises: Determining preset audio track elements and preset parameter thresholds that each of the plurality of real-time driving parameters matches; The sound effect parameters of each of the preset audio track elements are adjusted based on each of the preset parameter thresholds to generate first audio signals that match each of the multiple real-time driving parameters.

[0008] In one embodiment, the step of determining the first animation effects that match each of the plurality of first audio signals comprises: Based on a preset animation mapping relationship, first animation effects that match each of the plurality of first audio signals are determined.

[0009] In one embodiment, the step of determining a first display area on the first display screen to which each of the plurality of first animation effects matches includes: Based on a preset screen mapping relationship, first display areas matching each of the plurality of first animation effects on the first display screen are determined.

[0010] In one embodiment, after the step of controlling each of the first display areas to play the first animation effect corresponding to each of the first display areas, the method further includes: Obtaining a steering wheel rotation angle of the vehicle; Determining a plurality of animation movement parameters matching the first animation effect based on the steering wheel rotation angle; The plurality of first animation effects are controlled to move on the first display screen according to the animation movement parameters.

[0011] In one embodiment, the vehicle further comprises an external speaker, and after the step of controlling each of the first display areas to play the first animation effects corresponding to each other, the method further comprises: Input the plurality of the first audio signals to the external speaker, and control the external speaker to play the plurality of the first audio signals.

[0012] In one embodiment, after the step of controlling each of the first display areas to play the first animation effect corresponding to each of the first display areas, the method further includes: Acquiring real-time environmental parameters of the vehicle; Generate a second audio signal based on the real-time environmental parameter, and determine a second animation effect matched by the second audio signal; Determine a second display area on the first display screen that matches the second animation effect, and control the second display area to play the second animation effect; A third audio signal is obtained based on the second audio signal and the first audio signal, and the built-in speaker is controlled to play the third audio signal.

[0013] In one embodiment, after the step of controlling each of the first display areas to play the first animation effect corresponding to each of the first display areas, the method further includes: Capturing target image data including the driver's face by an image acquisition module in the vehicle; determining an emotional state parameter of the driver according to the target image data, and determining a first audio adjustment parameter and a first animation adjustment parameter based on the emotional state parameter; obtaining a fourth audio signal based on the first audio adjustment parameter and the first audio signal, and playing the fourth audio signal through a built-in speaker configured in the vehicle; The first animation effect is adjusted according to the first animation adjustment parameter to obtain a third animation effect, and each of the first display areas is controlled to play the third animation effect corresponding to the first animation effect.

[0014] In one embodiment, the vehicle further comprises a second display screen, and after the step of controlling each of the first display areas to play the first animation effects corresponding to each other, the method further comprises: receiving an audio adjustment instruction sent by the second display screen; Reading a second audio adjustment parameter included in the audio adjustment instruction, and determining a second animation adjustment parameter based on the second audio adjustment parameter; obtaining a fifth audio signal based on the second audio adjustment parameter and the first audio signal, and playing the fifth audio signal through a built-in speaker configured in the vehicle; The first animation effect is adjusted according to the second animation adjustment parameter to obtain a fourth animation effect, and each of the first display areas is controlled to play the fourth animation effect corresponding to the first display area.

[0015] In addition, to achieve the above-mentioned objectives, the present application also proposes a vehicle, which includes: multiple display screens, multiple speakers, an image acquisition module, a memory, a processor, and a computer program stored on the memory and executable on the processor, wherein the computer program is configured to implement the steps of the audio visualization method described above.

[0016] In addition, to achieve the above-mentioned purpose, the present application also proposes a storage medium, which is a computer-readable storage medium, and a computer program is stored on the storage medium. When the computer program is executed by a processor, the steps of the audio visualization method described above are implemented.

[0017] In addition, to achieve the above-mentioned purpose, the present application also provides a computer program product, wherein the computer program product includes a computer program, and when the computer program is executed by a processor, the steps of the audio visualization method described above are implemented.

[0018] The audio visualization method provided in the embodiment of the present application is applied to a vehicle, wherein the vehicle includes a first display screen and a speaker, and obtains multiple real-time driving parameters of the vehicle; generates multiple first audio signals based on the multiple real-time driving parameters, and determines the first animation effects that each of the multiple first audio signals matches; determines the first display areas that each of the multiple first animation effects matches on the first display screen; controls the speaker to play the first audio signal, and controls each of the first display areas to play the first animation effects that correspond to each other.

[0019] In this embodiment, when the vehicle is running, it first obtains multiple real-time driving parameters generated by itself during driving. Then, based on the obtained multiple real-time driving parameters, the vehicle generates multiple first audio signals that match each of the real-time driving parameters. At the same time, the vehicle selects first animation effects that match each of the multiple first audio signals. Then, the vehicle determines the first display areas that match each of the multiple first animation effects on the first display screen configured internally. Finally, the vehicle controls the speaker to play the first audio signal, and controls each first display area to play the corresponding first animation effect.

[0020] In this way, the present application solves the technical problem in the related art that the driver cannot intuitively feel the changes in the vehicle's driving state. That is, the present application generates corresponding audio signals and animation effects based on the real-time driving parameters generated by the vehicle during driving, controls the speaker to play the audio signal, and controls different areas of the display screen to play animation effects representing different driving parameters, so that the driver can more intuitively and comprehensively understand the changes in the vehicle during driving through visual and auditory information, thereby giving the driver a more immersive driving experience. BRIEF DESCRIPTION OF THE DRAWINGS

[0021] The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments consistent with the present application and, together with the description, serve to explain the principles of the present application.

[0022] In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, for ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.

[0023] Figure 1 A flowchart of the first embodiment of the audio visualization method of the present application is provided; Figure 2 This is a schematic diagram of the animation effect involved in an embodiment of the audio visualization method of the present application; Figure 3This is a schematic diagram of a display area involved in an embodiment of the audio visualization method of the present application; Figure 4 A schematic diagram of an audio signal involved in an embodiment of an audio visualization method of the present application; Figure 5 A schematic diagram of an animation moving process in a region involved in an embodiment of the audio visualization method of the present application; Figure 6 A schematic diagram of an out-of-region animation movement process involved in an embodiment of the audio visualization method of the present application; Figure 7 Add a schematic diagram for the animation effect involved in the first embodiment of the audio visualization method of the present application; Figure 8 This is a schematic diagram of animation effect modification involved in an embodiment of the audio visualization method of the present application; Fig. 9 This is a schematic diagram of the module structure of the audio visualization device according to an embodiment of the present application; Fig.10 Schematic diagram of the device structure of the hardware operating environment involved in the audio visualization method in the embodiment of the present application.

[0024] The purpose, features and advantages of this application will be further described in conjunction with the embodiments and with reference to the accompanying drawings. DETAILED DESCRIPTION

[0025] It should be understood that the specific embodiments described herein are only used to explain the technical solutions of the present application and are not used to limit the present application.

[0026] In order to better understand the technical solution of the present application, a detailed description will be given below in conjunction with the accompanying drawings and specific implementation methods.

[0027] In this embodiment, for ease of description, the following description will be based on a vehicle that is internally equipped with a camera, a built-in speaker, an external speaker, multiple display screens, or a mobile terminal, a data storage control terminal, a PC or other terminal connected to an electronic control unit of the vehicle as the execution subject.

[0028] Based on the above-mentioned vehicle, the overall concept of the audio visualization method of the present application is proposed here.

[0029] With the continuous development of the automobile industry, new energy vehicles have become the preferred means of transportation for more and more users' daily travel. In related technologies, in order to meet the driver's needs for audio-visual collaborative feedback during driving, the vehicle can detect hardware signals such as the steering wheel rotation angle, accelerator pedal opening and closing, and gear light generated by the driver during driving, and process the music file being played according to each hardware signal, so that the sound effects of the music are dynamically adjusted as the driving behavior changes, so that the driver can perceive the changes in the vehicle's driving process through auditory information, giving the driver a more immersive driving experience. However, although this method can modify the sound effects, it cannot allow the driver to intuitively feel the changes in the vehicle's driving process through visual information during driving.

[0030] In response to the above phenomenon, the present application provides an audio visualization method, which is applied to a vehicle, wherein the vehicle includes a first display screen and a speaker, and the method includes: obtaining multiple real-time driving parameters of the vehicle; generating multiple first audio signals based on the multiple real-time driving parameters, and determining first animation effects that each of the multiple first audio signals matches; determining first display areas that each of the multiple first animation effects matches on the first display screen; controlling the speaker to play the first audio signal, and controlling each of the first display areas to play the first animation effects that correspond to each other.

[0031] In this way, the present application solves the technical problem in the related art that the driver cannot intuitively feel the changes in the vehicle's driving state. That is, the present application generates corresponding audio signals and animation effects based on the real-time driving parameters generated by the vehicle during driving, controls the speaker to play the audio signal, and controls different areas of the display screen to play animation effects representing different driving parameters, so that the driver can more intuitively and comprehensively understand the changes in the vehicle during driving through visual and auditory information, thereby giving the driver a more immersive driving experience.

[0032] Based on the overall concept of the audio visualization method of the present application, the present application embodiment provides an audio visualization method, referring to Figure 1 , Figure 1 This is a flow chart of the first embodiment of the audio visualization method of the present application. In this embodiment, the audio visualization method is applied to a vehicle, the vehicle includes a first display screen and a speaker, and the audio visualization method includes steps S10 to S40: Step S10: Acquire multiple real-time driving parameters of the vehicle; It should be noted that in this embodiment, the real-time driving parameters are physical quantities generated by the vehicle during driving that can reflect changes in driving behavior, including but not limited to: vehicle speed parameters, accelerator pedal opening and closing, brake pedal opening and closing, etc. It is understandable that this application does not limit the specific types that real-time driving parameters can include.

[0033] In this embodiment, when the vehicle is running, the on-board control module integrated in the vehicle first controls the sensor module configured in the vehicle, so as to collect multiple real-time driving parameters such as real-time vehicle speed parameters, real-time accelerator pedal opening and closing degree, real-time brake pedal opening and closing degree, etc. generated by the vehicle during driving through the sensor module.

[0034] Exemplarily, for example, when the vehicle is driving, the ECU (Electronic Control Unit) configured in the vehicle controls the sensor module, thereby detecting the wheels through the magnetoelectric wheel speed sensor configured in the sensor module to determine the initial vehicle speed parameters of the vehicle, and the ECU then denoises and normalizes the initial vehicle speed parameters to obtain real-time vehicle speed parameters. At the same time, the ECU detects the accelerator pedal of the vehicle through the accelerator pedal sensor configured in the sensor module to obtain the initial accelerator pedal opening and closing degree generated during the vehicle driving process, and denoises and normalizes the initial accelerator pedal opening and closing degree to obtain the real-time accelerator pedal opening and closing degree. At the same time, the ECU detects the initial brake pedal opening and closing degree of the vehicle through the brake pedal sensor configured in the sensor module, and denoises and normalizes the initial brake pedal opening and closing degree to obtain the real-time brake pedal opening and closing degree.

[0035] In this way, the vehicle can quickly synchronize the various driving parameters generated during driving, and clean and denoise the various driving parameters to ensure the accuracy of the driving parameters.

[0036] Step S20: generating a plurality of first audio signals based on a plurality of real-time driving parameters, and determining a first animation effect that matches each of the plurality of first audio signals; It should be noted that the audio signal is a combination of tracks dynamically generated based on real-time driving parameters, including track elements and sound effect parameters. It is understandable that by adjusting the sound effect parameters, the generated audio signal can be made to perform a gradual fade-in / fade-out operation, and then different audio signals can be obtained through different combinations of sound effect parameters and track elements. In addition, the animation effect is a visual dynamic effect bound to the audio signal, including but not limited to: particle diffusion, displacement animation, color gradient and other animation effects. It is understandable that the amplitude, frequency, etc. of the animation effect are nonlinearly mapped to the driving parameters.

[0037] In this embodiment, after obtaining multiple real-time driving parameters such as real-time vehicle speed parameters, real-time accelerator pedal opening and closing, and real-time brake pedal opening and closing, the vehicle-mounted control module can determine the audio track elements and sound effect parameters corresponding to the real-time vehicle speed parameters, the real-time accelerator pedal opening and closing, and the real-time brake pedal opening and closing, and combine the audio track elements and sound effect parameters corresponding to the real-time vehicle speed parameters, the real-time accelerator pedal opening and closing, and the real-time brake pedal opening and closing to generate multiple first audio signals. At the same time, the vehicle-mounted control module reads the storage module configured in the vehicle to obtain a preset animation mapping relationship, thereby querying the animation mapping relationship based on the multiple first audio signals to determine the first animation effects that match each of the multiple first audio signals.

[0038] Exemplarily, for example, after acquiring the real-time vehicle speed parameter, the real-time accelerator pedal opening and closing degree, and the real-time brake pedal opening and closing degree, the ECU first processes the real-time vehicle speed parameter, thereby determining that the audio track element corresponding to the real-time vehicle speed parameter is the keyboard audio track, and determines the first sound effect parameter corresponding to the real-time vehicle speed parameter. The ECU then combines the keyboard audio track and the first sound effect parameter to obtain the first vehicle speed audio signal corresponding to the real-time vehicle speed parameter. At the same time, the ECU processes the real-time accelerator pedal opening and closing degree, thereby determining that the audio track element corresponding to the real-time accelerator pedal opening and closing degree is the bass audio track, and determines the second sound effect parameter corresponding to the real-time accelerator pedal opening and closing degree. The ECU then combines the bass audio track and the second sound effect parameter to obtain the first throttle audio signal corresponding to the real-time accelerator pedal opening and closing degree. At the same time, the ECU processes the real-time brake pedal opening and closing degree. Processing is performed to determine that the audio track element corresponding to the real-time brake pedal opening and closing degree is a DJ disc audio track, and the third sound effect parameter corresponding to the real-time brake pedal opening and closing degree is determined. The ECU then combines the DJ disc audio track and the third sound effect parameter to obtain a first brake audio signal corresponding to the real-time brake pedal opening and closing degree. Thereafter, the ECU reads a storage module configured in the vehicle to obtain a preset animation effect query MAP (table). The ECU then queries the animation effect query MAP based on the first vehicle speed audio signal, the first throttle audio signal, and the first brake audio signal to determine that the first animation effect corresponding to the first vehicle speed audio signal is a ripple animation effect, and determines that the first animation effect corresponding to the first throttle audio signal is a particle injection effect, and determines that the first animation effect corresponding to the first brake audio signal is a line gradient effect.

[0039] In this way, the vehicle can convert real-time driving parameters into audio signals and animation effects, so that the driver can intuitively determine the changes in the vehicle during driving through the information carried by the audio signals and animation effects, and thus obtain a more immersive driving experience.

[0040] In a feasible implementation manner, the step of “generating a plurality of first audio signals based on a plurality of real-time driving parameters” in the above step S20 may specifically include steps S201 to S202: Step S201: determining preset track elements and preset parameter thresholds that are matched by a plurality of the real-time driving parameters; Step S202: adjusting the sound effect parameters of each of the preset audio track elements based on each of the preset parameter thresholds to generate a first audio signal that matches each of the plurality of real-time driving parameters.

[0041] It should be noted that the preset track element is a track element pre-stored in the track database and bound to a certain driving parameter, including but not limited to: drum set, bass, keyboard, chord, DJ playing and other track elements. It is understandable that this application does not limit the specific types of track elements. In addition, the preset parameter threshold is the critical driving parameter that triggers audio adjustment, including: vehicle speed threshold (such as 50km / h), throttle / brake opening and closing threshold (such as 50%). It is understandable that the preset parameter threshold can be used to determine how to modify the preset sound effect parameters corresponding to the preset track element.

[0042] In this embodiment, after obtaining multiple real-time driving parameters such as real-time vehicle speed parameters, real-time accelerator pedal opening and closing, and real-time brake pedal opening and closing, the on-board control module can first read the above-mentioned storage module to obtain a preset track database, and read the track database to determine the preset track elements and preset parameter thresholds corresponding to the real-time vehicle speed parameters, the real-time accelerator pedal opening and closing, and the real-time brake pedal opening and closing. After that, the on-board electronic module compares the real-time vehicle speed parameters, the real-time accelerator pedal opening and closing, and the real-time brake pedal opening and closing with the matching preset parameter thresholds, thereby determining the sound effect adjustment parameters corresponding to each preset track element based on each comparison result, and the on-board electronic module adjusts the preset sound effect parameters corresponding to each preset track element according to each sound effect adjustment parameter to obtain the first sound effect parameters corresponding to the real-time vehicle speed parameters, the real-time accelerator pedal opening and closing, and the real-time brake pedal opening and closing, and combines the preset track elements and the first sound effect parameters corresponding to the real-time vehicle speed parameters, the real-time accelerator pedal opening and closing, and the real-time brake pedal opening and closing to obtain multiple first audio signals.

[0043] For example, see Figure 4 , Figure 4 This is a schematic diagram of an audio signal involved in an embodiment of the audio visualization method of the present application, such as Figure 4As shown, after obtaining the real-time vehicle speed parameters, the real-time accelerator pedal opening and closing degree, and the real-time brake pedal opening and closing degree, the ECU first reads the above-mentioned storage module to obtain the preset track database, and queries the track database to determine that the preset track element that matches the real-time vehicle speed parameters is a keyboard track element, and determines that the preset track element that matches the real-time accelerator pedal opening and closing degree is a bass track element, and determines that the preset track element that matches the real-time brake pedal opening and closing degree is a DJ track element. At the same time, the ECU determines through the track database that the vehicle speed threshold value corresponding to the real-time vehicle speed parameters is 50km / h, and determines that the real-time accelerator The opening and closing degree threshold value corresponding to the pedal opening and closing degree is 50%, and the opening and closing degree threshold value corresponding to the real-time brake pedal opening and closing degree is determined to be 50%. After that, the ECU compares the real-time vehicle speed parameter with the vehicle speed threshold value, so as to determine a first keyboard sound effect adjustment parameter that can increase the preset keyboard sound effect parameter corresponding to the keyboard track element when it is detected that the real-time vehicle speed parameter is greater than 50km / h, and adjust the preset keyboard sound effect parameter according to the first keyboard sound effect adjustment parameter to generate the first sound effect parameter, or, when it is detected that the real-time vehicle speed parameter is less than or equal to 50km / h, determine a first keyboard sound effect adjustment parameter that can reduce the preset keyboard sound effect parameter corresponding to the keyboard track element. A second keyboard sound effect adjustment parameter of the keyboard sound effect parameter is set, and the preset keyboard sound effect parameter is adjusted according to the second keyboard sound effect adjustment parameter to generate the first sound effect parameter. The ECU then combines the keyboard track element and the corresponding first sound effect parameter to obtain a first vehicle speed audio signal corresponding to the real-time vehicle speed parameter and including a keyboard sound effect with a gradual in / fade-out effect; similarly, the ECU compares the real-time accelerator pedal opening and closing degree with the opening and closing degree threshold, so as to determine the first bass sound effect adjustment parameter that can increase the preset bass sound effect parameter corresponding to the bass track element when it is detected that the real-time accelerator pedal opening and closing degree is greater than 50%. The ECU determines a preset bass sound effect parameter according to the first bass sound effect adjustment parameter to generate a second sound effect parameter, or, when it is detected that the real-time accelerator pedal opening and closing degree is less than or equal to 50%, determines a second bass sound effect adjustment parameter that can reduce the preset bass sound effect parameter corresponding to the bass track element, and adjusts the preset bass sound effect parameter according to the second bass sound effect adjustment parameter to generate a second sound effect parameter, and the ECU further combines the bass track element and the corresponding second sound effect parameter to obtain a first throttle audio signal of a keyboard sound effect including a gradual in / fade-out effect corresponding to the real-time accelerator pedal opening and closing degree;Similarly, the ECU compares the real-time brake pedal opening and closing degree with the opening and closing degree threshold, so that when it is detected that the real-time brake pedal opening and closing degree is greater than 50%, the ECU determines a first DJ playing sound effect adjustment parameter that can increase the preset DJ playing sound effect parameter corresponding to the DJ playing sound track element, and adjusts the preset DJ playing sound effect parameter according to the first DJ playing sound effect adjustment parameter to generate a third sound effect parameter, or, when it is detected that the real-time brake pedal opening and closing degree is less than or equal to 50%, the ECU determines a second DJ playing sound effect adjustment parameter that can reduce the preset DJ playing sound effect parameter corresponding to the DJ playing sound track element, and adjusts the preset DJ playing sound effect parameter according to the second DJ playing sound effect adjustment parameter to generate a third sound effect parameter, and the ECU further combines the DJ playing sound track element and the corresponding third sound effect parameter to obtain a first brake audio signal corresponding to the real-time brake pedal opening and closing degree and including a keyboard sound effect with a gradual in / fade-out effect. ;

[0044] In this way, the vehicle can convert real-time driving parameters into audio signals, so that the driver can intuitively determine the changes in the vehicle during driving through the information carried by the audio signals, thereby obtaining a more immersive driving experience.

[0045] In a feasible implementation manner, the step of “determining first animation effects that match each of the plurality of first audio signals” in the above step S20 may specifically include step S203: Step S203: determining first animation effects that match each of the plurality of first audio signals based on a preset animation mapping relationship.

[0046] It should be noted that the animation mapping relationship is a mapping relationship between multiple preset animation effects and preset audio signals bound to each of the multiple preset animation effects. It can be understood that the preset animation effect is a visual animation effect, including but not limited to: particle diffusion, displacement animation, color gradient and other animation effects.

[0047] In this embodiment, after determining the first audio signals corresponding to the real-time vehicle speed parameters, the real-time accelerator pedal opening and closing degree, and the real-time brake pedal opening and closing degree, the on-board control module further reads the above-mentioned storage module to obtain an animation mapping relationship including multiple above-mentioned preset audio signals and preset animation effects bound to each of the multiple preset audio signals. The on-board control module queries the animation mapping relationship based on the multiple first audio signals to compare each of the multiple first audio signals with the multiple preset audio signals included in the animation mapping relationship to determine the preset audio signals that each of the multiple first audio signals matches in the animation mapping relationship. Finally, the on-board control module determines the preset animation effects corresponding to each of the multiple preset audio signals in the animation mapping relationship as the first animation effects that each of the multiple first audio signals matches.

[0048] Exemplarily, for example, after determining the first vehicle speed audio signal, the first throttle audio signal, and the first brake audio signal, the ECU further reads the above-mentioned storage module to obtain an animation effect query MAP (table) containing multiple preset audio signals and preset animation effects that match each of the multiple preset audio signals. Thereafter, the ECU queries the animation effect query MAP based on the first vehicle speed audio signal to compare the first vehicle speed audio signal with the animation effect query MAP containing multiple preset audio signals, thereby determining the first preset audio signal containing keyboard track elements in the animation effect query MAP. The ECU determines that the preset animation effect corresponding to the first preset audio signal in the animation effect query MAP is a ripple animation effect, and the ECU thereby determines the ripple animation effect as the first animation effect corresponding to the first vehicle speed audio signal. Similarly, the ECU queries the animation effect query MAP based on the first throttle audio signal to compare the first vehicle speed audio signal with the animation effect query MAP containing multiple preset audio signals. The ECU compares the signal with multiple preset audio signals included in the animation effect query MAP, thereby determining the second preset audio signal containing bass track elements in the animation effect query MAP, and the ECU determines that the preset animation effect corresponding to the second preset audio signal in the animation effect query MAP is the particle injection effect, and the ECU thereby determines the particle injection effect as the first animation effect corresponding to the first throttle audio signal; similarly, the ECU queries the animation effect query MAP based on the first brake audio signal to compare the first brake audio signal with the multiple preset audio signals included in the animation effect query MAP, thereby determining the third preset audio signal containing DJ track elements in the animation effect query MAP, and the ECU determines that the preset animation effect corresponding to the third preset audio signal in the animation effect query MAP is the line gradient effect, and the ECU thereby determines the line gradient effect as the first animation effect corresponding to the first brake audio signal.

[0049] In this way, the vehicle can convert real-time driving parameters into audio signals and animation effects, so that the driver can intuitively determine the changes in the vehicle during driving through the information carried by the audio signals and animation effects, and thus obtain a more immersive driving experience.

[0050] Step S30: determining first display areas on the first display screen to which the first animation effects are matched; Step S40: controlling the speaker to play the first audio signal, and controlling each first display area to play its corresponding first animation effect.

[0051] It should be noted that the display screen is a display device configured in the vehicle that supports dynamic rendering, specifically the HUD (Head-Up Display) corresponding to the driver's seat. In addition, the display area is a preset area in the display screen that can be independently rendered. Please refer to Figure 3 , Figure 3This is a schematic diagram of a display area involved in an embodiment of the audio visualization method of the present application. The display screen includes multiple display areas (such as Figure 3 The core display area and multiple auxiliary display areas shown in the figure) can each display area play different animation effects by independent rendering. It is understandable that the size of the coverage area and the coverage position corresponding to each display area can be set by the technicians according to the actual needs of the display screen, and this application does not limit this.

[0052] In this embodiment, after determining the first animation effects that each of the multiple first audio signals matches, the vehicle-mounted control module further queries the first display areas that each of the multiple first animation effects corresponds to on the first display screen. Finally, the vehicle-mounted control module controls the speakers configured in the vehicle to enable the speakers to play the first audio signal, and performs independent rendering operations on each first display area to control each first display area to play its corresponding first animation effect.

[0053] Exemplarily, for example, after determining that the first animation effect corresponding to the first vehicle speed audio signal is the ripple animation effect, the first animation effect corresponding to the first throttle audio signal is the particle injection effect, and the first animation effect corresponding to the first brake audio signal is the line gradient effect, the ECU may first determine that the display area included in the HUD is a core display area and multiple auxiliary display areas. The ECU further determines that the first display area corresponding to the ripple animation effect is the core display area of ​​the HUD, the first display area corresponding to the particle injection effect is the first auxiliary display area of ​​the HUD, and the first display area corresponding to the line gradient effect is the second auxiliary display area of ​​the HUD. Finally, the ECU calls the speaker to play the first audio signal, and at the same time, calls the GPU (Graphics Processing Unit, graphics processor) independently renders the core display area to display the ripple animation on the core display area, so that the ripple special effect spreads on the core display area at a frequency that matches the real-time vehicle speed parameter. Similarly, ECU calls GPU to independently render the first auxiliary display area corresponding to the particle injection effect, so that the particle injection special effect is displayed on the first auxiliary display area, so that the density of the particles is proportional to the throttle opening and closing degree. Similarly, ECU calls GPU to independently render the second auxiliary display area corresponding to the line gradient effect, so that the line gradient effect is displayed on the second auxiliary display area, so that the color gradient degree is proportional to the brake opening and closing degree.

[0054] In this way, the vehicle can partition and independently render multiple areas of the display screen, and play multiple animation effects simultaneously through multiple display areas, so that the driver can more intuitively determine the changes that occur in the vehicle during driving based on multiple animation effects, and thus obtain a more immersive driving experience.

[0055] In a feasible implementation manner, the above step S30 may specifically include step S301: Step S301: based on a preset screen mapping relationship, determining first display areas on the first display screen to which a plurality of first animation effects are matched.

[0056] It should be noted that the screen mapping relationship is a mapping relationship that includes multiple preset animation effects and preset display areas to which the multiple preset animation effects are respectively bound. It can be understood that the preset display area is a display area that is pre-divided in the vehicle display screen and can be independently rendered, and each preset display area is bound to a specific animation effect (for example, the ripple special effect corresponds to the core display area, the particle special effect corresponds to the first auxiliary display area, and the gradient special effect corresponds to the second auxiliary display area).

[0057] In this embodiment, after determining the first animation effects that each of the multiple first audio signals matches, the on-board control module may also first read the above-mentioned storage module to obtain a screen mapping relationship that includes the multiple above-mentioned preset animation effects and the preset display areas that each of the multiple preset animation effects matches. The on-board control module queries the screen mapping relationship based on the multiple first animation effects, thereby comparing each of the multiple first animation effects with the multiple preset animation effects included in the screen mapping relationship to determine the preset display areas that each of the multiple first animation effects matches within the screen mapping relationship. Finally, the on-board control module determines the first display areas that each of the multiple first animation effects matches based on the preset display areas that each of the multiple preset display areas corresponds to within the screen mapping relationship.

[0058] For example, see Figure 2 , Figure 2 This is a schematic diagram of the animation effect involved in an embodiment of the audio visualization method of the present application. After the ECU determines that the first animation effect corresponding to the first vehicle speed audio signal is a ripple animation effect, the first animation effect corresponding to the first throttle audio signal is a particle injection effect, and the first animation effect corresponding to the first brake audio signal is a line gradient effect, the ECU can also first read the above-mentioned storage module to obtain a rendering area mapping MAP of a preset display area that includes multiple of the above-mentioned preset animation effects and each of the multiple preset animation effects. The ECU queries the rendering area mapping MAP based on the ripple animation effect corresponding to the real-time vehicle speed parameter to determine that the preset display area corresponding to the ripple animation effect in the rendering area mapping MAP is the core display area. The ECU determines the core display area as the first display area for playing the animation effect representing the vehicle driving parameters. Afterwards, the ECU controls the core display area of ​​the first display screen to play the ripple animation effect to allow the driver to adjust the display area based on the following conditions: Figure 2 The ripple animation effect shown determines the real-time speed of the vehicle; Similarly, the ECU queries the rendering area map MAP based on the particle injection effect corresponding to the real-time accelerator pedal opening and closing degree to determine that the preset display area corresponding to the particle injection effect in the rendering area map MAP is the first auxiliary display area. The ECU determines the first auxiliary display area as the first display area for playing the animation effect representing the vehicle driving parameters. After that, the ECU controls the first auxiliary display area of ​​the first display screen to play the particle injection effect, so that the driver can Figure 2 The particle injection effect shown in determines the degree of opening and closing of the vehicle's accelerator pedal; Similarly, the ECU queries the rendering area mapping MAP based on the line gradient effect corresponding to the real-time brake pedal opening and closing degree to determine that the preset display area corresponding to the line gradient effect in the rendering area mapping MAP is the second auxiliary display area. The ECU determines the second auxiliary display area as the first display area for playing the animation effect representing the vehicle driving parameters. After that, the ECU controls the second auxiliary display area of ​​the first display screen to play the line gradient effect, so that the driver can Figure 2 The line gradient shown determines the vehicle's accelerator pedal opening and closing.

[0059] In this embodiment, when the vehicle is running, the on-board control module integrated in the vehicle first controls the sensor module configured in the vehicle to collect multiple real-time driving parameters such as real-time vehicle speed parameters, real-time accelerator pedal opening and closing degree, and real-time brake pedal opening and closing degree generated by the vehicle during driving through the sensor module. Afterwards, the on-board control module determines the audio track elements and sound effect parameters corresponding to the real-time vehicle speed parameters, the real-time accelerator pedal opening and closing degree, and the real-time brake pedal opening and closing degree, respectively, and combines the audio track elements and sound effect parameters corresponding to the real-time vehicle speed parameters, the real-time accelerator pedal opening and closing degree, and the real-time brake pedal opening and closing degree to generate multiple a first audio signal, and at the same time, the on-board control module reads the storage module configured in the vehicle to obtain a preset animation mapping relationship, so as to query the animation mapping relationship based on the multiple first audio signals to determine the first animation effects that each of the multiple first audio signals matches. After that, the on-board control module queries the first display areas corresponding to each of the multiple first animation effects on the first display screen. Finally, the on-board control module controls the speakers configured in the vehicle to make the speakers play the first audio signal, and performs independent rendering operations on each first display area to control each first display area to play the first animation effect corresponding to each first display area.

[0060] In this way, the present application solves the technical problem in the related art that the driver cannot intuitively feel the changes in the vehicle's driving state. That is, the present application generates corresponding audio signals and animation effects based on the real-time driving parameters generated by the vehicle during driving, controls the speaker to play the audio signal, and controls different areas of the display screen to play animation effects representing different driving parameters, so that the driver can more intuitively and comprehensively understand the changes in the vehicle during driving through visual and auditory information, thereby giving the driver a more immersive driving experience.

[0061] Based on the first embodiment of the present application, a second embodiment of the present application is proposed. In the second embodiment of the present application, the same or similar contents as those of the above embodiments can be referred to the above description and will not be repeated later. On this basis, after the above step S40, the audio visualization method of the present application can also include steps A10 to A30: Step A10: Obtaining the steering wheel rotation angle of the vehicle; Step A20: determining a plurality of animation movement parameters matching the first animation effect based on the steering wheel rotation angle; Step A30: controlling the movement of the plurality of first animation effects on the first display screen according to the animation movement parameters.

[0062] It should be noted that the animation movement parameter is a quantitative index used to change the display area where the above animation effect is located, including: horizontal displacement, vertical displacement, movement speed and other parameters.

[0063] In this embodiment, after the vehicle-mounted control module controls the first display screen to display the first animation effects corresponding to the real-time vehicle speed parameters, the real-time accelerator pedal opening and closing degree, and the real-time brake pedal opening and closing degree, the vehicle-mounted control module can also detect the steering wheel to determine the steering wheel rotation angle generated during the vehicle's driving process. Afterwards, the vehicle-mounted control module determines the steering direction and angle response range corresponding to the steering wheel according to the steering wheel rotation angle, and then determines the animation movement parameters for adjusting multiple first animation effects according to the steering direction and the angle response range. Finally, the vehicle-mounted control module controls the multiple first animation effects to move on the first display screen based on each animation movement parameter, thereby playing each first animation effect after movement on the first display screen.

[0064] For example, see Figure 5 , Figure 5This is a schematic diagram of the animation movement process in the area involved in an embodiment of the audio visualization method of the present application. After the ECU controls the first display screen to display the ripple animation effect, the particle injection effect, and the line gradient effect, it can also call the above-mentioned sensor module to detect the steering wheel in the vehicle to determine the corresponding steering wheel rotation angle during the vehicle driving process. Afterwards, if the ECU determines that the steering wheel rotation angle is 45°, it calculates the horizontal displacement and horizontal movement direction corresponding to each first animation effect based on the steering wheel rotation angle of 45° and the response coefficient of the steering wheel, and determines the horizontal displacement and horizontal movement direction as the animation movement parameters corresponding to each first animation effect. Then, as Figure 5 As shown, the ECU determines the first display position of the ripple animation effect in the core display area, and determines the second display position of the adjusted ripple animation effect based on the first display position and animation movement parameters corresponding to the ripple animation effect, thereby independently rendering the second display position of the adjusted ripple animation effect through the GPU to control the movement of the ripple animation effect, and then playing the moved ripple animation effect at the second display position.

[0065] In addition, please refer to Figure 6 , Figure 6 FIG. 1 is a schematic diagram of an animation moving process outside a region involved in an embodiment of the audio visualization method of the present application. It can be understood that the second display position after the ripple animation effect moves can be located not only inside the corresponding display region, but also outside the corresponding display region. Figure 6 As shown, it is located in the other display area on the first display screen.

[0066] Similarly, the ECU also determines the first display position of the particle spray effect in the first auxiliary display area, and determines the adjusted second display position of the particle spray effect based on the first display position and animation movement parameters corresponding to the particle spray effect, so that the GPU independently renders the adjusted second display position of the particle spray effect to control the movement of the particle spray effect, and then plays the moved particle spray effect at the corresponding second display position. It can be understood that if Figure 5 and Figure 6 As shown, the second display position where the particle spray effect is moved can be located in the above-mentioned first auxiliary display area, or in other auxiliary display areas or a core display area on the first display screen.

[0067] Similarly, the ECU also determines the first display position of the line gradient effect in the second auxiliary display area, and determines the second display position of the line gradient effect after adjustment based on the first display position and animation movement parameters corresponding to the line gradient effect. The ECU then independently renders the second display position after adjustment of the line gradient effect through the GPU to control the movement of the line gradient effect, and then plays the moved line gradient effect at the corresponding second display position. It can be understood that if Figure 5 and Figure 6 As shown, the second display position where the line gradient effect moves can be located in the second auxiliary display area mentioned above, or in other auxiliary display areas or core display areas on the first display screen.

[0068] In this way, the vehicle can synchronize the animation effects being played based on the driving operations issued by the driver, allowing the driver to more intuitively view the vehicle's driving changes and giving the driver a more immersive driving experience.

[0069] Based on the first embodiment and / or the second embodiment of the present application, a third embodiment of the present application is proposed. In the third embodiment of the present application, the same or similar contents as those of the above embodiments can be referred to the above description and will not be described in detail later. On this basis, the vehicle further includes an external speaker. After the above step S40, the audio visualization method of the present application may further include step B10: Step B10: inputting the plurality of first audio signals into the external speaker, and controlling the external speaker to play the plurality of first audio signals.

[0070] It should be noted that the external speaker is an independent audio output device configured on the outside of the vehicle, including but not limited to: roof speakers, chassis sound field diffusers, etc. It can be understood that the external speaker is used to play the above-mentioned audio signals that have been dynamically adjusted to the outside.

[0071] In this embodiment, after the vehicle-mounted control module controls the first display screen to display the first animation effects corresponding to the real-time vehicle speed parameters, the real-time accelerator pedal opening and closing degree, and the real-time brake pedal opening and closing degree, the vehicle-mounted control module can also input the above-generated first audio signals into the external speakers configured for the vehicle, so that the multiple first audio signals are played by the external speakers.

[0072] In this way, the vehicle can transmit the generated sound effects to the external environment through an external player, allowing other users outside the vehicle to understand the driving status of the vehicle based on the audio signal, and enable vehicles to interact with each other, thereby allowing the driver to obtain a more immersive driving experience.

[0073] Based on the various embodiments of the present application, a fourth embodiment of the present application is proposed here. In the fourth embodiment of the present application, the same or similar contents as the above embodiments can be referred to the above introduction, and will not be repeated later. On this basis, after the above step S40, the audio visualization method of the present application can also include steps C10~C40: Step C10: obtaining real-time environmental parameters of the vehicle; Step C20: generating a second audio signal based on the real-time environmental parameter, and determining a second animation effect matched by the second audio signal; Step C30: determining a second display area on the first display screen that matches the second animation effect, and controlling the second display area to play the second animation effect; Step C40: obtaining a third audio signal based on the second audio signal and the first audio signal, and controlling the built-in speaker to play the third audio signal.

[0074] It should be noted that the real-time environmental parameters are status data of the vehicle's environment, including but not limited to: weather type, light intensity, road type, external noise level, etc.

[0075] In this embodiment, after the vehicle-mounted control module controls the first display screen to display the first animation effects corresponding to the real-time vehicle speed parameters, the real-time accelerator pedal opening and closing degree, and the real-time brake pedal opening and closing degree, the vehicle-mounted control module can also collect the real-time environmental parameters generated by the vehicle during driving through the above-mentioned sensor module. After that, the vehicle-mounted control module determines the audio track elements and sound effect parameters corresponding to the real-time environmental parameters, and integrates the audio track elements and sound effect parameters corresponding to the real-time environmental parameters to generate a second audio signal matching the real-time environmental parameters. At the same time, the vehicle-mounted control module queries the above-mentioned animation mapping relationship based on the second audio signal to determine the second animation effect matching the second audio signal. Finally, the vehicle-mounted control module queries the above-mentioned screen mapping relationship to determine the second display area corresponding to the second animation effect on the first display screen, and performs an independent rendering operation on the second display area, thereby controlling the second display area to play the second animation effect. At the same time, the vehicle-mounted control module fuses the generated second audio signal with the above-mentioned first audio signal to obtain a third audio signal, and controls the speaker to play the third audio signal.

[0076] For example, see Figure 7 , Figure 7 A schematic diagram is added for the animation effect involved in the first embodiment of the audio visualization method of the present application, such as Figure 7As shown, after controlling the first display screen to display the ripple animation effect, particle injection effect, and line gradient effect, the ECU can also call the above-mentioned sensor module to detect the environment in which the vehicle is located, so as to obtain the real-time environmental parameters such as weather type, light intensity, road type, and external noise level corresponding to the environment in which the vehicle is located. After that, the ECU processes the real-time environmental parameters to determine that the audio track element corresponding to the real-time environmental parameters is a violin track, and integrates the violin track and the corresponding sound effect parameters to obtain a second audio signal that matches the real-time environmental parameters. At the same time, the ECU queries the above-mentioned animation effect query MAP based on the second audio signal to determine The second animation effect corresponding to the second audio signal is a raindrop animation effect. Finally, the ECU queries the above-mentioned rendering area mapping MAP to determine that the display area corresponding to the raindrop animation effect is the third auxiliary display area in the above-mentioned HUD. The ECU then determines the third auxiliary display area as the second display area for playing animation effects representing real-time environmental parameters. The ECU then calls the GPU to independently render the third auxiliary display area to display the raindrop animation effect on the third auxiliary area. At the same time, the ECU fuses the generated second audio signal with the above-mentioned first audio signal to generate a third audio signal, and controls the speaker to play the third audio signal.

[0077] In this way, the vehicle can enable the driver to more intuitively determine the specific changes in the external environment while the vehicle is driving based on the animation effects, thereby allowing the driver to obtain a more immersive driving experience.

[0078] Based on the various embodiments of the present application, a fifth embodiment of the present application is proposed here. In the fifth embodiment of the present application, the same or similar contents as the above embodiments can be referred to the above introduction, and will not be repeated later. On this basis, after the above step S40, the audio visualization method of the present application can also include steps D10~D40: Step D10: capturing target image data including the driver's face through an image acquisition module in the vehicle; Step D20: determining the driver's emotional state parameter according to the target image data, and determining a first audio adjustment parameter and a first animation adjustment parameter based on the emotional state parameter; Step D30: obtaining a fourth audio signal based on the first audio adjustment parameter and the first audio signal, and playing the fourth audio signal through a built-in speaker configured in the vehicle; Step D40: adjusting the first animation effect according to the first animation adjustment parameter to obtain a third animation effect, and controlling each of the first display areas to play the third animation effect corresponding to the first animation effect.

[0079] It should be noted that the emotional state parameter is a quantitative index that can indicate the emotional characteristics of the driver, including but not limited to: tension index (0% to 100%), pleasure index (0% to 100%), concentration level (low / medium / high), etc. It is understandable that by adjusting the audio signal and animation effect, the audio signal and animation effect are more in line with the emotional characteristics of the driver, which can further enhance the driver's sense of immersion.

[0080] In this embodiment, after the vehicle control module controls the first display screen to display the first animation effects corresponding to the real-time vehicle speed parameter, the real-time accelerator pedal opening and closing degree, and the real-time brake pedal opening and closing degree, the vehicle control module can also first shoot the driver's face through the image acquisition module in the vehicle to capture the target image data containing the driver's face. After that, the vehicle control module extracts the facial features contained in the target image data, and determines the emotional state parameters corresponding to the driver according to the facial features. The vehicle control module queries the preset emotional sound effect database to determine the first audio adjustment parameters that match the emotional state parameters. At the same time, the vehicle control module queries the preset emotional animation database to determine the first animation adjustment parameters that match the emotional state parameters. Finally, the vehicle control module adjusts the audio track elements and / or sound effect parameters in the above-mentioned first audio signal according to the first audio adjustment parameters to obtain a fourth audio signal, and plays the fourth audio signal through the built-in speaker of the vehicle. At the same time, the vehicle control module adjusts the above-mentioned first animation effect according to the first animation adjustment parameters to obtain a third animation signal, and controls the above-mentioned first display screen to continue to play multiple third animation effects.

[0081] For example, see Figure 8 , Figure 8 This is a schematic diagram of animation effect modification involved in an embodiment of the audio visualization method of the present application, such as Figure 8 As shown, after controlling the first display screen to display the ripple animation effect, particle injection effect, and line gradient effect, the ECU can also call the camera configured in front of the driver's seat to shoot the driver's face to capture the target image data containing the driver's face. After that, the ECU inputs the target image data into the neural network model configured by itself, and the neural network model extracts facial features such as eyes, mouth corners, and eyebrow movements contained in the target image data, and identifies the driver's emotional state parameters based on each facial feature. At this time, if the neural network model identifies that the emotional state parameter is nervous emotion, the vehicle control module queries the preset emotional sound effect database based on the nervous emotion to determine that the first audio adjustment parameter matching the emotional state parameter is "increase the noise reduction track by 4dB, and reduce the keyboard element track by 8dB". At the same time, the vehicle control module queries the preset animation effect database based on the nervous emotion to determine the first animation background color adjustment parameter matching the emotional state parameter (i.e. Figure 8 ), finally, the ECU first adjusts each first audio signal according to the first audio adjustment parameter to reduce the keyboard track elements contained in each first audio signal, and adds white noise to each first audio signal to obtain a fourth audio signal. The ECU then plays the fourth audio signal through a speaker in the vehicle to relieve the driver's nervousness through the fourth audio signal. At the same time, the ECU adjusts each first animation effect according to the first animation background color adjustment parameter to adjust the background color of each first animation effect to a dark color, and plays the third animation effects such as ripple animation, particle injection, color gradient and raindrop animation with the background color deepened through the above-mentioned first display screen, so as to reduce the driver's nervousness through the third animation effect.

[0082] In this way, the vehicle can further modify the played audio signals and animation effects based on the driver's emotional parameters to achieve customization of audio signals and animation effects, further allowing the driver to obtain a more immersive driving experience.

[0083] Based on the embodiments of the present application, a sixth embodiment of the present application is proposed. In the sixth embodiment of the present application, the same or similar contents as the above embodiments can be referred to the above description and will not be repeated later. On this basis, the vehicle further includes a second display screen. After the above step S30, the audio visualization method of the present application can also include steps E10 to E40: Step E10: receiving an audio adjustment instruction sent by the second display screen; Step E20: reading a second audio adjustment parameter included in the audio adjustment instruction, and determining a second animation adjustment parameter based on the second audio adjustment parameter; Step E30: obtaining a fifth audio signal based on the second audio adjustment parameter and the first audio signal, and playing the fifth audio signal through a built-in speaker configured in the vehicle; Step E40: adjusting the first animation effect according to the second animation adjustment parameter to obtain a fourth animation effect, and controlling each of the first display areas to play the fourth animation effect corresponding to the first display area.

[0084] It should be noted that the second display screen is a display device that can interact with the above-mentioned first display screen, including but not limited to: rear entertainment screen, co-pilot screen or mobile terminal projection screen. It can be understood that the second display screen can receive audio handover instructions input by the user.

[0085] In this embodiment, after controlling the first display screen to display the ripple animation effect, particle spray effect, and line gradient effect, the on-board control module can also receive audio adjustment instructions triggered by users other than the driver in the vehicle through the second display screen configured in the vehicle. After that, the on-board control module parses the second audio adjustment parameter contained in the audio adjustment instruction. At the same time, the on-board control module determines the second animation adjustment parameter that matches the second audio adjustment parameter. Finally, the on-board control module adjusts the audio track elements and / or sound effect parameters in the above-mentioned first audio signal according to the second audio adjustment parameter to obtain a fifth audio signal, and plays the fifth audio signal through the vehicle's built-in speakers. At the same time, the on-board control module adjusts the above-mentioned first animation effect according to the second animation adjustment parameter to obtain a fourth animation signal, and controls the above-mentioned first display screen to continue playing multiple fourth animation effects.

[0086] Exemplarily, for example, after controlling the display of ripple animation effects, particle injection effects, and line gradient effects on the first display screen, the ECU can also receive an audio adjustment instruction sent by the second display screen through the CAN bus. After that, the ECU parses the audio adjustment instruction to determine the second audio adjustment parameter contained in the audio adjustment instruction. At the same time, the ECU reads the above-mentioned storage module to obtain the preset animation adjustment MAP, thereby determining the second animation adjustment parameter that matches the second audio adjustment parameter based on the multiple preset audio adjustment parameters contained in the animation adjustment MAP and the preset animation adjustment parameters that match the multiple preset audio adjustment parameters. Finally, the ECU first adjusts each first audio signal according to the second audio adjustment parameter to obtain a fifth audio signal, and the ECU then plays the fifth audio signal through the speaker in the vehicle. At the same time, the ECU adjusts each first animation effect according to the second animation adjustment parameter to play each fourth animation effect through the above-mentioned first display screen.

[0087] In this way, other passengers in the vehicle can make further modifications to the audio signals and animation effects being played to customize the audio signals and animation effects, thereby enhancing the sense of interaction between the passengers and the vehicle.

[0088] This application also provides an audio visualization device, please refer to Fig. 9 The audio visualization device is applied to a vehicle, the vehicle comprises a first display screen and a speaker, and the device comprises: A parameter detection module 10, used to obtain a plurality of real-time driving parameters of the vehicle; An animation generation module 20, used to generate a plurality of first audio signals based on a plurality of the real-time driving parameters, and determine a first animation effect that matches each of the plurality of the first audio signals; An area screening module 30, configured to determine a first display area on the first display screen to which each of the plurality of first animation effects matches; The animation playing module 40 is used to control the speaker to play the first audio signal, and control each of the first display areas to play its corresponding first animation effect.

[0089] In a feasible implementation manner, the animation generation module 20 is further used for: Determining preset audio track elements and preset parameter thresholds that each of the plurality of real-time driving parameters matches; The sound effect parameters of each of the preset audio track elements are adjusted based on each of the preset parameter thresholds to generate first audio signals that match each of the multiple real-time driving parameters.

[0090] In a feasible implementation manner, the animation generation module 20 is further used for: Based on a preset animation mapping relationship, first animation effects that match each of the plurality of first audio signals are determined.

[0091] In a feasible implementation manner, the above-mentioned area screening module 30 is further used for: Based on a preset screen mapping relationship, first display areas matching each of the plurality of first animation effects on the first display screen are determined.

[0092] In a feasible implementation manner, the animation playing module 40 is further used for: Obtaining a steering wheel rotation angle of the vehicle; Determining a plurality of animation movement parameters matching the first animation effect based on the steering wheel rotation angle; The plurality of first animation effects are controlled to move on the first display screen according to the animation movement parameters.

[0093] In a feasible implementation manner, the vehicle further comprises an external speaker, and the animation playing module 40 is further used for: Obtaining a steering wheel rotation angle of the vehicle; Input the plurality of the first audio signals to the external speaker, and control the external speaker to play the plurality of the first audio signals.

[0094] In a feasible implementation manner, the animation playing module 40 is further used for: Acquiring real-time environmental parameters of the vehicle; Generate a second audio signal based on the real-time environmental parameter, and determine a second animation effect matched by the second audio signal; Determine a second display area on the first display screen that matches the second animation effect, and control the second display area to play the second animation effect; A third audio signal is obtained based on the second audio signal and the first audio signal, and the built-in speaker is controlled to play the third audio signal.

[0095] In a feasible implementation manner, the animation playing module 40 is further used for: Capturing target image data including the driver's face by an image acquisition module in the vehicle; determining an emotional state parameter of the driver according to the target image data, and determining a first audio adjustment parameter and a first animation adjustment parameter based on the emotional state parameter; obtaining a fourth audio signal based on the first audio adjustment parameter and the first audio signal, and playing the fourth audio signal through a built-in speaker configured in the vehicle; The first animation effect is adjusted according to the first animation adjustment parameter to obtain a third animation effect, and each of the first display areas is controlled to play the third animation effect corresponding to the first animation effect.

[0096] In a feasible implementation manner, the vehicle further comprises a second display screen, and the animation playing module 40 is further used for: receiving an audio adjustment instruction sent by the second display screen; Reading a second audio adjustment parameter included in the audio adjustment instruction, and determining a second animation adjustment parameter based on the second audio adjustment parameter; obtaining a fifth audio signal based on the second audio adjustment parameter and the first audio signal, and playing the fifth audio signal through a built-in speaker configured in the vehicle; The first animation effect is adjusted according to the second animation adjustment parameter to obtain a fourth animation effect, and each of the first display areas is controlled to play the fourth animation effect corresponding to the first display area.

[0097] The audio visualization device provided by the present application adopts the audio visualization method in the above embodiment, which can solve the technical problem in the related art that it is difficult for the vehicle to accurately identify the sitting posture of the occupant. Compared with the prior art, the beneficial effects of the audio visualization device provided by the present application are the same as the beneficial effects of the audio visualization method provided by the above embodiment, and the other technical features in the audio visualization device are the same as the features disclosed in the above embodiment method, which will not be repeated here.

[0098] The present application provides a vehicle, comprising: at least one processor; and a memory communicatively connected to the at least one processor; wherein the memory stores instructions executable by the at least one processor, and the instructions are executed by the at least one processor so that the at least one processor can execute the audio visualization method in the above-mentioned embodiment 1.

[0099] Reference below Fig.10 , which shows a schematic diagram of the structure of a vehicle suitable for implementing the embodiment of the present application. The vehicle in the embodiment of the present application may include, but is not limited to, a vehicle with a camera, a built-in speaker, an external speaker, a plurality of display screens, or a mobile terminal, a data storage control terminal, a PC, etc. connected to an electronic control unit of the vehicle. Fig.10 The vehicle shown is merely an example and should not bring any limitation to the functions and scope of use of the embodiments of the present application.

[0100] like Fig.10 As shown, the vehicle may include a processing device 1001 (e.g., a central processing unit, a graphics processing unit, etc.), which may perform various appropriate actions and processes according to a program stored in a read-only memory 1002 or a program loaded from a storage device 1003 to a random access memory 1004. Various programs and data required for vehicle operation are also stored in the random access memory 1004. The processing device 1001, the read-only memory 1002, and the random access memory 1004 are connected to each other via a bus 1005. An input / output interface 1006 is also connected to the bus. Generally, the following systems may be connected to the input / output interface 1006: an input device 1007 including, for example, a touch screen, a touch pad, a keyboard, a mouse, an image sensor, a microphone, an accelerometer, a gyroscope, etc.; an output device 1008 including, for example, a liquid crystal display (LCD: Liquid Crystal Display), a speaker, a vibrator, etc.; a storage device 1003 including, for example, a magnetic tape, a hard disk, etc.; and a communication device 1009. The communication device 1009 may allow the vehicle to communicate with other devices wirelessly or by wire to exchange data. Although the figure shows a vehicle with various systems, it should be understood that it is not required to implement or have all the systems shown. More or fewer systems may be implemented or have instead.

[0101] In particular, according to the embodiments disclosed in the present application, the process described above with reference to the flowchart can be implemented as a computer software program. For example, the embodiments disclosed in the present application include a computer program product, which includes a computer program carried on a computer-readable medium, and the computer program includes a program code for executing the method shown in the flowchart. In such an embodiment, the computer program can be downloaded and installed from a network through a communication device, or installed from a storage device 1003, or installed from a read-only memory 1002. When the computer program is executed by the processing device 1001, the above-mentioned functions defined in the method of the embodiment disclosed in the present application are executed.

[0102] The vehicle provided by the present application adopts the audio visualization method in the above embodiment, which can solve the technical problem in the related art that the driver cannot intuitively feel the changes in the vehicle's driving state. Compared with the prior art, the beneficial effects of the vehicle provided by the present application are the same as the beneficial effects of the audio visualization method provided by the above embodiment, and the other technical features in the vehicle are the same as the features disclosed in the method of the previous embodiment, which will not be repeated here.

[0103] It should be understood that the various parts disclosed in this application can be implemented by hardware, software, firmware or a combination thereof. In the description of the above embodiments, specific features, structures, materials or characteristics can be combined in any one or more embodiments or examples in a suitable manner.

[0104] The above is only a specific implementation of the present application, but the protection scope of the present application is not limited thereto. Any person skilled in the art who is familiar with the present technical field can easily think of changes or substitutions within the technical scope disclosed in the present application, which should be included in the protection scope of the present application. Therefore, the protection scope of the present application should be based on the protection scope of the claims.

[0105] The present application provides a computer-readable storage medium having computer-readable program instructions (ie, computer programs) stored thereon, and the computer-readable program instructions are used to execute the audio visualization method in the above-mentioned embodiment.

[0106] The computer-readable storage medium provided in the present application may be, for example, a USB flash drive, but is not limited to electrical, magnetic, optical, electromagnetic, infrared, or semiconductor systems or devices, or any combination of the above. More specific examples of computer-readable storage media may include, but are not limited to: an electrical connection with one or more wires, a portable computer disk, a hard disk, a random access memory (RAM: Random Access Memory), a read-only memory (ROM: Read Only Memory), an erasable programmable read-only memory (EPROM: Erasable Programmable Read Only Memory or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM: CD-Read Only Memory), an optical storage device, a magnetic storage device, or any suitable combination of the above. In this embodiment, the computer-readable storage medium may be any tangible medium containing or storing a program, which may be used by or in combination with an instruction execution system or device. The program code contained on the computer-readable storage medium may be transmitted using any appropriate medium, including but not limited to: wires, optical cables, RF (Radio Frequency: Radio Frequency), etc., or any suitable combination of the above.

[0107] The computer-readable storage medium may be included in the vehicle, or may exist independently without being installed in the vehicle.

[0108] The above-mentioned computer-readable storage medium carries one or more programs. When the above-mentioned one or more programs are executed by the vehicle, the vehicle: obtains multiple real-time driving parameters of the vehicle; generates multiple first audio signals based on the multiple real-time driving parameters, and determines the first animation effects that each of the multiple first audio signals matches; determines the first display areas that each of the multiple first animation effects matches on the first display screen, and controls each of the first display areas to play the corresponding first animation effects.

[0109] Computer program code for performing the operations of the present application may be written in one or more programming languages ​​or a combination thereof, including object-oriented programming languages ​​such as Java, Smalltalk, C++, and conventional procedural programming languages ​​such as "C" or similar programming languages. The program code may be executed entirely on the user's computer, partially on the user's computer, as a separate software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In the case of a remote computer, the remote computer may be connected to the user's computer through any type of network, including a local area network (LAN) or a wide area network (WAN), or may be connected to an external computer (e.g., via the Internet using an Internet service provider).

[0110] The flow chart and block diagram in the accompanying drawings illustrate the possible architecture, function and operation of the system, method and computer program product according to various embodiments of the present application. In this regard, each square box in the flow chart or block diagram can represent a module, a program segment or a part of a code, and the module, the program segment or a part of the code contains one or more executable instructions for realizing the specified logical function. It should also be noted that in some alternative implementations, the functions marked in the square box can also occur in a sequence different from that marked in the accompanying drawings. For example, two square boxes represented in succession can actually be executed substantially in parallel, and they can sometimes be executed in the opposite order, depending on the functions involved. It should also be noted that each square box in the block diagram and / or flow chart, and the combination of the square boxes in the block diagram and / or flow chart can be implemented with a dedicated hardware-based system that performs a specified function or operation, or can be implemented with a combination of dedicated hardware and computer instructions.

[0111] The modules involved in the embodiments described in this application may be implemented by software or hardware, wherein the name of the module does not constitute a limitation on the unit itself in some cases.

[0112] The readable storage medium provided in the present application is a computer-readable storage medium, which stores computer-readable program instructions (i.e., computer programs) for executing the above-mentioned audio visualization method, and can solve the technical problem in the related art that the driver cannot intuitively feel the changes in the vehicle's driving state. Compared with the prior art, the beneficial effects of the computer-readable storage medium provided in the present application are the same as the beneficial effects of the audio visualization method provided in the above-mentioned embodiment, and will not be elaborated here.

[0113] The present application also provides a computer program product, including a computer program, which implements the steps of the above-mentioned audio visualization method when executed by a processor.

[0114] The computer program product provided by the present application can solve the technical problem in the related art that the driver cannot intuitively feel the changes in the vehicle's driving state. Compared with the prior art, the beneficial effects of the computer program product provided by the present application are the same as the beneficial effects of the audio visualization method provided by the above embodiment, which will not be repeated here.

[0115] The above descriptions are only some embodiments of the present application, and are not intended to limit the patent scope of the present application. All equivalent structural changes made using the contents of the present application specification and drawings under the technical concept of the present application, or direct / indirect applications in other related technical fields are included in the patent protection scope of the present application.

Claims

1. An audio visualization method, characterized in that: The audio visualization method is applied to a vehicle, wherein the vehicle comprises a first display screen and a speaker, and the method comprises: Acquiring multiple real-time driving parameters of the vehicle; Generate a plurality of first audio signals based on a plurality of the real-time driving parameters, and determine a first animation effect that matches each of the plurality of the first audio signals; Determine a first display area on the first display screen to which each of the plurality of first animation effects matches; The speaker is controlled to play the first audio signal, and each of the first display areas is controlled to play its corresponding first animation effect.

2. The audio visualization method according to claim 1, characterized in that: The step of generating a plurality of first audio signals based on a plurality of the real-time driving parameters comprises: Determining preset audio track elements and preset parameter thresholds that each of the plurality of real-time driving parameters matches; The sound effect parameters of each of the preset audio track elements are adjusted based on each of the preset parameter thresholds to generate first audio signals that match each of the multiple real-time driving parameters.

3. The audio visualization method according to claim 1, characterized in that: The step of determining first animation effects that match each of the plurality of first audio signals comprises: Based on a preset animation mapping relationship, first animation effects that match each of the plurality of first audio signals are determined.

4. The audio visualization method according to claim 3, characterized in that: The step of determining a first display area on the first display screen to which each of the plurality of first animation effects matches includes: Based on a preset screen mapping relationship, first display areas on the first display screen to which each of the plurality of first animation effects matches are determined.

5. The audio visualization method according to claim 1, characterized in that: After the step of controlling each of the first display areas to play the first animation effects corresponding to each of the first display areas, the method further includes: Obtaining a steering wheel rotation angle of the vehicle; Determining a plurality of animation movement parameters matching the first animation effect based on the steering wheel rotation angle; The plurality of first animation effects are controlled to move on the first display screen according to the animation movement parameters.

6. The audio visualization method according to claim 1, characterized in that: The vehicle further includes an external speaker. After the step of controlling each of the first display areas to play the first animation effects corresponding to each other, the method further includes: Input the plurality of the first audio signals to the external speaker, and control the external speaker to play the plurality of the first audio signals.

7. The audio visualization method according to any one of claims 1 to 6, characterized in that: After the step of controlling each of the first display areas to play the first animation effects corresponding to each of the first display areas, the method further includes: Acquiring real-time environmental parameters of the vehicle; Generate a second audio signal based on the real-time environmental parameter, and determine a second animation effect matched by the second audio signal; Determine a second display area on the first display screen that matches the second animation effect, and control the second display area to play the second animation effect; A third audio signal is obtained based on the second audio signal and the first audio signal, and the built-in speaker is controlled to play the third audio signal.

8. The audio visualization method according to any one of claims 1 to 6, characterized in that: After the step of controlling each of the first display areas to play the first animation effects corresponding to each of the first display areas, the method further includes: Capturing target image data including the driver's face by an image acquisition module in the vehicle; determining an emotional state parameter of the driver according to the target image data, and determining a first audio adjustment parameter and a first animation adjustment parameter based on the emotional state parameter; obtaining a fourth audio signal based on the first audio adjustment parameter and the first audio signal, and playing the fourth audio signal through a built-in speaker configured in the vehicle; The first animation effect is adjusted according to the first animation adjustment parameter to obtain a third animation effect, and each of the first display areas is controlled to play the third animation effect corresponding to the first animation effect.

9. The audio visualization method according to any one of claims 1 to 6, characterized in that: The vehicle further includes a second display screen. After the step of controlling each of the first display areas to play the first animation effects corresponding to each other, the method further includes: receiving an audio adjustment instruction sent by the second display screen; Reading a second audio adjustment parameter included in the audio adjustment instruction, and determining a second animation adjustment parameter based on the second audio adjustment parameter; obtaining a fifth audio signal based on the second audio adjustment parameter and the first audio signal, and playing the fifth audio signal through a built-in speaker configured in the vehicle; The first animation effect is adjusted according to the second animation adjustment parameter to obtain a fourth animation effect, and each of the first display areas is controlled to play the fourth animation effect corresponding to the first display area.

10. A vehicle, characterized in that: The vehicle comprises: a plurality of display screens, a plurality of speakers, an image acquisition module, a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the computer program is configured to implement the steps of the audio visualization method as described in any one of claims 1 to 9.

11. A storage medium, characterized in that: The storage medium is a computer-readable storage medium, and a computer program is stored on the storage medium. When the computer program is executed by a processor, the steps of the audio visualization method according to any one of claims 1 to 9 are implemented.

12. A computer program product, characterized in that The computer program product comprises a computer program, and when the computer program is executed by a processor, the steps of the audio visualization method according to any one of claims 1 to 9 are implemented.

Images