Multimedia content interaction method, device and equipment based on vehicle cabin and storage medium

Abstract

The invention provides a multimedia content interaction method and device based on a vehicle cabin, equipment and a storage medium, and the method comprises the steps: enabling an interface packaging platform system to obtain the content marking information in corresponding playing information in response to the starting of a video and audio mode, determining parameter information corresponding to at least one vehicle-mounted hardware module according to the content labeling information; wherein the parameter information is used for generating a corresponding control instruction in combination with the currently played content at a corresponding time node, so as to trigger the at least one corresponding vehicle-mounted hardware module to execute a corresponding action. According to the invention, for various types of cabin multimedia contents such as videos, audios and interactive games, the interactivity between a plurality of hardware modules of the vehicle and the cabin entertainment system is increased, so that a user can experience multi-sense interactive experience from multiple dimensions, and the diversified use experience of the user in a digital intelligent cabin is improved.

Description

Technical Field

[0001] The present invention relates to the technical field of vehicle intelligent cockpits, and particularly to a multimedia content interaction method, device, equipment and storage medium based on a vehicle cockpit. Background Art

[0002] With the continuous progress of technology and the increasing demand for entertainment experience in vehicle cockpits, cars are gradually evolving from simple travel tools to intelligent cockpit multimedia systems.

[0003] However, the existing intelligent cockpit multimedia systems only focus on the collaborative invocation of the atomic capabilities of the cockpit part, and cannot meet the needs of multimedia content producers for diversified content creation and editing, nor can they meet the diversified consumption experience of users for multimedia content in the cockpit, lacking immersive sensory experience. Summary of the Invention

[0004] To solve the problems existing in the prior art, the present invention provides a multimedia content interaction method, device, equipment and storage medium based on a vehicle cockpit, so as to improve the entertainment space experience of users in the vehicle intelligent cockpit from multiple dimensions.

[0005] The object of the present invention is achieved by the following technical solutions:

[0006] According to one aspect of the present invention, there is provided a multimedia content interaction method based on a vehicle cockpit, wherein the vehicle cockpit is communicatively connected to an interface encapsulation platform system, and the method includes:

[0007] In response to the start of the audio-visual mode, the interface encapsulation platform system obtains the content annotation information in the corresponding playback information, and determines the parameter information corresponding to at least one vehicle-mounted hardware module according to the content annotation information; wherein, the parameter information is used to generate corresponding control instructions in combination with the currently played content at corresponding time nodes, so as to trigger the corresponding at least one vehicle-mounted hardware module to execute corresponding actions.

[0008] According to another aspect of the present invention, there is also provided a multimedia content interaction device based on a vehicle cockpit, wherein the vehicle cockpit is communicatively connected to an interface encapsulation platform system, and the device includes:

[0009] An acquisition and sending module, configured to, in response to the start of the audio-visual mode, the interface encapsulation platform system obtains the content annotation information in the corresponding playback information, and determines the parameter information corresponding to at least one vehicle-mounted hardware module according to the content annotation information; wherein, the parameter information is used to generate corresponding control instructions in combination with the currently played content at corresponding time nodes, so as to trigger the corresponding at least one vehicle-mounted hardware module to execute corresponding actions.

[0010] According to another aspect of the present invention, there is also provided an electronic device, including a memory, a processor, and a computer program stored on the memory and executable on the processor. When the processor executes the computer program, it implements the multimedia content interaction method based on the vehicle cockpit as described in any one of the above.

[0011] According to another aspect of the present invention, there is also provided a non-transitory computer-readable storage medium, on which a computer program is stored. When the computer program is executed by a processor, it implements the multimedia content interaction method based on the vehicle cockpit as described in any one of the above.

[0012] The multimedia content interaction method, device, equipment, and storage medium based on the vehicle cockpit provided by the present invention, by responding to the start of the audio-visual mode, the interface encapsulation platform system obtains the content annotation information in the corresponding playback information, and determines the parameter information corresponding to at least one vehicle-mounted hardware module according to the content annotation information; wherein, the parameter information is used to generate corresponding control instructions in combination with the currently played content at the corresponding time node to trigger the corresponding at least one vehicle-mounted hardware module to execute corresponding actions. The present invention increases the interactivity between multiple hardware modules of the vehicle itself and the cockpit multimedia system for multi-type cockpit multimedia content such as video, audio, and interactive games, enabling users to experience multi-sensory interactive experiences from multiple dimensions, meeting users' more multimedia interaction needs, and enhancing users' diverse usage experiences in the digital intelligent cockpit. Description of the Drawings

[0013] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the following will briefly introduce the drawings required for the description of the embodiments. Obviously, the following drawings are only some embodiments of the present application. For those skilled in the art, without creative efforts, other implementation manners can also be obtained according to these drawings.

[0014] Figure 1 It is a flowchart of the content interaction method of the vehicle cockpit multimedia system provided by the embodiment of the present invention;

[0015] Figure 2 It is a schematic diagram of the logical implementation of the content interaction method of the vehicle cockpit multimedia system provided by the embodiment of the present invention;

[0016] Figure 3 It is a flowchart of the content interaction method of the vehicle cockpit multimedia system provided by another embodiment of the present invention;

[0017] Figure 4 It is a flowchart of the content interaction method of the vehicle cockpit multimedia system provided by another embodiment of the present invention;

[0018] Figure 5It is a schematic structural diagram of an electronic device provided by an embodiment of the present invention. Detailed implementation manners

[0019] The above description is only an overview of the technical solution of the present invention. In order to be able to understand the technical means of the present invention more clearly, it can be implemented according to the content of the specification. And in order to make the above and other purposes, features and advantages of the present invention more obvious and understandable, the following specifically gives preferred embodiments and, in conjunction with the drawings, details are described as follows.

[0020] The terms "first", "second", etc. in the specification and claims are used to distinguish between similar elements and are not necessarily used to describe a specific order or time sequence. It is to be understood that, where appropriate, these terms so used may be interchanged, for example, so that the embodiments of the present invention herein can be operated in an order different from that herein or shown. Similarly, if the method herein includes a series of steps, and the order of these steps presented herein is not necessarily the only order in which these steps can be executed, and some of the steps may be omitted and / or some other steps not described herein may be added to the method. If the components in a certain drawing are the same as those in other drawings, although these components can be easily recognized in all the drawings, for the sake of clearer description of the drawings, the present specification will not label the reference numerals of all the same components in each drawing.

[0021] Embodiment 1

[0022] Referring to Figure 1 As shown, this embodiment provides a multimedia content interaction method based on a vehicle cockpit. The vehicle cockpit is communicatively connected to an interface encapsulation platform system. The method includes:

[0023] In response to the start of the audio-visual mode, the interface encapsulation platform system obtains the content annotation information in the corresponding playback information, and determines the parameter information corresponding to at least one in-vehicle hardware module according to the content annotation information, where the parameter information is used to generate corresponding control instructions in combination with the currently played content at corresponding time nodes to trigger the corresponding at least one in-vehicle hardware module to perform corresponding actions.

[0024] The playback information corresponds to the relevant information of the multimedia content. The multimedia content is, for example, entertainment content to improve the user experience, such as including videos, audios, and interactive games.

[0025] It should be understood that in the embodiments of the present invention, the audio - video mode of the vehicle cockpit should be carried out when the vehicle is in a parked state. That is, when the vehicle is in a stationary state, such as when the vehicle speed is zero, the audio - video mode of the vehicle cockpit can be triggered. The start button can be a virtual button or a physical button, which is not limited herein. The user of the vehicle can also issue control instructions by voice. On the power - on display interface of the terminal that provides multimedia content consumption in the vehicle cockpit, application programs such as "Game", "Video", "Music" are displayed. When the user issues a voice message such as "Start the game", after the sound collection device set in the vehicle computer collects this voice message, the control instruction of "Start the game" can be extracted.

[0026] The content annotation information is characterized in that it can correspond to the obtained multimedia content, and it is hoped that an immersive interactive information can be created through in - vehicle hardware. The content annotation information usually can include time - node information and interactive information of the multimedia content matching the event node.

[0027] Exemplarily, if the multimedia content currently played in the cockpit is a video file, the content annotation information can be the weather conditions (rainy, lightning, vibration), environmental special effects (smoke, blowing wind, earthquake or volcanic eruption), motion special effects (racing, flying), etc. that the cockpit is expected to simulate corresponding to the picture of the video file; extracting the content annotation information of the video file also includes picture color, picture brightness, etc. Those skilled in the art can understand that the video file can be a movie picture, a cut - scene animation of an interactive game, a picture of an interactive game, or other equivalent digital content that can present pictures.

[0028] If the multimedia content played in the cockpit is an audio file, extract the content annotation information of the audio file, such as the lyric information of the audio file or audio segments (such as the chorus segment of a song, wonderful segments of cross - talk or storytelling, etc.); extracting the content annotation information of the audio file also includes volume, frequency, music genre, rhythm, etc., so that the cockpit can create a corresponding listening environment.

[0029] Exemplarily, the interface encapsulation platform system obtains the content annotation information in the corresponding playback information including one or more of the following steps:

[0030] Obtain the content annotation information provided by the content producer of the playback information;

[0031] Obtain the content annotation information set by the user;

[0032] Obtain the content annotation information output by the trained scene analysis model in response to the playback information.

[0033] Specifically, the content annotation information can be pre-annotated by content producers during the creation and editing of multimedia content, and can be the desired interactive content expressed in natural language. Or users can create content annotation information according to their own preferences based on different multimedia content. Or the interface encapsulation platform system can analyze the multimedia content to be played based on a trained scene analysis model, automatically mark points, and confirm the content annotation information.

[0034] As Figure 2 shown, a unified interface encapsulation platform system that is easy for content producers to understand is defined. Content producers can create content annotation information for multimedia content according to the unified interface encapsulation standard. Again, as Figure 2 shown in the embodiments described, the playback information can be stored in the cloud so that the interface encapsulation platform system can obtain the playback information in response to the opening of the theater mode.

[0035] The interface encapsulation platform system is a system based on the atomic capabilities of the vehicle cockpit. Exemplarily, the interface encapsulation platform system includes a packaging module and a parsing module. The packaging module has a unified interface that communicates with all in-vehicle hardware modules to obtain the parameters that all in-vehicle hardware modules can respond to. The parsing module is used to parse the received parameter information and generate corresponding control instructions in real time in combination with the currently played content at the corresponding time node.

[0036] Specifically, based on the packaging module of the interface encapsulation platform, the situation of the in-vehicle hardware modules that can be used for multimedia content interaction in the cockpit can be obtained. In one embodiment, for cockpit safety considerations, the situation of the in-vehicle hardware modules also includes the reachable parameters that the in-vehicle hardware modules can respond to. Thus, among the parameters that can be responded to, the content annotation information is converted into the in-vehicle hardware modules that can be responded to and their parameter information.

[0037] Furthermore, according to this encapsulation platform, the in-vehicle hardware modules that can be responded to and their reachable parameters that can be responded to can also be provided to content producers or users so that content producers or users can customize and select the cockpit hardware capabilities that they hope to interact with the multimedia content within a safe range.

[0038] Based on the interface packaging platform system, it is possible to read the content annotation information in the playback information, and then convert the obtained content annotation information into parameter information corresponding to at least one vehicle-mounted hardware module and send it to the corresponding vehicle-mounted hardware module. For example, the interface packaging platform system can obtain the corresponding playback information, and then when interpreting the file content of the corresponding playback information, this file content will be accompanied by a script (content annotation information) of this file, and the script can mark what kind of interaction this video wants to produce in the vehicle cabin at which time point. After parsing this content annotation information, the interface packaging platform system will determine the parameter information of which vehicle-mounted component (vehicle-mounted hardware module) in the vehicle cabin will perform what kind of action at which time point.

[0039] Figure 3 It is a flowchart of a content interaction method of a vehicle cockpit multimedia system provided by another embodiment of the present invention.

[0040] Furthermore, if Figure 3 As shown, in some embodiments, in the step of parsing the received parameter information and generating corresponding control instructions in real time in combination with the currently played content at the corresponding time node, the method further includes:

[0041] Acquire vehicle interior and exterior environment data, and generate corresponding control instructions at corresponding time nodes based on the result of parsing the parameter information and combining the vehicle interior and exterior environment data.

[0042] The vehicle environment data is obtained from one or more sensors provided in the vehicle, such as GPS data, temperature data inside and outside the vehicle, weather data, air pressure data, etc.

[0043] Figure 4 It is a flow chart of a content interaction method of a vehicle cockpit multimedia system provided by another embodiment of the present invention.

[0044] Furthermore, if Figure 4 As shown, in some embodiments, in the step of parsing the received parameter information and generating corresponding control instructions in real time in combination with the currently played content at the corresponding time node, the method further includes:

[0045] The user status data is obtained, and based on the result of parsing the parameter information and the user status data, a corresponding control instruction is generated at a corresponding time node.

[0046] The camera system or microphone system in the cockpit is used to collect the user's facial expressions and voice to analyze the user's status and obtain user status data, such as emotions, attention and sitting posture.

[0047] Specifically, the result of parsing parameter information, vehicle environment data, and / or user status data are jointly converted into an execution sequence to generate corresponding control instructions.

[0048] Specifically, corresponding simulation control instructions can be generated based on the result of parsing parameter information in combination with vehicle environment data and / or user status data, so as to control at least one hardware module of the vehicle to perform corresponding actions. The hardware module can be understood as various possible forms of controls, devices, or components directly or indirectly related to providing entertainment functions installed inside and / or outside the vehicle, including but not limited to air conditioning systems, fragrance systems, suspension systems, ambient light systems, seat systems, audio systems, steer-by-wire systems, and augmented reality / virtual reality systems, etc.

[0049] According to a preferred example of the present invention, the content annotation information includes at least one response mode information, and the response mode information is configured to adjust the response mode of the multimedia content and the in-vehicle hardware module.

[0050] That is, multiple audio-visual content and in-vehicle hardware response modes are preset in the system, such as soothing / normal / vigorous, etc. The response mode between the multimedia content and the in-vehicle hardware module can be adjusted according to user preferences. Or according to the multimedia content annotation information corresponding to each time node and the real-time analysis result combined with the multimedia content, the response mode of the multimedia content and the in-vehicle hardware is adjusted.

[0051] The multimedia content interaction method based on the vehicle cockpit provided by the embodiments of the present invention increases the interactivity between multiple hardware modules of the vehicle itself and the cockpit entertainment system for various types of cockpit multimedia content such as video, audio, and interactive games, thereby realizing the linkage between the multimedia content and the vehicle, directly associating the user's perception from multiple senses such as vision, hearing, smell, and touch, enabling the user to experience a multi-sensory interaction experience from multiple dimensions, meeting the user's more entertainment interaction needs, and enhancing the user's diverse usage experience in the digital intelligent cockpit.

[0052] In a preferred example, the parameter information is used to generate corresponding control instructions in combination with the currently played content at the corresponding time node to trigger the at least one in-vehicle hardware module to perform corresponding actions, including:

[0053] In the case where the control instruction includes controlling the suspension system, based on the control instruction, the suspension system is triggered to execute corresponding action information, where the action information includes at least one or more of response intensity, response amplitude, response times, and response frequency in the vertical or horizontal direction.

[0054] Among them, the control instruction for controlling the suspension system is generated by annotating the picture change information and sound change information of each time node in the multimedia content. The picture change information is generated by at least one of a picture intense event, a horizontal change in the picture, and a vertical change in the picture. The sound change information is generated by at least one of a sound azimuth, a volume change, a rhythm change, and a style change.

[0055] A preferred example is to generate a simulated control instruction for controlling the suspension system according to the result of parsing the parameter information corresponding to the suspension system in combination with the emotional state information in the user state data, and send it to the suspension system, so that the suspension system can perform at least one or more adjustments including response intensity, response amplitude, response frequency, and response number in the vertical or horizontal direction. Among them, the parameter information corresponding to the suspension system is generated by annotating the picture change information and sound change information of each time node in the multimedia content. The picture change information is generated by at least one of a picture intense event, a horizontal change in the picture, and a vertical change in the picture. The sound change information is generated by at least one of a sound azimuth, a volume change, a rhythm change, and a style change. The emotional state information is obtained by analyzing the voice input by the current user through a voice interaction system.

[0056] Specifically, the parameter information of the suspension system is generated by annotating the picture change information of each time node in the multimedia content. The picture change information includes: when the video picture or game picture presented in the multimedia content changes in the horizontal direction and / or the vertical direction, such as when an airplane turns or ascends and descends, etc., control the combined movement of the suspensions in different directions to create a spatial change in the horizontal and / or vertical direction; or when there is an intense event such as an explosion in the video picture, control the suspension to make a corresponding response, causing the vehicle body to shake, and simulate various actions through the suspension to increase the interactive experience.

[0057] Generate a control instruction according to the parameter information corresponding to the suspension system to control the suspension system to push and pull the wheels, and it can also enable the user to experience an effect similar to knocking in the cockpit. For example, the knocking effect can be felt at each wheel angle. One wheel can be knocked alone, or multiple wheels can be knocked simultaneously. The knocking intensity and the number of knocks can be set according to requirements.

[0058] Generate control instructions according to the parameter information corresponding to the suspension system to control the suspension system to lift, lower or tilt, so that the user can experience the effect of vehicle body shaking in the cockpit. The shaking can be back and forth or left and right, and the duration and frequency of the shaking period can be set. The user can also experience the effects of vehicle nodding, raising the head, and vehicle body tilting in the cockpit. The tilting angle, the action time of nodding and raising the head (the time from the starting position to the triggered target position), and the duration can be set. The user can also experience the effects of jumping and squatting in the cockpit. The amplitude of jumping and squatting, as well as the action time and duration, can be set.

[0059] It is also possible to generate parameter information corresponding to the suspension system by annotating the sound change information at each time node in the multimedia content. For example, different azimuth suspensions can be called according to the sound in different azimuths (front, rear, left, right, middle, etc.), and the instantaneous action of the suspension can respond to the sound in different azimuths. Or the action amplitude range of the suspension can be called according to the volume change to respond to the volume change. Or the response frequency of the suspension can be adjusted according to the rhythm change of the sound. Or according to the change of the sound style, the suspension change can be weighted. For example, for the soothing country music, the number of suspension responses and / or the amplitude can be controlled to decrease accordingly, while for rock and electronic music, etc., it can be increased accordingly.

[0060] It is also possible to generate control instructions corresponding to the suspension system by combining the emotional state information in the user state data. Specifically, through the voice interaction system, by collecting the user's voice information, the emotional state information of the current user (such as happy, excited, calm, low, etc.) can be analyzed according to the user's voice information. For example, when the user is in a low mood, the response intensity and / or frequency of the suspension can be reduced.

[0061] It should be noted that the actuator of the suspension system provided in the embodiment of the present invention is an independent system at each corner. Among them, the motor can respond quickly to obtain the lifting and lowering of the target height. In some embodiments, the function of raising the vehicle body can also be realized. For example, the vehicle body shakes 3 - 4 times, and the shaking period is 1.5 s per time. For example, the direction of the suspension shaking and the shaking period can be defined. In some embodiments, the functions of nodding / raising the head and vehicle body roll can also be realized by using the suspension system. For example, the tilting angle of the suspension, the action time, and the duration can be defined. In some embodiments, the functions of (lowering the vehicle body) jumping and squatting can also be realized by using the suspension system. For example, the up and down height of the suspension, the action time, and the duration can be defined.

[0062] Exemplarily, the actuator of the suspension system is an integrated mechanical - hydraulic - pneumatic - electric structure, which can realize the instantaneous adjustment of the vehicle body posture in a short time (1 ms). For example, the suspension can be lowered or raised to the target stroke within 0.5 s to complete the response to the target demand, so as to complete the real sensory experiences such as vehicle shaking, jittering, and impact.

[0063] A preferred example is that the parameter information is used to generate corresponding control instructions in combination with the currently played content at corresponding time nodes, so as to trigger the corresponding at least one vehicle-mounted hardware module to execute corresponding actions, including:

[0064] In the case where the control instruction includes the air-conditioning system, the air-conditioning system is triggered based on the control instruction to execute corresponding adjustment information, where the adjustment information includes at least one or more of ambient temperature, blowing intensity, and blowing angle.

[0065] Among them, the air-conditioning parameter information corresponding to the control instruction is generated by annotating the ambient temperature information, air volume information, and wind direction information at each time node in the multimedia content.

[0066] Generate a simulated control instruction according to the parameter information of the air-conditioning system and / or the ambient temperature information inside and outside the vehicle of the vehicle environment data, and send it to the air-conditioning system, so that the air-conditioning system performs at least one or more adjustments including ambient temperature, blowing intensity, and blowing angle. Among them, the control instruction of the air-conditioning system is generated by annotating the ambient temperature information, air volume information, and wind direction information at each time node in the multimedia content.

[0067] Specifically, according to the ambient temperature information of the scene presented at each time node of the video file, control the air-conditioning system to simulate the temperature of the scene, or according to the air volume information and / or wind direction information of the scene presented at each time node of the video file, control the air-conditioning system to adjust the blowing intensity and / or blowing angle to simulate the scene.

[0068] It is also possible to combine the ambient temperature information inside and outside the vehicle of the vehicle environment data for weighting to control the change intensity of the air-conditioning. For example, when the outside ambient temperature is low, the temperature change range of the air-conditioning is correspondingly reduced, and when the outside ambient temperature is high, the temperature change range of the air-conditioning is correspondingly increased to enhance the user's tactile experience.

[0069] The air outlet modes of the air-conditioning system include: normal air outlet, strong cooling, and strong heating; the wind direction modes include: symmetric, free, sweeping, concentrated, dispersed, intelligent, automatic, face blowing, and foot blowing; the air volume adjustment is a stepped adjustment; the temperature adjustment is automatic or stepped adjustment.

[0070] A preferred example is that the parameter information is used to generate corresponding control instructions in combination with the currently played content at corresponding time nodes, so as to trigger the corresponding at least one vehicle-mounted hardware module to execute corresponding actions, including:

[0071] In the case where the control instruction includes the ambient light system, trigger the ambient light system to execute the corresponding adjustment information based on the control instruction, where the adjustment information includes at least one or more of brightness, color, and blinking frequency.

[0072] Among them, the parameter information of the ambient light system corresponding to the control instruction is obtained by analyzing the video information and audio information at each time point in the multimedia content. The video information includes picture brightness information, picture color information, and picture change frequency, and the audio information includes volume, frequency, music style, and rhythm.

[0073] Generate a simulated control instruction according to the parameter information of the ambient light system and the light intensity information inside and outside the vehicle in the vehicle environment data, and send it to the ambient light system, so that the ambient light system performs at least one or more adjustments including brightness, color, and blinking frequency. Among them, the parameter information of the ambient light system is obtained by analyzing the video information and audio information at each time point in the multimedia content. The video information includes picture brightness information, picture color information, and picture change frequency, and the audio information includes volume, frequency, music style, and rhythm.

[0074] Specifically, according to the brightness, color, change frequency, etc. presented by the pictures at each time node of the video file, control the response of the ambient light in terms of brightness, color, and blinking frequency, or control the response of the ambient light according to the sound volume, rhythm, style, etc. of each time node of the video file, and enhance the visual experience by adjusting the blinking frequency, color, and brightness of the ambient light.

[0075] It is also possible to combine the light intensity information inside and outside the vehicle in the vehicle environment data for weighting to adjust the brightness of the ambient light.

[0076] A preferred example is that the parameter information is used to generate a corresponding control instruction in combination with the currently played content at the corresponding time node to trigger the corresponding at least one in-vehicle hardware module to execute the corresponding action, including: in the case where the control instruction includes the fragrance system, trigger the fragrance system to execute the corresponding adjustment information based on the control instruction, where the adjustment information includes at least releasing corresponding fragrances according to different environmental scenes presented by the multimedia content.

[0077] Specifically, according to the environmental scenes presented by the pictures at each time node of the video file, such as mountain streams, dining in a restaurant, social dance parties, etc., control the fragrance system to release corresponding fragrances according to different environmental scenes presented by the video pictures to enhance the olfactory experience.

[0078] A preferred example is that the parameter information is used to generate a corresponding control instruction in combination with the currently played content at the corresponding time node to trigger the corresponding at least one in-vehicle hardware module to execute the corresponding action, including:

[0079] In the case where the control instruction includes an audio system, trigger the audio system to play by adjusting the audio in different directions in the cockpit based on the control instruction, so as to change the sound direction information.

[0080] Specifically, play the sounds in different directions in the multimedia content through the audio in different directions, so as to provide an immersive sound experience and enhance the stereophonic auditory feeling.

[0081] Based on the sound direction information and the noise information in the vehicle environment data, include the sound direction information and the noise information in the simulated control instruction and send it to the audio system, so that the audio system filters the noise information and plays the sound direction information through the audio in different directions in the cockpit.

[0082] That is, collect the noise information in the vehicle environment data that affects the playback experience through the sound collection device, and filter and eliminate the noise information to achieve the noise reduction playback function.

[0083] A preferred example is that the parameter information is used to generate corresponding control instructions in combination with the currently played content at corresponding time nodes, so as to trigger the at least one in-vehicle hardware module to perform corresponding actions, including:

[0084] In the case where the control instruction includes a seat system, trigger the seat system to perform at least one or more adjustments including massage direction, massage intensity, heating amplitude, and ventilation intensity according to different environmental scenes presented in the multimedia content based on the control instruction.

[0085] Among them, the parameter information controlled by the seat system is generated by annotating the scene information presented in the video content and / or audio content of each time node in the multimedia content.

[0086] A preferred example is to generate corresponding simulated control instructions based on the parameter information controlled by the seat system and the seat pressure information in the user status data, and send them to the seat system, so that the seat system performs at least one or more adjustments including massage direction, massage intensity, heating amplitude, and ventilation intensity according to different environmental scenes presented in the multimedia content. Among them, the parameter information controlled by the seat system is generated by annotating the scene information presented in the video content and / or audio content of each time node in the multimedia content, and the seat pressure information is obtained by the seat perception system to obtain the pressure value of the current user on the seat.

[0087] Specifically, when the video file being played presents a scene of driving on a rough road, the seat massage in different directions is controlled according to the different road conditions at each time node of the video; when the video file presents a scene of a scorching summer day, the seat ventilation function is enabled; when the video file presents a scene of a freezing winter day, the seat heating function is enabled. The pressure value of the current user on the seat can also be obtained through the seat sensing system, and this pressure value is weighted to control the intensities of seat massage, heating, and ventilation, so as to enhance the user's immersive experience.

[0088] In a preferred example, the parameter information is used to generate corresponding control instructions in combination with the currently played content at corresponding time nodes, so as to trigger the corresponding at least one in-vehicle hardware module to perform corresponding actions, including:

[0089] In the case where the control instruction includes a steer-by-wire system, based on the control instruction, trigger the steer-by-wire system to perform control of at least one of the steering angle, steering force, and steering speed when the direction of the character's perspective changes in the interactive game.

[0090] Among them, the parameter information of the steer-by-wire control system is generated by annotating the direction changes of the character's perspective generated at each time node in the interactive game.

[0091] Specifically, when the multimedia content being played is a game, the angle, force, and speed of the steering wheel are simulated according to the direction change of the character's perspective in the game screen. Especially in racing games, by controlling the steering wheel to simulate driving, the experience is made closer to the real environment to enhance the game experience. It should be noted that by controlling the steer-by-wire system for simulation, the steering wheel can be separated from the tire steering mechanism, making the game safer and reducing the friction between the vehicle tires and the ground at the same time.

[0092] In this embodiment, the multimedia content of the vehicle cockpit can also be set in the form of an APP program on the mobile terminal. The mobile terminal can remotely control the vehicle. The user's mobile device (mobile phone, tablet computer) enables the multimedia content of the vehicle cockpit and obtains the currently played multimedia content on the mobile device. By operating on the user's mobile device, the vehicle makes corresponding responses according to the multimedia content (audio, video, and games), such as calling the suspension for response to achieve vehicle jitter, swaying, etc.

[0093] An external viewing device can also be provided. Among them, the external viewing device is connected to the audio-visual mode of the vehicle cockpit. For example: VR head-mounted devices or AR glasses, etc. This embodiment does not make specific limitations on this.

[0094] It should be noted that for the multimedia content interaction method based on a vehicle cockpit provided in the embodiments of the present invention, the execution subject may be a control device of a vehicle cockpit entertainment system, or a control module in the control device of the vehicle cockpit entertainment system for executing the multimedia content interaction method based on the vehicle cockpit.

[0095] On the other hand, the present invention also provides a multimedia content interaction device based on a vehicle cockpit. The vehicle cockpit is communicatively connected to an interface encapsulation platform system. The device includes:

[0096] An acquisition and sending module, configured to, in response to the start of a video and audio mode, acquire content annotation information in corresponding playback information by the interface encapsulation platform system, and determine parameter information corresponding to at least one in-vehicle hardware module according to the content annotation information; wherein the parameter information is used to generate corresponding control instructions in combination with the currently played content at corresponding time nodes to trigger the at least one corresponding in-vehicle hardware module to perform corresponding actions.

[0097] For the specific implementation processes of the functions and roles of each module in the above device, please refer to the implementation processes of the corresponding steps in the above method. Therefore, for the relevant parts, please refer to the description in the method embodiments, and details will not be repeated here.

[0098] The device embodiments described above are merely illustrative. For example, the division of the modules is only a logical function division. In actual implementation, there may be other division methods. In the embodiments, all the functional modules may be integrated in one processor, or each module may be separately used as a device, or two or more modules may be integrated in one device; each functional module in the embodiments may be implemented in the form of hardware, or in the form of a combination of hardware and software functional units.

[0099] Referring to Figure 5 As shown, on the other hand, the present invention also provides an electronic device, which includes: a processor 310, a communication interface 320, a memory 330, and a communication bus 340. Among them, the processor 310, the communication interface 320, and the memory 330 communicate with each other through the communication bus 340. The processor 310 can call logical instructions in the memory 330, and the processor 310 executes the multimedia content interaction method based on the vehicle cockpit as described in Method Embodiment 1. The method includes:

[0100] In response to the activation of the audio-visual mode, the interface encapsulation platform system obtains the content annotation information in the corresponding playback information, and determines the parameter information corresponding to at least one in-vehicle hardware module according to the content annotation information; wherein, the parameter information is used to generate corresponding control instructions in combination with the currently played content at corresponding time nodes, so as to trigger the corresponding at least one in-vehicle hardware module to perform corresponding actions.

[0101] In addition, when the logic instructions in the above-mentioned memory 330 are implemented in the form of software functional units and sold or used as independent products, they can be stored in a computer-readable storage medium. Based on such an understanding, the technical solution of the present invention, in essence, or the part that contributes to the prior art, or a part of this technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to enable a computer device (which can be a personal computer, a server, or a network device, etc.) to execute all or part of the steps of the methods described in various embodiments of the present invention. The aforementioned storage medium includes: various media such as USB flash drives, mobile hard disks, read-only memory (ROM, Read-Only Memory), random access memory (RAM, Random Access Memory), magnetic disks, or optical discs that can store program codes.

[0102] Another aspect of the present invention also provides a non-transitory computer-readable storage medium, on which a computer program is stored. When the computer program is executed by a processor, it implements the multimedia content interaction method based on a vehicle cockpit as described in any one of the above. The method includes:

[0103] In response to the activation of the audio-visual mode, the interface encapsulation platform system obtains the content annotation information in the corresponding playback information, and determines the parameter information corresponding to at least one in-vehicle hardware module according to the content annotation information; wherein, the parameter information is used to generate corresponding control instructions in combination with the currently played content at corresponding time nodes, so as to trigger the corresponding at least one in-vehicle hardware module to perform corresponding actions.

[0104] The devices and media provided in the embodiments of the present invention correspond one-to-one with the methods. Therefore, the devices and media also have beneficial technical effects similar to those of their corresponding methods. Since the beneficial technical effects of the methods have been described in detail above, the beneficial technical effects of the devices and media will not be repeated here.

[0105] The technical features of the above-described embodiments can be combined arbitrarily. For the sake of brevity of description, not all possible combinations of the technical features in the above-described embodiments are described. However, as long as the combinations of these technical features do not conflict, they should be considered as falling within the scope described in this specification.

[0106] The embodiments described above merely represent several implementation manners of the present application. The description is relatively specific and detailed, but it should not be construed as a limitation on the scope of the patent application. It should be noted that for those of ordinary skill in the art, without departing from the concept of the present application, several modifications and improvements can still be made, and these all fall within the protection scope of the present application. Therefore, the protection scope of the patent of the present application shall be subject to the appended claims.

Claims

1. A multimedia content interaction method based on a vehicle cockpit, characterized in that, The vehicle cockpit is communicatively connected to the interface encapsulation platform system, and the method includes: In response to the start of the audio-visual mode, The interface encapsulation platform system obtains the content annotation information in the corresponding playback information, and determines the parameter information corresponding to at least one in-vehicle hardware module according to the content annotation information; wherein, the parameter information is used to generate corresponding control instructions in combination with the currently played content at corresponding time nodes to trigger the corresponding at least one in-vehicle hardware module to perform corresponding actions.

2. The method according to claim 1, wherein The interface encapsulation platform system obtaining the content annotation information in the corresponding playback information includes one or more of the following steps: Obtaining the content annotation information provided by the content producer of the playback information; Obtaining the content annotation information set by the user; Obtaining the content annotation information output by the trained scene analysis model in response to the playback information.

3. The method according to claim 1, wherein The interface encapsulation platform system includes an encapsulation module and a parsing module. The steps for the interface encapsulation platform system to obtain the content annotation information in the corresponding playback information and determine the parameter information corresponding to at least one in-vehicle hardware module include: The encapsulation module has a unified interface communicatively connected to all in-vehicle hardware modules for obtaining the parameters that all in-vehicle hardware modules can respond to; The parsing module is used to parse the received parameter information and generate corresponding control instructions in real time in combination with the currently played content at corresponding time nodes.

4. The method according to claim 3, characterized in that, In the step of parsing the received parameter information and generating corresponding control instructions in real time in combination with the currently played content at corresponding time nodes, the method further includes: Obtaining the vehicle interior and exterior environment data, and generating corresponding control instructions at corresponding time nodes based on the result of parsing the parameter information in combination with the vehicle interior and exterior environment data.

5. The method according to claim 3 or 4, characterized in that In the step of parsing the received parameter information and generating corresponding control instructions in real time in combination with the currently played content at corresponding time nodes, the method further includes: Obtaining the user status data, and generating corresponding control instructions at corresponding time nodes based on the result of parsing the parameter information in combination with the user status data.

6. The method according to claim 1, characterized in that, The parameter information being used to generate corresponding control instructions in combination with the currently played content at corresponding time nodes to trigger the corresponding at least one in-vehicle hardware module to perform corresponding actions includes: In the case where the control instruction includes controlling the suspension system, triggering the suspension system to perform corresponding action information based on the control instruction, wherein the action information includes at least one or more of the response intensity, response amplitude, response times, and response frequency in the vertical or horizontal direction.

7. The method according to claim 1, wherein The parameter information being used to generate corresponding control instructions in combination with the currently played content at corresponding time nodes to trigger the corresponding at least one in-vehicle hardware module to perform corresponding actions includes: In the case where the control instruction includes the air conditioning system, triggering the air conditioning system to perform corresponding adjustment information based on the control instruction, wherein the adjustment information includes at least one or more of the ambient temperature, blowing intensity, and blowing angle.

8. The method according to claim 1, wherein The parameter information is used to generate corresponding control instructions in combination with the currently played content at corresponding time nodes, so as to trigger the corresponding at least one vehicle-mounted hardware module to execute corresponding actions, including: In the case where the control instruction includes an ambient light system, based on the control instruction, the ambient light system is triggered to execute corresponding adjustment information, where the adjustment information at least includes one or more of brightness, color, and blinking frequency.

9. The method according to claim 1, characterized in that, The parameter information is used to generate corresponding control instructions in combination with the currently played content at corresponding time nodes, so as to trigger the corresponding at least one vehicle-mounted hardware module to execute corresponding actions, including: In the case where the control instruction includes an aroma system, based on the control instruction, the aroma system is triggered to execute corresponding adjustment information, where the adjustment information at least includes corresponding aroma release according to different environmental scenarios presented in the multimedia content.

10. The method according to claim 1, characterized in that, The parameter information is used to generate corresponding control instructions in combination with the currently played content at corresponding time nodes, so as to trigger the corresponding at least one vehicle-mounted hardware module to execute corresponding actions, including: In the case where the control instruction includes an audio system, based on the control instruction, the audio system is triggered to play by adjusting the speakers at different positions in the cockpit to change the sound azimuth information.

11. The method according to claim 1, wherein The parameter information is used to generate corresponding control instructions in combination with the currently played content at corresponding time nodes, so as to trigger the corresponding at least one vehicle-mounted hardware module to execute corresponding actions, including: In the case where the control instruction includes a seat system, based on the control instruction, the seat system is triggered to perform at least one or more adjustments including massage azimuth, massage intensity, heating amplitude, and ventilation intensity according to different environmental scenarios presented in the multimedia content.

12. The method according to claim 1, wherein The parameter information is used to generate corresponding control instructions in combination with the currently played content at corresponding time nodes, so as to trigger the corresponding at least one vehicle-mounted hardware module to execute corresponding actions, including: In the case where the control instruction includes a steer-by-wire system, based on the control instruction, the steer-by-wire system is triggered to perform at least one or more controls including steering angle, steering force, and steering speed when the direction of the character's perspective changes in the interactive game.

13. A multimedia content interaction device based on a vehicle cockpit, characterized in that, The vehicle cockpit is communicatively connected to the interface encapsulation platform system, and the device includes: An acquisition and sending module, configured to, in response to the start of the audio-visual mode, the interface encapsulation platform system acquires the content annotation information in the corresponding playback information, and determines the parameter information corresponding to at least one vehicle-mounted hardware module according to the content annotation information; where the parameter information is used to generate corresponding control instructions in combination with the currently played content at corresponding time nodes, so as to trigger the corresponding at least one vehicle-mounted hardware module to execute corresponding actions.

14. An electronic device, comprising a memory, a processor, and a computer program stored on the memory and executable on the processor, characterized in that, When the processor executes the computer program, it implements the multimedia content interaction method based on the vehicle cockpit according to any one of claims 1-12.

15. A non-transitory computer-readable storage medium storing a computer program thereon, characterized in that, When the computer program is executed by the processor, it implements the multimedia content interaction method based on the vehicle cockpit according to any one of claims 1-12.

Images