Cockpit control method, electronic system, and vehicle

Abstract

Provided are a cockpit control method, an electronic system, and a vehicle, relating to the technical field of electronic control, allowing a user to experience media resources more realistically in dimensions such as a visual dimension and an auditory dimension within a cockpit, thereby satisfying diverse experience demands of users. In the method, a vehicle can acquire pose information corresponding to different poses of a user, and an image and an audio in media resources are rendered on the basis of the currently acquired pose information; and when the pose of the user changes, both the rendering of the image and the rendering of the audio can be changed, so that the display effect of the image and the audio can also be changed. A head-mounted device connected to the vehicle can present for the user a dynamic image 3D visual effect changing along with the pose of the user, and a plurality of loudspeakers in the cockpit can present for the user a dynamic and immersive auditory effects changing along with the pose of the user. The user can experience the media resources from the aspects such as a visual aspect and an auditory aspect within the cockpit, enabling the user to have better feeling, and satisfying diverse experience demands of the user.

Description

Cockpit control method, electronic system and vehicle Technical Field

[0001] The present application relates to the field of electronic control technology, and in particular to a cockpit control method, an electronic system, and a vehicle. Background Art

[0002] With the rapid development of smart cars in recent years, users have increasingly demanded higher functionality from them. For example, users not only expect smart cars to have excellent driving performance, but also to provide a better entertainment experience.

[0003] Currently, some smart cars' audio and video systems, display systems, or autonomous driving systems can be connected to head-mounted devices such as virtual reality (VR) and augmented reality (AR) devices, providing users with a high-definition, large-screen viewing experience in the car's cabin. Alternatively, in autonomous driving scenarios, users can be provided with multi-dimensional virtual driving scene images, making them feel as if they are in the driving situation.

[0004] However, the movie-watching experience or virtual scene experience provided above are relatively single or simple in function and cannot meet the user's various experience needs.

[0005] Summary of the Invention

[0006] The embodiments of the present application provide a cockpit control method, an electronic system, and a vehicle, which can enable users to experience media resources more realistically in the cockpit from visual and auditory dimensions, thereby meeting the user's various experience needs.

[0007] To achieve the above objectives, the embodiments of the present application adopt the following technical solutions:

[0008] In a first aspect, a cockpit control method is provided for a vehicle, the vehicle comprising at least two speakers disposed at different locations within the cockpit, and the vehicle being connected to a head-mounted device. In this method, the vehicle obtains media resources and first-position information of a user within the cockpit, wherein the media resources include an image and audio corresponding to the image. The vehicle then transmits an image including a first display effect to the head-mounted device based on the first-position information, wherein the first display effect is a three-dimensional effect derived from the first-position information. Based on the first-position information, the vehicle controls each speaker to play audio corresponding to the speaker, wherein the audio corresponding to each speaker is derived from the audio in the media resources, the speaker's attribute information, and the first-position information. The vehicle then obtains second-position information of the user within the cockpit and, based on the second-position information, transmits an image including a second display effect to the head-mounted device, wherein the second display effect is a three-dimensional effect derived from the second-position information. The vehicle then controls each speaker to play audio corresponding to the speaker based on the second-position information, wherein the audio corresponding to each speaker is derived from the audio in the media resources, the speaker's attribute information, and the second-position information.

[0009] In the above method, the first posture information and the second posture information can represent the information corresponding to the user in different postures in the cabin. The vehicle can send an image corresponding to or matching the user's current posture to the head-mounted device, and control the speaker to play audio corresponding to or matching the user's current posture. Then, the head-mounted device can present a dynamic three-dimensional visual effect that changes with the user's posture to the user. The multiple speakers in the cabin can present a dynamic and immersive auditory effect that changes with the user's posture to the user. The user can experience media resources from visual and auditory aspects in the cabin, giving the user a better experience and meeting the user's various experience needs.

[0010] In one implementation of the first aspect, when a vehicle obtains media resources, it may obtain the media resources from the vehicle's storage space or from a server. Furthermore, the media resources include two-dimensional media resources, three-dimensional media resources, panoramic media resources, VR media resources, or AR media resources. Thus, the vehicle can provide users with different media resource experiences.

[0011] In one implementation of the first aspect, when a vehicle obtains first-position information of a user in the cabin, it may obtain first information sent by a head-mounted device, where the first information is obtained by the head-mounted device via at least one sensor and includes the user's head posture information. Furthermore, the vehicle obtains second information via at least one sensor in the cabin, where the second information includes one or more of the following: image information of the user, infrared information, seat position information of the user's seat, seat back angle information, seat angle information, or seat pressure information. The vehicle then determines the first-position information based on the first and second information.

[0012] In the above implementation, the vehicle can obtain information collected by multiple sensors through multiple information input channels, and combine the information from multiple sensors to more accurately determine the user's posture information.

[0013] In one possible implementation of the first aspect, the second display effect is a modified version of the first display effect, and the modified display effect includes a change in one or more of the image viewing angle, image content, or image size. This allows the head-mounted device to present richer image display effects to the user that vary with the user's posture.

[0014] In one possible implementation of the first aspect, the audio corresponding to each speaker changes based on changes in user posture information, and the audio changes include changes in one or more of the following: audio frequency, audio amplitude, audio delay, audio phase, or audio spectrum characteristics. This allows the multiple speakers in the cabin to present a richer sound experience that changes with the user's posture.

[0015] In one possible implementation of the first aspect, a vehicle obtains image information from a media resource, where the image information includes one or more of seasonal information, weather information, temperature information, location information, scene information, or event information. Furthermore, the vehicle controls functional components within the cabin to provide functions that match the image information, where the functional components include one or more of a seat, an onboard air conditioner, an onboard air humidifier, an onboard fragrance, or a vibration unit on the seat. In this way, the vehicle can also combine the image information to allow users to experience physical, olfactory, and other sensory experiences corresponding to the content presented in the image, providing a richer user experience.

[0016] In one possible implementation of the first aspect, the functional component includes a vibration unit on a seat, and the vibration unit includes one or more vibration units, and each vibration unit is arranged at a different position on the seat. When the functional component in the vehicle control cabin provides a function matching the image information, the vibration information corresponding to the vibration unit can be determined based on the image information, where the vibration information includes vibration frequency and / or vibration intensity. The vehicle then controls the vibration unit to vibrate based on the vibration information. In this way, the user can experience a vibration sensation corresponding to the content presented by the image.

[0017] In one possible implementation of the first aspect, the functional components include one or more of an onboard air conditioner, an onboard air humidifier, or an onboard fragrance. When the vehicle controls the functional components in the cabin to provide functions that match the image information, the setting information of the functional components can be determined based on the image information, wherein the setting information includes one or more of the temperature setting value corresponding to the onboard air conditioner, the humidity setting value corresponding to the onboard air humidifier, or the odor information corresponding to the onboard fragrance, wherein the odor information includes the target fragrance type and / or odor concentration. The vehicle adjusts one or more of the temperature of the onboard air conditioner, the humidity of the onboard air humidifier, or the odor of the onboard fragrance based on the setting information. In this way, the user can experience the seat posture, temperature, humidity, odor, etc. corresponding to the content presented in the image.

[0018] In one possible implementation of the first aspect, when the vehicle determines to transmit an image including a first display effect to the head-mounted device based on the first pose information, it may determine first display parameters of the image based on the first pose information, where the first display parameters are used to characterize the first display effect of the image. The vehicle then renders the image based on the first display parameters and transmits the rendered image to the head-mounted device.

[0019] In the above implementation, the vehicle can determine the display parameters of the image that matches the user's current posture based on the user's posture information, and render the image according to the display parameters, so that the head-mounted device can display the image that matches the user's current posture, bringing a better visual experience to the user.

[0020] In one possible implementation of the first aspect, when the vehicle controls each speaker to play the audio corresponding to the speaker based on the first posture information, the vehicle can obtain the attribute information of the speaker for each speaker, wherein the attribute information includes one or more of the position of the speaker in the cabin, the sound direction, or the frequency response. The vehicle determines the first playback parameter of the audio corresponding to the speaker based on the audio in the media resource, the first posture information, and the attribute information of the speaker. The vehicle renders the audio corresponding to the speaker based on the first playback parameter and sends the rendered audio to the speaker so that the speaker plays the rendered audio.

[0021] In the above implementation method, the vehicle can determine the audio playback parameters that match the user's current posture based on the user's posture information and the speaker's attribute information, and render the audio according to the playback parameters, so that each speaker can play the sound that matches the user's current posture, bringing the user a better auditory experience.

[0022] In a second aspect, a cockpit control method is provided, which is applied to a vehicle, the vehicle including at least one functional component, and the vehicle being connected to a head-mounted device. In this method, the vehicle obtains media resources and first posture information of the user in the cockpit, wherein the media resources include images. Based on the first posture information, the vehicle sends an image including a first display effect to the head-mounted device, wherein the first display effect is a three-dimensional effect obtained based on the first posture information. The vehicle continues to obtain second posture information of the user in the cockpit, and based on the second posture information, sends an image including a second display effect to the head-mounted device, wherein the second display effect is a three-dimensional effect obtained based on the second posture information. Furthermore, based on the image information of the image, the vehicle controls the functional components in the cockpit to provide functions that match the image information.

[0023] In the above method, the first and second posture information can represent information corresponding to different user postures within the cabin. The vehicle can send an image corresponding to or matching the user's current posture to the head-mounted device. Furthermore, the vehicle can control functional components within the cabin based on the image information to provide functions that match the image information. Furthermore, not only can the head-mounted device present dynamic three-dimensional visual effects that change with the user's posture, but the vehicle can also provide users with experiences in other dimensions. Users can experience media resources from multiple perspectives within the cabin, providing a better experience and satisfying their diverse experience needs.

[0024] In one implementation of the second aspect, when the vehicle obtains the first posture information of the user in the cabin, it may obtain first information sent by a head-mounted device, where the first information is obtained by the head-mounted device via at least one sensor and includes the user's head posture information. The vehicle obtains second information via at least one sensor in the cabin, where the second information includes one or more of the following: image information of the user, infrared information, seat position information of the user's seat, seat back angle information, seat angle information, or seat pressure information. The vehicle determines the first posture information based on the first and second information.

[0025] In the above implementation, the vehicle can obtain information collected by multiple sensors through multiple information input channels, and combine the information from multiple sensors to more accurately determine the user's posture information.

[0026] In one possible implementation of the second aspect, the second display effect is a modified version of the first display effect, and the modified display effect includes a change in one or more of the image viewing angle, image content, or image size. This allows the head-mounted device to present richer image display effects to the user that vary with the user's posture.

[0027] In one possible implementation of the second aspect, the functional component includes a vibration unit on the seat, and the vibration unit includes one or more vibration units, and each vibration unit is arranged at a different position on the seat. When the functional component in the vehicle control cabin provides a function matching the image information, the vibration information corresponding to the vibration unit can be determined based on the image information, where the vibration information includes vibration frequency and / or vibration intensity. The vehicle then controls the vibration unit to vibrate based on the vibration information. In this way, the user can experience a vibration sensation corresponding to the content presented by the image.

[0028] In one implementation of the second aspect, the functional components include one or more of a seat, an onboard air conditioner, an onboard air humidifier, or an onboard fragrance. When the vehicle controls the functional components in the cockpit to provide functions that match the image information, the setting information of the functional components can be determined based on the image information, wherein the setting information includes one or more of the temperature setting value corresponding to the onboard air conditioner, the humidity setting value corresponding to the onboard air humidifier, or the odor information corresponding to the onboard fragrance, wherein the odor information includes the target fragrance type and / or odor concentration. The vehicle adjusts one or more of the temperature of the onboard air conditioner, the humidity of the onboard air humidifier, or the odor of the onboard fragrance based on the setting information. In this way, the user can experience the seat posture, temperature, humidity, odor, etc. corresponding to the content presented in the image.

[0029] In one possible implementation of the second aspect, when the vehicle transmits an image including a first display effect to the head-mounted device based on the first pose information, first display parameters of the image may be determined based on the first pose information, where the first display parameters are used to characterize the first display effect of the image. The vehicle then renders the image based on the first display parameters and transmits the rendered image to the head-mounted device.

[0030] In the above implementation, the vehicle can determine the display parameters of the image that matches the user's current posture based on the user's posture information, and render the image according to the display parameters, so that the head-mounted device can display the image that matches the user's current posture, bringing a better visual experience to the user.

[0031] In one implementation of the second aspect, the vehicle further includes at least two speakers respectively arranged at different positions in the cabin, and the media resource further includes audio corresponding to the image. After obtaining the first posture information of the user in the cabin, the vehicle can also control each speaker to play the audio corresponding to the speaker based on the first posture information, and the audio corresponding to each speaker is obtained based on the audio in the media resource, the attribute information of the speaker, and the first posture information. Thereafter, after obtaining the second posture information of the user in the cabin, the vehicle can control each speaker to play the audio corresponding to the speaker based on the second posture information, and the audio corresponding to each speaker is obtained based on the audio in the media resource, the attribute information of the speaker, and the second posture information.

[0032] In the above implementation, the vehicle can control the speakers to play audio corresponding to or matching the user's current posture, and then the multiple speakers in the cabin can present the user with a dynamic and immersive playback effect that changes with the user's posture. The user can experience media resources from the visual, auditory and other aspects in the cabin, giving the user a better feeling and meeting the user's various experience needs.

[0033] In one possible implementation of the second aspect, the audio corresponding to each speaker changes based on changes in user posture information, and the audio changes include changes in one or more of the following: audio frequency, audio amplitude, audio delay, audio phase, or audio spectrum characteristics. This allows the multiple speakers in the cabin to present a richer sound experience that changes with the user's posture.

[0034] In one possible implementation of the second aspect, when the vehicle controls each speaker to play the audio corresponding to the speaker based on the first posture information, the attribute information of the speaker can be obtained for each speaker, wherein the attribute information includes one or more of the position of the speaker in the cabin, the sound direction, or the frequency response. The vehicle determines the first playback parameter of the audio corresponding to the speaker based on the audio in the media resource, the first posture information, and the attribute information of the speaker. The vehicle then renders the audio corresponding to the speaker based on the first playback parameter, and sends the rendered audio to the speaker so that the speaker plays the rendered audio.

[0035] In the above implementation method, the vehicle can determine the audio playback parameters that match the user's current posture based on the user's posture information and the speaker's attribute information, and render the audio according to the playback parameters, so that each speaker can play the sound that matches the user's current posture, bringing the user a better auditory experience.

[0036] In a third aspect, an electronic system is provided, comprising at least two speakers respectively arranged at different positions in a cockpit, wherein the electronic system is connected to a head-mounted device; the electronic system further comprises an information acquisition module and a resource processing module.

[0037] The information acquisition module is used to obtain media resources and the user's first position information in the cockpit; the media resources include images and audio corresponding to the images.

[0038] A resource processing module is used to send an image including a first display effect to a head-mounted device based on the first posture information; the first display effect is a three-dimensional effect obtained based on the first posture information; based on the first posture information, each speaker is controlled to play the audio corresponding to the speaker; the audio corresponding to each speaker is obtained based on the audio in the media resource, the attribute information of the speaker and the first posture information.

[0039] The information acquisition module is also used to obtain the second posture information of the user in the cockpit.

[0040] The resource processing module is also used to send an image including a second display effect to the head-mounted device based on the second posture information; the second display effect is a three-dimensional effect obtained based on the second posture information; based on the second posture information, each speaker is controlled to play the audio corresponding to the speaker; the audio corresponding to each speaker is obtained based on the audio in the media resource, the attribute information of the speaker and the second posture information.

[0041] In an implementation method of the third aspect, the information acquisition module is specifically used to obtain first information sent by a head-mounted device; the first information is obtained by the head-mounted device through at least one sensor, and the first information includes the user's head posture information; the second information is obtained through at least one sensor in the cabin; the second information includes one or more of the user's image information, infrared information, seat position information of the user's seat, seat back angle information, seat angle information, or seat pressure information; the first posture information is determined based on the first information and the second information.

[0042] In an implementation manner of the third aspect, the second display effect is a changed effect of the first display effect, and the change in the display effect includes a change in one or more of the image viewing angle, image content, or image size.

[0043] In one implementation of the third aspect, the audio corresponding to each speaker changes according to the change of the user posture information, and the change of the audio includes a change in one or more of the audio frequency, audio amplitude, audio delay, audio phase, or audio spectrum characteristics.

[0044] In one implementation method of the third aspect, the resource processing module is also used to obtain image information of images in media resources; the image information includes one or more of season information, weather information, temperature information, location information, scene information, or event information; and the functional components in the cabin are controlled to provide functions matching the image information; the functional components include one or more of seats, vehicle air conditioners, vehicle air humidifiers, vehicle fragrances, or vibration units on seats.

[0045] In one implementation of the third aspect, the functional component includes a vibration unit on the seat, the vibration unit includes one or more, and each vibration unit is arranged at a different position on the seat; a resource processing module is specifically used to determine the vibration information corresponding to the vibration unit based on the image information; the vibration information includes vibration frequency and / or vibration intensity; and according to the vibration information, the vibration unit is controlled to vibrate.

[0046] In an implementation method of the third aspect, the functional components include one or more of a vehicle air conditioner, a vehicle air humidifier, or a vehicle fragrance; the resource processing module is specifically used to determine the setting information of the functional components based on the image information; the setting information includes one or more of the temperature setting value corresponding to the vehicle air conditioner, the humidity setting value corresponding to the vehicle air humidifier, or the odor information corresponding to the vehicle fragrance; the odor information includes the target fragrance and / or odor concentration; according to the setting information, adjust one or more of the temperature of the vehicle air conditioner, the humidity of the vehicle air humidifier, or the odor of the vehicle fragrance.

[0047] In an implementation method of the third aspect, the resource processing module is specifically used to determine a first display parameter of the image based on the first posture information; the first display parameter is used to characterize a first display effect of the image; the image is rendered according to the first display parameter; and the rendered image is sent to the head-mounted device.

[0048] In an implementation method of the third aspect, the resource processing module is specifically used to obtain the attribute information of the speaker for each speaker; the attribute information includes one or more of the position of the speaker in the cabin, the sound direction, and the frequency response; based on the audio in the media resource, the first posture information and the attribute information of the speaker, the first playback parameter of the audio corresponding to the speaker is determined; based on the first playback parameter, the audio corresponding to the speaker is rendered; and the rendered audio is sent to the speaker so that the speaker plays the rendered audio.

[0049] In a fourth aspect, an electronic system is provided, the electronic system including at least one functional component, the electronic system being connected to a head-mounted device, and the electronic system also including an information acquisition module and a resource processing module.

[0050] The information acquisition module is used to obtain media resources and the user's first posture information in the cockpit; the media resources include images.

[0051] A resource processing module, configured to send an image including a first display effect to the head mounted device according to the first posture information; the first display effect being a three-dimensional effect obtained according to the first posture information;

[0052] The information acquisition module is also used to obtain the second posture information of the user in the cockpit;

[0053] The resource processing module is further configured to send an image including a second display effect to the head mounted device according to the second posture information; the second display effect is a three-dimensional effect obtained according to the second posture information;

[0054] The resource processing module is also used to control the functional components in the cockpit to provide functions matching the image information based on the image information.

[0055] In an implementation method of the fourth aspect, the information acquisition module is specifically used to obtain first information sent by a head-mounted device, wherein the first information is obtained by the head-mounted device through at least one sensor, and the first information includes the user's head posture information; second information is obtained through at least one sensor in the cabin, wherein the second information includes one or more of the user's image information, infrared information, seat position information of the user's seat, seat back angle information, seat angle information, or seat pressure information; and the first posture information is determined based on the first information and the second information.

[0056] In an implementation manner of the fourth aspect, the second display effect is a change in the first display effect, and the change in the display effect includes a change in one or more of the image viewing angle, image content, or image size.

[0057] In one implementation of the fourth aspect, the functional component includes a vibration unit on the seat, the vibration unit including one or more units, and each vibration unit being disposed at a different location on the seat. The resource processing module, specifically configured to control the functional component in the cabin to provide a function matching the image information, may determine vibration information corresponding to the vibration unit based on the image information, where the vibration information includes vibration frequency and / or vibration intensity; and control the vibration unit to vibrate based on the vibration information.

[0058] In one implementation of the fourth aspect, the functional components include one or more of a seat, an onboard air conditioner, an onboard air humidifier, or an onboard fragrance. The resource processing module is specifically configured to determine setting information for the functional components based on the image information, wherein the setting information includes one or more of a temperature setting value corresponding to the onboard air conditioner, a humidity setting value corresponding to the onboard air humidifier, or scent information corresponding to the onboard fragrance, wherein the scent information includes a target fragrance type and / or scent concentration; and adjust one or more of the temperature of the onboard air conditioner, the humidity of the onboard air humidifier, or the scent of the onboard fragrance based on the setting information.

[0059] In one implementation of the fourth aspect, the resource processing module is specifically configured to determine a first display parameter of the image based on the first pose information, wherein the first display parameter is used to represent a first display effect of the image. The electronic system renders the image based on the first display parameter and transmits the rendered image to the head-mounted device.

[0060] In one implementation of the fourth aspect, the electronic system further includes at least two speakers respectively arranged at different positions in the cockpit, and the media resources further include audio corresponding to the image. After obtaining the first posture information of the user in the cockpit, the resource processing module is specifically used to control each speaker to play the audio corresponding to the speaker based on the first posture information, and the audio corresponding to each speaker is obtained based on the audio in the media resource, the attribute information of the speaker, and the first posture information. Thereafter, after obtaining the second posture information of the user in the cockpit, the resource processing module is specifically used to control each speaker to play the audio corresponding to the speaker based on the second posture information, and the audio corresponding to each speaker is obtained based on the audio in the media resource, the attribute information of the speaker, and the second posture information.

[0061] In one implementation of the fourth aspect, the audio corresponding to each speaker changes according to the change of the user posture information, and the change of the audio includes a change in one or more of the audio frequency, audio amplitude, audio delay, audio phase, or audio spectrum characteristics.

[0062] In an implementation method of the fourth aspect, the resource processing module is specifically used to obtain attribute information of the speaker for each speaker, wherein the attribute information includes one or more of the position of the speaker in the cabin, the sound direction, or the frequency response; determine the first playback parameter of the audio corresponding to the speaker based on the audio in the media resource, the first posture information and the attribute information of the speaker; render the audio corresponding to the speaker based on the first playback parameter, and send the rendered audio to the speaker so that the speaker plays the rendered audio.

[0063] In a fifth aspect, a vehicle is provided, comprising a memory and one or more processors; the memory is coupled to the processor; wherein computer program code is stored in the memory, and the computer program code comprises computer instructions, which, when executed by the processor, enable the vehicle to execute the cockpit control method as described in the first aspect and any one of its implementations, or to execute the cockpit control method as described in the second aspect and any one of its implementations.

[0064] In a sixth aspect, a computer-readable storage medium is provided, comprising computer instructions. When the computer instructions are executed on a vehicle, the vehicle executes the cockpit control method as described in the first aspect and any one of its implementations, or executes the cockpit control method as described in the second aspect and any one of its implementations.

[0065] In the seventh aspect, a computer program product is provided. When the computer program product is run on a vehicle, the vehicle executes the cabin control method as described in the first aspect and any one of its implementations, or executes the cabin control method as described in the second aspect and any one of its implementations.

[0066] The beneficial effects that can be achieved by the electronic system provided by the third aspect, the electronic system provided by the fourth aspect, the vehicle provided by the fifth aspect, the computer-readable storage medium provided by the sixth aspect, and the computer program product provided by the seventh aspect can refer to the beneficial effects that can be achieved by the first aspect and any of its implementation methods, or refer to the beneficial effects that can be achieved by the second aspect and any of its implementation methods, and will not be repeated here. BRIEF DESCRIPTION OF THE DRAWINGS

[0067] FIG1 is a structural schematic diagram of a vehicle according to an embodiment of the present application;

[0068] FIG2 is a second structural diagram of a vehicle according to an embodiment of the present application;

[0069] FIG3 is a third structural diagram of a vehicle according to an embodiment of the present application;

[0070] FIG4 is a fourth structural diagram of a vehicle according to an embodiment of the present application;

[0071] FIG5 is a flowchart of a cockpit control method according to an embodiment of the present application;

[0072] FIG6 is a first schematic diagram of controlling a vehicle speaker to play audio according to an embodiment of the present application;

[0073] FIG7 is a second schematic diagram of controlling a vehicle speaker to play audio according to an embodiment of the present application;

[0074] FIG8 is a second flow chart of a cockpit control method according to an embodiment of the present application;

[0075] FIG9 is a schematic diagram of the structure of an electronic system according to an embodiment of the present application.

[0076] FIG10 is a fifth structural diagram of a vehicle according to an embodiment of the present application. DETAILED DESCRIPTION

[0077] The technical solutions in the embodiments of the present application will be described below in conjunction with the drawings in the embodiments of the present application. Among them, in the description of the present application, unless otherwise specified, " / " indicates that the objects associated before and after are in an "or" relationship. For example, A / B can represent A or B; "and / or" in the present application is only a description of the association relationship of the associated objects, indicating that there can be three relationships. For example, A and / or B can represent: A exists alone, A and B exist at the same time, and B exists alone, where A and B can be singular or plural. In addition, in the description of the present application, unless otherwise specified, "multiple" refers to two or more than two. "At least one of the following" or similar expressions refers to any combination of these items, including any combination of single or plural items. For example, at least one of a, b, or c can represent: a, b, c, ab, ac, bc, or abc, where a, b, c can be single or multiple. In addition, in order to facilitate the clear description of the technical solutions of the embodiments of the present application, in the embodiments of the present application, words such as "first" and "second" are used to distinguish between identical or similar items with substantially the same functions and effects. Those skilled in the art will understand that words such as "first" and "second" do not limit the quantity and execution order, and words such as "first" and "second" do not necessarily limit differences. At the same time, in the embodiments of the present application, words such as "exemplary" or "for example" are used to indicate examples, illustrations or explanations. Any embodiment or design described as "exemplary" or "for example" in the embodiments of the present application should not be interpreted as being more preferred or more advantageous than other embodiments or design schemes. Specifically, the use of words such as "exemplary" or "for example" is intended to present related concepts in a concrete way for easy understanding.

[0078] In addition, the business scenarios described in the embodiments of the present application are intended to more clearly illustrate the technical solutions of the embodiments of the present application, and do not constitute a limitation on the technical solutions provided in the embodiments of the present application. Ordinary technicians in this field can know that with the emergence of new business scenarios, the technical solutions provided in the embodiments of the present application are also applicable to similar technical problems.

[0079] Currently, the development of smart cars has attracted widespread attention. More and more video and audio processing technologies are being applied to car cabin audio systems to meet users' increasingly higher requirements and increasing demands for smart car functions.

[0080] The audio and video systems of some smart cars can be connected to VR / AR glasses. The data cable of the VR / AR glasses is connected to the USB-C port in the back row of the smart car, and the image on the display screen set in the back row is projected onto the VR / AR glasses, thereby providing users with a high-definition large-screen viewing experience, and using the rich video resources on the smart car computer (or on-board terminal) to provide mobile cinema services. Smart cars can also interact with VR / AR and other head-mounted devices. For example, the smart car is responsible for collecting images around the vehicle and obtaining VR / AR three-dimensional (3D) virtual scenes (such as autonomous driving scenes, etc.) based on image modeling. Afterwards, it can respond to the interactive control instructions input by the user, synthesize the dynamic effects with the 3D virtual scene to obtain a virtual scene picture, and present it on VR / AR and other head-mounted devices.

[0081] However, the above method only provides users with a visual experience, and does not allow users to obtain multiple experiences from the senses of hearing, touch, smell, etc., that is, users cannot obtain a more immersive experience.

[0082] In addition, some VR / AR glasses provide users with fixed 3D visual effects, that is, the content viewed by the user will not change dynamically with the rotation of the user's head, etc., so the user experience will be poor.

[0083] Some smart cars are equipped with 3D / 4D audio systems, which can provide users with 3D / 4D stereo and surround sound audio effects. However, due to the physical limitations of the actual cabin, the positional relationship between the user and the cabin audio system often does not meet the user's requirements for standard surround sound. Moreover, when the user's position and / or posture changes, the user's auditory experience does not change accordingly, resulting in a poor auditory experience.

[0084] In the above-mentioned various solutions, the experience functions provided to users are relatively single or simple, and cannot meet the user's various experience needs.

[0085] Based on the above, an embodiment of the present application provides a cockpit control method. A vehicle includes at least two speakers disposed at different locations within the cockpit, and the vehicle is also connected to a head-mounted device. The vehicle obtains media resources and a first position information of a user within the cockpit, wherein the media resources include an image and audio corresponding to the image. The vehicle then transmits an image including a first display effect to the head-mounted device based on the first position information, wherein the first display effect is a three-dimensional effect obtained based on the first position information. Based on the first position information, the vehicle controls each speaker to play audio corresponding to the speaker, wherein the audio corresponding to each speaker is obtained based on the audio in the media resources, the speaker's attribute information, and the first position information. The vehicle then obtains a second position information of the user within the cockpit, and based on the second position information, transmits an image including a second display effect to the head-mounted device, wherein the second display effect is a three-dimensional effect obtained based on the second position information. The vehicle then controls each speaker to play audio corresponding to the speaker based on the second position information, wherein the audio corresponding to each speaker is obtained based on the audio in the media resources, the speaker's attribute information, and the second position information.

[0086] In the above method, the first posture information and the second posture information can represent the information corresponding to different postures of the user in the cabin. The vehicle can send an image corresponding to or matching the user's current posture to the head-mounted device, and control the speaker to play audio corresponding to or matching the user's current posture. Then, the head-mounted device can present dynamic 3D visual effects that change with the user's posture to the user. The multiple speakers in the cabin can present dynamic and immersive auditory effects that change with the user's posture to the user. The user can experience media resources from visual and auditory aspects in the cabin, giving the user a better experience and meeting the user's various experience needs.

[0087] Alternatively, an embodiment of the present application also provides another cockpit control method, which is applied to a vehicle, the vehicle including at least one functional component, and the vehicle is connected to a head-mounted device. The vehicle obtains media resources and first posture information of the user in the cockpit, wherein the media resources include images. The vehicle sends an image including a first display effect to the head-mounted device based on the first posture information, wherein the first display effect is a three-dimensional effect obtained based on the first posture information. The vehicle continues to obtain second posture information of the user in the cockpit, and sends an image including a second display effect to the head-mounted device based on the second posture information, wherein the second display effect is a three-dimensional effect obtained based on the second posture information. And, the vehicle controls the functional components in the cockpit to provide functions matching the image information based on the image information of the image.

[0088] In the above method, the first and second posture information can also represent information corresponding to different postures of the user in the cabin. The vehicle can send an image corresponding to or matching the user's current posture to the head-mounted device. Furthermore, the vehicle can control functional components in the cabin based on the image information to provide functions that match the image information. Furthermore, not only can the head-mounted device present dynamic 3D visual effects that change with the user's posture, but the vehicle can also provide users with experiences in other dimensions. Users can experience media resources from multiple perspectives in the cabin, giving them a better experience and meeting their diverse experience needs.

[0089] In some possible implementations, the vehicle may include at least one controller for controlling different components or areas of the vehicle. For example, as shown in FIG1 , the vehicle may include a vehicle domain controller, an autonomous driving domain controller, and an intelligent cockpit domain controller, and each controller may communicate with each other.

[0090] The vehicle domain controller (VDC) is responsible for overall vehicle control, requiring high real-time and safety performance. The autonomous driving domain controller (ADAS / AD domain controller, ADC) is responsible for autonomous driving-related perception, decision-making, and control functions. The intelligent cockpit domain controller (CDC) is responsible for intelligent cockpit functions such as human-computer interaction. Alternatively, the autonomous driving domain controller can be referred to as the mobile data center (MDC).

[0091] The cockpit control method provided in the embodiment of the present application can be applied to the intelligent cockpit domain controller shown in Figure 1.

[0092] In other possible implementations, the vehicle may further include at least one functional module for providing vehicle functions. For example, as shown in FIG2 , the vehicle may include a resource acquisition module, a rendering engine, a posture detection module, and the like. Furthermore, the vehicle may further include multiple speakers, each of which may be located at a different location within the vehicle cabin. The vehicle may also be connected to a head-mounted device. The vehicle and the head-mounted device may be connected via wired or wireless communication.

[0093] The resource acquisition module can acquire media resources, which may include images and audio corresponding to the images, etc. Furthermore, the resource acquisition module can also send the media resources to the rendering engine.

[0094] The posture detection module can obtain the posture information of the user in the cabin and send the posture information of the user to the rendering engine.

[0095] Since the user's posture (position and posture) in the cockpit is not fixed, the posture detection module can obtain the user's posture information in real time, so that the subsequent rendering engine can determine whether the user's posture has changed based on the different posture information received before and after, and render the media resources based on the currently received posture information, so that the display and playback of the media resources match the current posture information, that is, match the user's current posture.

[0096] The above-mentioned rendering engine can render and process the images, audio, etc. in the media resources according to the currently received user's posture information, so that the image display effect, audio playback effect, etc. can better match the user's current position and posture, or, so that the user can receive better image display effects, audio playback effects, etc. in the current position and posture.

[0097] Among them, the rendering engine can send the rendered image to the head-mounted device for display, and send the rendered audio to different speakers in the cockpit for playback.

[0098] The above-mentioned head-mounted device can be VR / AR glasses, or VR / AR helmets, etc.

[0099] When an image is displayed on a head-mounted device, it can present a 3D display effect. The rendering engine receives the user's posture information in real time and renders the image based on the current user's posture information. If the user's posture changes, the rendering engine will also change the image rendering, and thus the 3D display presented on the head-mounted device will also change. This achieves the purpose of dynamically adjusting the image display effect according to the user's posture and matching the 3D display characteristics of the head-mounted device to provide users with a better visual experience.

[0100] For example, the image viewing angle, image size, image content, etc. displayed in the head-mounted device are adjusted according to the user's posture changes.

[0101] The above-mentioned speakers can be door speakers, sky sound speakers, surround speakers, center speakers, headrest speakers and subwoofers, etc. This embodiment of the present application does not make specific restrictions on this.

[0102] When audio is played on a speaker, since there are multiple speakers in the cabin, the rendering engine can determine the playback effect of each speaker based on the attribute information of each speaker in the cabin (such as the speaker's position in the cabin, sound direction, frequency response, and the device properties of the speaker itself), the user's posture information, and the audio. Based on this playback effect, the audio is rendered for each speaker separately and sent to the corresponding speaker for playback. The audio corresponding to each speaker is different.

[0103] It is understandable that due to the different properties of each speaker, the distance and relative position between them and the user are also different. Therefore, the same audio obtained by the resource acquisition module will be different after rendering and processing for different speakers. As a result, after the sound emitted by different speakers is transmitted to the user, the user can obtain a more three-dimensional, surround and other immersive listening experience in the current posture.

[0104] If the user's posture changes, the rendering engine will change the audio rendering for each speaker, and the audio corresponding to each speaker will also change, thereby achieving the purpose of dynamically adjusting the audio playback effect according to the user's posture, bringing a better listening experience to the user.

[0105] For example, the audio change may include changes in one or more of the following: audio frequency, audio amplitude, audio delay, audio phase, and audio spectral characteristics. Alternatively, because the vehicle inputs an audio signal to the speaker, the audio change may also be represented by changes in one or more of the following: audio signal frequency, signal amplitude, signal delay, signal phase, and signal spectral characteristics.

[0106] For example, the user's posture may include head posture, body posture, etc. The head posture further includes the angle of head rotation, the direction of rotation, the angle between the head and the body, etc.; the body posture includes the angle of body rotation, the direction of rotation, etc.

[0107] In some possible implementations, the rendering engine may further include multiple rendering modules, each of which performs rendering for different types of media resources. For example, as shown in FIG3 , the rendering engine may include a visual rendering module and an auditory rendering module. The visual rendering module may render images in the media resources, and the auditory rendering module may render audio in the media resources. Furthermore, the visual rendering module sends the rendered images to the head-mounted device for display, and the auditory rendering module sends the audio rendered for different speakers to the corresponding speakers for playback.

[0108] In some possible implementations, the posture detection module may further include at least one sensor that can acquire information used to characterize the user's position and posture. For example, as shown in FIG3 , the posture detection module may include a pressure sensor, a camera, an infrared sensor (or infrared probe), a seat sensor, and the like.

[0109] Among them, the pressure sensor can be installed under each seat in the cabin to detect seat information such as the pressure the seat is subjected to, and send the seat pressure information to the rendering engine. The rendering engine determines whether there is a user sitting in the seat, the user's weight, etc. based on the seat pressure information.

[0110] One or more cameras can be installed at any position in the cabin that can capture a seat. When a user is seated in a seat, the camera can capture the user's image and send the user's image information to the rendering engine, which then determines the user's posture based on the image information.

[0111] Alternatively, the camera may capture an image of the seat and send the image information of the seat to the rendering engine, which then determines whether there is a user sitting on the seat and the user's posture based on the image information of the seat.

[0112] Multiple infrared sensors (or infrared probes) can be provided and placed at corresponding positions for each seat in the cabin. The infrared sensors can detect infrared information on the corresponding seat and send the infrared information to the rendering engine, which then determines whether the seat is occupied, etc. based on the infrared information.

[0113] The seat sensor can be installed on the seat to obtain other seat information, such as seat back angle, seat position, seat height, seat angle (such as the angle between the seat cushion and the bottom plane of the vehicle, or the pitch angle), etc. The seat sensor can send the seat information to the rendering engine, which can then determine the user's position and posture based on the seat information.

[0114] Alternatively, the seat sensor may be a seat adjustment unit provided on the seat, which is used to adjust the seat. The seat adjustment unit may send seat information to the rendering engine, which then determines the user's position and posture based on the seat information.

[0115] In some possible implementations, since the head-mounted device includes a posture sensor, the posture detection module can also obtain the user's head posture information obtained by the posture sensor in the head-mounted device. The posture detection module then sends the user's head posture information to the rendering engine, which determines the user's head posture based on the head posture information.

[0116] Alternatively, in some possible implementations, the head-mounted device may also obtain the user's head posture information through a posture sensor and directly send the user's head posture information to the rendering engine, which then determines the user's head posture based on the user's head posture information. The embodiments of this application do not specifically limit how the head posture information in the head-mounted device is obtained.

[0117] The rendering engine can combine information obtained from multiple information input channels (such as multiple sensors) to more accurately determine the user's posture, so that it can render media resources more accurately based on the user's posture. When the user uses a head-mounted device connected to the vehicle to experience media resources, the head-mounted device can present a better 3D display effect to the user based on the rendered image sent by the rendering engine, and the multiple speakers in the cabin can present a better immersive sound to the user based on the rendered audio sent by the rendering engine.

[0118] In some possible implementations, the images in the aforementioned media resources may also include image information, such as season information, weather information, temperature information, location information, scene information, event information, etc., and the image information may be identified by the rendering engine from the content displayed by the image, or may be carried in the media resources. There is no specific limitation on this in the embodiments of the present application.

[0119] The above-mentioned rendering engine can also be connected to the functional components in the vehicle cabin, and according to the image information, control the functional components in the cabin to provide functions that match the image information, so that the functions provided by the functional components can also meet the needs of users when viewing media resources, allowing users to experience multi-dimensional physical experiences corresponding to the images, providing users with a richer experience.

[0120] For example, as shown in FIG4 , the functional components described above may include seats, vehicle air conditioners, vehicle air humidifiers, vehicle fragrances, and vibration units on the seats. Furthermore, the rendering engine can perform functional simulations using different functional components, such as position simulation using the seats, temperature simulation using the vehicle air conditioners, humidity simulation using the vehicle air humidifiers, scent simulation using the vehicle fragrances, and vibration simulation using the vibration units on the seats. Accordingly, the functions of the functional components may include adjusting the seat back angle, seat angle, and seat position during position simulation; adjusting the temperature setting of the vehicle air conditioner during temperature simulation; adjusting the humidity setting of the vehicle air humidifier during humidity simulation; adjusting the fragrance type and odor concentration of the vehicle fragrance during scent simulation; and adjusting the vibration frequency and intensity of the vibration unit during vibration simulation.

[0121] For example, if the image information contains scene information such as a game (such as a roller coaster), the rendering engine can determine the target posture information of the user riding the roller coaster in the game based on the image information, such as the user's height, position, upper body angle, etc., and then adjust one or more of the seat back angle, seat angle, and seat position based on the user's target posture information, thereby providing the user with a more realistic scene effect in terms of physical perception.

[0122] Furthermore, according to the aforementioned embodiment, since the seat may be provided with a seat adjustment unit, the rendering engine may further adjust the seat by controlling the seat adjustment unit.

[0123] It is understandable that in some possible implementations, in order to ensure the user's experience when watching the game screen, the seat adjustment unit can also be set to a low-latency / fast adjustment unit, so that the rendering engine can adjust the seat more quickly and match the changes in the scene screen.

[0124] For another example, if the image information contains seasonal information (e.g., cold winter, hot summer, spring, late autumn), weather information (e.g., sunny, cloudy, foggy), temperature information (e.g., hot, cold), and location information (e.g., desert, rainforest, outdoors, garden), the rendering engine can determine, based on the image information, the vehicle air conditioner temperature setting, vehicle humidifier humidity setting, and vehicle fragrance corresponding odor information (e.g., target fragrance type and / or odor concentration), etc., that match the current image information. Based on this setting information, the rendering engine can adjust the vehicle air conditioner temperature, vehicle humidifier humidity, and vehicle fragrance scent, etc. This provides users with a more realistic scene effect in terms of physical perception, allowing users to experience the seat posture, temperature, humidity, and odor corresponding to the content presented in the image.

[0125] For another example, if the image information contains scene information such as explosion, earthquake, horror and thriller, the rendering engine can determine the vibration information of the vibration unit based on the image information, such as the vibration frequency and vibration intensity that should be applied to the user, and then control the vibration unit to vibrate and / or adjust the vibration frequency and vibration intensity of the vibration unit based on the vibration information, thereby providing the user with a realistic vibration sensation corresponding to the scene of the image and improving the user experience.

[0126] Furthermore, the seat can be equipped with multiple vibration units, which can be placed in various locations such as the seat back, seat cushion, and armrests. Based on the image information, the rendering engine can determine the vibration information of the vibration units corresponding to different positions of the user leaning on the seat, and then control the different vibration units to vibrate at different frequencies and / or intensities, thus providing the user with a more realistic vibration sensation.

[0127] It should be understood that the aforementioned adjustments to various functional components within the cabin, and the types of image information used to perform these adjustments, are merely examples. In other possible implementations, one or more of various functional components may be adjusted based on one or more of various image information types, and this is not specifically limited in the present embodiments.

[0128] In some possible implementations, the aforementioned media resources may also include 2D media resources, 3D media resources, panoramic media resources (or 360° panoramic media resources), virtual reality (VR) media resources, or augmented reality (AR) media resources. Regardless of the media resource type, the rendering engine can render it into a resource with a 3D display effect that can be displayed by a head-mounted device, thereby providing a better viewing experience for the user wearing the head-mounted device.

[0129] Furthermore, the media resources may be stored in the vehicle's local storage space, or may be obtained from a server by the resource acquisition module, or, when the vehicle is connected to some terminal devices (such as a mobile phone, tablet computer, etc.), the media resources may also be obtained from the terminal devices. This is not specifically limited in the embodiments of the present application.

[0130] The aforementioned various implementation methods for providing users with visual, auditory, and physical experiences can be combined in any two or all of them, for example, achieving a visual + auditory experience, a visual + physical experience, a visual + auditory + physical experience, etc. In this way, the rendering engine can use the user's posture information to adjust the image display effect on the head-mounted device, adjust the audio output of the speakers in the cabin, or adjust the functions provided by functional components in the cabin, thereby allowing users to experience media resources more realistically from multiple angles or dimensions, making users feel as if they are in the scene.

[0131] The cockpit control method can provide users with visual, auditory, and other dimensional experiences of media resources. For example, referring to FIG5 , the method may include the following steps S501 - S506 .

[0132] S501: The vehicle obtains media resources and the first position information of the user in the cabin.

[0133] The media resource includes an image and audio corresponding to the image. The media resource may be any one of a 2D media resource, a 3D media resource, a panoramic media resource, a VR media resource, or an AR media resource. Furthermore, the image may also be any one of a 2D image, a 3D image, a panoramic image, a VR image, or an AR image.

[0134] Furthermore, the vehicle can obtain the user's first posture information in the cockpit based on at least one sensor set in the cockpit and / or a sensor in the head-mounted device (such as a posture sensor).

[0135] Since the user is not always stationary in the cockpit, their position and posture information will also change. The first position information can represent the user's position and posture at a specific moment. The vehicle will obtain the position and posture information corresponding to the user's position and posture in the cockpit in real time.

[0136] In some embodiments, the sensor in the head-mounted device can acquire first information, where the first information may include the user's head posture information. The at least one sensor in the cabin can acquire second information, where the second information may include one or more of the following: user image information, infrared information, seat position information of the user's seat, seat back angle information, seat angle information, and seat pressure information.

[0137] The vehicle can determine the user's first position information in the cabin based on the first information and the second information. In this way, since there are multiple sources or channels of information (such as at least one sensor, head-mounted device, etc.), there are also multiple types of information (such as user's head posture information, user's image information, seat angle information, seat position information, seat back angle information, seat pressure information, etc.), so the vehicle can combine multiple information to more accurately determine the user's first position information, and then provide the user with better display images and more immersive sound based on the first position information.

[0138] S502: The vehicle sends an image including a first display effect to the head-mounted device according to the first posture information.

[0139] Since the image in the media resource needs to be displayed in the head-mounted device, the first display effect of the image is a 3D effect obtained according to the first posture information, and the first display effect matches the user's current position and posture.

[0140] In some embodiments, the vehicle may also determine the first display parameters of the image based on the first posture information. The first display parameters are used to characterize the first display effect of the image, or in other words, the first display parameters correspond to the first display effect. In addition, the first display parameters may include the color, pixel, refresh rate, brightness, viewing angle, etc. of the image displayed in accordance with the user's current posture. Afterwards, the vehicle renders the image according to the first display parameters and sends the rendered image to the head-mounted device, so that the head-mounted device displays the rendered image and presents the first display effect.

[0141] In this way, the content displayed by the head-mounted device can match the user's current posture, bringing a better viewing experience to the user.

[0142] S503: The vehicle controls each speaker to play the audio corresponding to the speaker according to the first posture information.

[0143] The audio corresponding to each speaker is different, and the audio corresponding to each speaker is obtained according to the audio in the media resource, the attribute information of the speaker and the first posture information.

[0144] The attribute information of the speaker may include one or more of the device attributes of the speaker itself, the position of the speaker in the cabin, the sound direction, and the frequency response.

[0145] The vehicle can use the user as the center of the in-cabin sound field and determine how each speaker in the cabin should sound so that the audio played by all speakers creates an immersive, stereo, surround, or other playback effect at the user's location. For example, the user can receive the same sound from different speakers at their current location. The user's current location can be determined based on the first position information mentioned above, and the sound output of each speaker can be determined based on the audio and speaker attribute information and the first position information in the media resource.

[0146] It is understandable that how each speaker makes sound is not only related to the attribute information of the speaker, but also related to the distance between the speaker and the user, the user's posture, etc.

[0147] Furthermore, in some embodiments, for each speaker, the vehicle can obtain the attribute information of the speaker and determine the first playback parameters of the audio corresponding to the speaker based on the audio in the media resource, the first posture information, and the attribute information of the speaker. In addition, the first playback parameters may include the amplitude of the audio, the frequency of the audio, the delay of the audio, the phase of the audio, the spectral characteristics of the audio, etc. that match the user's current posture. Afterwards, the vehicle renders the audio according to the first playback parameters and sends the rendered audio to the corresponding speaker, so that the speaker plays the rendered audio.

[0148] For example, as shown in FIG6 , the cabin includes speakers 1, 2, 3, and 4. Each speaker is located in a different location within the cabin, such as a door, center console, or headrest. After a user takes a seat in the cabin, the vehicle can determine a first playback parameter corresponding to each speaker, as each speaker may have different attribute information and be located at a different distance from the user. For example, speaker 1 corresponds to playback parameter 1, speaker 2 corresponds to playback parameter 2, speaker 3 corresponds to playback parameter 3, and speaker 4 corresponds to playback parameter 4, with each playback parameter being different. The vehicle then renders the audio according to these playback parameters, producing rendered audio 1, audio 2, audio 3, and audio 4. Audio 1 is then sent to speaker 1 for playback, audio 2 to speaker 2, audio 3 to speaker 3, and audio 4 to speaker 4 for playback. After each speaker plays sound, the user experiences immersive audio effects, such as stereo and surround sound, at their current location and in their current posture.

[0149] In some embodiments, the vehicle may obtain corresponding media resources in response to a user's selection operation on a media resource.

[0150] For example, after the vehicle is connected to the head-mounted device, the in-vehicle terminal in the cabin may display a resource page for the user to select media resources. The resource page may include a variety of media resources, such as video resources, game resources, etc. When the user selects a media resource on the resource page, the vehicle will respond to the user's operation, obtain the corresponding media resource, and render and process the images and sounds in the media resource. After processing, the rendered image is sent to the head-mounted device for display, and the rendered audio is sent to the speakers for playback.

[0151] As another example, after the vehicle is connected to the head-mounted device, a resource page may be displayed on the head-mounted device for the user to select media resources. The resource page may also include multiple media resources, such as video resources, game resources, etc. After the user puts on the head-mounted device, they can select a media resource on the resource page by moving their head, etc. The head-mounted device will respond to the user's operation and send a resource request to the vehicle. The vehicle obtains the corresponding media resource based on the resource request and processes the image and sound in the media resource by rendering and other processing. After processing, the rendered image is sent to the head-mounted device for display, and the rendered audio is sent to the speakers for playback.

[0152] S504: The vehicle obtains second posture information of the user in the cabin.

[0153] The second posture information may represent the user's position and posture at another time after the first posture information is obtained. The first posture information and the second posture information may represent the information corresponding to different postures of the user in the cabin.

[0154] Also, for the method and related content of obtaining the second posture information, reference may be made to the related content of the aforementioned S501.

[0155] S505: The vehicle determines to send an image including a second display effect to the head-mounted device based on the second posture information.

[0156] The second display effect is a 3D effect obtained according to the second posture information.

[0157] Also, the method for determining the second display effect and the determination of the image including the second display effect may refer to the relevant contents of the aforementioned S502.

[0158] S506: The vehicle controls each speaker to play the audio corresponding to the speaker according to the second posture information.

[0159] The audio corresponding to each speaker is different, and the audio corresponding to each speaker is obtained according to the audio in the media resource, the attribute information of the speaker and the second posture information.

[0160] Also, for determining the audio corresponding to each speaker, reference may be made to the relevant content of the aforementioned S503.

[0161] It's understood that as the user's position and posture within the cabin change, the vehicle will output different images to the head-mounted device accordingly. Consequently, the image quality presented by the head-mounted device will change with the user's position and posture. This change in display quality can include changes in one or more of the image's viewing angle, image content, or image size. This results in a better visual experience for the user.

[0162] Furthermore, as the user's position and posture change, the audio presented to the user by each speaker will also change with the user's posture. The audio changes include changes in one or more of the following: audio frequency, audio amplitude, audio delay, audio phase, and audio spectrum characteristics. This way, the vehicle controls the corresponding audio sound output from the speakers based on the user's posture, ensuring that the user experiences the same immersive, three-dimensional sound regardless of their posture, providing a better auditory experience.

[0163] For example, referring to (a) in FIG7 , the user is in the first posture, such as the user is in the main driving seat with the face facing directly in front of the user. The vehicle can determine the audio corresponding to each speaker in the cockpit based on the first posture information corresponding to the user, the attribute information of each speaker, and the audio in the media resources, and control the speaker to play the corresponding audio, so that the virtual sound image M formed by the sound played by multiple speakers is located 45° to the right front of the user's head.

[0164] The user then changes their position to a second position, where the user remains in the driver's seat but their face is rotated 45° to the left, as shown in Figure 7(b). At this point, the user needs to hear the exact virtual image M, which should be located 90° (45° + 45°) to the right of the user's head. However, due to the asymmetric arrangement of the user's position and the speakers in the vehicle, the virtual image M may not be at that 90° position. Therefore, the vehicle can adjust the audio (e.g., amplitude, delay, and spectrum) corresponding to each speaker based on the user's current second position, the attributes of each speaker, and the audio in the media resources. This ensures that after each speaker plays sound, the virtual image M appears in the correct position, 90° to the right of the user's head.

[0165] It is understandable that in addition to the changes in the direction and angle of the facial rotation mentioned above, the user may also have changes in the sitting position, movement of the body forward and backward, etc. In response to these changes in posture, the vehicle can dynamically track and adaptively adjust the audio played by the speakers, so that when the user perceives virtual sound and image, he can break free from the constraints of the properties of the speakers in the car and obtain an immersive auditory experience.

[0166] From the above content, it can be seen that in the cockpit control method provided in the embodiment of the present application, the first posture information and the second posture information can represent the information corresponding to the user in different postures in the cockpit. The vehicle can send an image corresponding to or matching the user's current posture to the head-mounted device, and control the speaker to play audio corresponding to or matching the user's current posture. Then, the head-mounted device can present a dynamic 3D display effect that changes with the user's posture to the user. The multiple speakers in the cockpit can present a dynamic and immersive playback effect that changes with the user's posture to the user. The user can experience media resources from visual and auditory aspects in the cockpit, giving the user a better experience and meeting the user's various experience needs.

[0167] Furthermore, in the cockpit control method shown in FIG5 , the vehicle can also control functional components within the cockpit based on the image information to provide functions that match the image information. For details, please refer to the aforementioned embodiments. This allows users to experience media resources more realistically from multiple angles or dimensions, making them feel as if they are in the scene.

[0168] The cockpit control method can provide users with a multi-dimensional experience of media resources, such as visual and experiential aspects. For example, as shown in FIG8 , the method may include the following steps S801 - S805 .

[0169] S801: The vehicle obtains media resources and the user's first position information in the cabin.

[0170] Among them, media resources include images.

[0171] Also, the content of S801 may refer to the relevant content of S501 in the aforementioned embodiment.

[0172] S802: The vehicle determines to send an image including a first display effect to the head-mounted device based on the first posture information.

[0173] The first display effect is a 3D effect obtained according to the first posture information.

[0174] Also, the content of S802 may refer to the relevant content of S502 in the aforementioned embodiment.

[0175] S803: The vehicle obtains second posture information of the user in the cabin.

[0176] The second posture information may represent the position and posture of the user at another moment after the first posture information is obtained. The first posture information and the second posture information may represent the information corresponding to different postures of the user in the cockpit.

[0177] Also, the content of S803 may refer to the related content of the aforementioned S504.

[0178] S804: The vehicle determines to send an image including a second display effect to the head-mounted device based on the second posture information.

[0179] The second display effect is a 3D effect obtained according to the second posture information.

[0180] Also, the content of S804 may refer to the related content of the aforementioned S505.

[0181] S805: The vehicle controls functional components in the cockpit to provide functions matching the image information based on the image information.

[0182] Among them, the functional components include one or more of a seat, a vehicle air conditioner, a vehicle air humidifier, a vehicle fragrance, and a vibration unit on the seat.

[0183] In some embodiments, a vehicle may obtain image information from a media resource, where the image information includes one or more of season information, weather information, temperature information, location information, scene information, and event information. The vehicle then controls functional components within the cabin to provide functions that match the image information.

[0184] Exemplarily, the functional component includes a vibration unit on the seat, which may include one or more units, each positioned at a different location on the seat. Based on the image information, the vehicle determines vibration information corresponding to the vibration unit, where the vibration information includes vibration frequency and / or vibration intensity. The vehicle then controls the vibration unit to vibrate based on the vibration information, allowing the user to experience a vibration sensation corresponding to the content presented in the image.

[0185] As another example, the functional components include one or more of an onboard air conditioner, an onboard air humidifier, and an onboard fragrance. Based on the image information, the vehicle determines setting information for the functional components. This setting information includes one or more of a temperature setting value for the onboard air conditioner, a humidity setting value for the onboard air humidifier, and odor information for the onboard fragrance, where the odor information includes a target fragrance type and / or odor concentration. The vehicle then adjusts one or more of the temperature of the onboard air conditioner, the humidity of the onboard air humidifier, and the odor of the onboard fragrance based on the setting information. This allows the user to experience seat position, temperature, humidity, odor, and so on, corresponding to the content presented in the image.

[0186] As can be seen from the above, the first and second posture information can represent the information corresponding to different user postures in the cabin. The vehicle can send an image corresponding to or matching the user's current posture to the head-mounted device. Furthermore, the vehicle can control functional components in the cabin to provide functions that match the image information based on the image information. Furthermore, not only can the head-mounted device present a dynamic 3D display effect that changes with the user's posture, but the vehicle can also provide users with experiences in other dimensions. Users can experience media resources from multiple perspectives in the cabin, giving them a better experience and meeting their various experience needs.

[0187] Furthermore, in the cockpit control method shown in FIG8 , the vehicle can also control each speaker to play the audio corresponding to the image in the media resource. Furthermore, as the user's posture changes, the audio corresponding to each speaker will also change. For details, please refer to the above embodiments. This allows the user to experience the media resource more realistically from multiple angles or dimensions, making the user feel as if they are in the scene.

[0188] It is understood that the collection, storage, use, processing, transmission, provision, and disclosure of the aforementioned user information (including user image information, user location, user posture, etc.) are in compliance with the provisions of relevant laws and regulations and do not violate public order and good morals. For example, in the embodiments of the present application, the processing of user information is performed with the user's authorization.

[0189] In some schemes, multiple embodiments of the present application can be combined, and the combined scheme can be implemented. Optionally, some operations in the process of each method embodiment are optionally combined, and / or the order of some operations is optionally changed. In addition, the execution order between the steps of each process is only exemplary and does not constitute a restriction on the execution order between the steps. There can also be other execution orders between the steps. It is not intended to indicate that the execution order is the only order in which these operations can be performed. Those of ordinary skill in the art will think of many ways to reorder the operations described in the embodiments of the present application. In addition, it should be noted that the process details involved in a certain embodiment of the present application are also applicable to other embodiments in a similar manner, or different embodiments can be used in combination.

[0190] Furthermore, some steps in the method embodiments may be equivalently replaced with other possible steps. Alternatively, some steps in the method embodiments may be optional and may be deleted in certain usage scenarios. Alternatively, other possible steps may be added to the method embodiments.

[0191] Furthermore, the various method embodiments may be implemented separately or in combination.

[0192] It is understandable that in order to achieve the above functions, the aforementioned vehicle includes hardware and / or software modules corresponding to the execution of each function. In combination with the algorithm steps of each example described in the embodiments disclosed herein, the present application can be implemented in the form of hardware or a combination of hardware and computer software. Whether a function is executed in the form of hardware or computer software driving hardware depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application in combination with the embodiments, but such implementation should not be considered to be beyond the scope of this application.

[0193] This embodiment can divide the vehicle into functional modules based on the above-described method examples. For example, each functional module can be divided into corresponding functional modules, or two or more functions can be integrated into a single processing module. The integrated modules can be implemented in hardware. It should be noted that the module division in this embodiment is illustrative and represents only a logical functional division. In actual implementation, other division methods may be used.

[0194] The embodiment of the present application provides an electronic system, which may include at least two speakers respectively arranged at different positions in the cabin, and the electronic system is connected to a head-mounted device. As shown in FIG9 , the electronic system further includes an information acquisition module 901 and a resource processing module 902 .

[0195] The information acquisition module 901 is used to acquire media resources and the first position information of the user in the cockpit, for example, by executing the relevant steps of the above S501.

[0196] The resource processing module 902 is configured to determine, based on the first posture information, whether to send an image including the first display effect to the head mounted device; and to control each speaker to play the audio corresponding to the speaker based on the first posture information, for example, by performing the above steps S502 and S503.

[0197] The information acquisition module 901 is further configured to acquire the second posture information of the user in the cockpit, for example, by executing the relevant steps of the above S504.

[0198] The resource processing module 902 is further configured to determine, based on the second posture information, whether to send an image including the second display effect to the head mounted device; and further configured to control each speaker to play audio corresponding to the speaker based on the second posture information, for example, by performing steps S505 and S506 above.

[0199] Optionally, the information acquisition module 901 may include the posture detection module and resource acquisition module in FIG2 , and the resource processing module 902 may include the rendering engine in FIG2 .

[0200] An embodiment of the present application provides another electronic system, including at least one functional component, the electronic system is connected to a head-mounted device, and the electronic system may also include an information acquisition module and a resource processing module.

[0201] The information acquisition module is used to acquire media resources and the user's first position information in the cabin, for example, by executing the relevant steps of S801 above.

[0202] The resource processing module is configured to determine, based on the first posture information, to send an image including the first display effect to the head mounted device, for example, by executing the relevant steps of S802 above.

[0203] The information acquisition module is further configured to acquire the second posture information of the user in the cockpit, for example, by executing the relevant steps of the above S803.

[0204] The resource processing module is further configured to determine, based on the second posture information, to send an image including the second display effect to the head mounted device, for example, by executing the relevant steps of the above S804.

[0205] The resource processing module is further configured to control the functional components in the cockpit to provide functions matching the image information based on the image information, for example, executing the relevant steps of the above S805.

[0206] An embodiment of the present application also provides a vehicle, as shown in FIG10 , which may include one or more processors 1001 , a memory 1002 , and a communication interface 1003 .

[0207] The memory 1002 and the communication interface 1003 are coupled to the processor 1001. For example, the memory 1002, the communication interface 1003 and the processor 1001 may be coupled together via a bus 1004.

[0208] The communication interface 1003 is used to transmit data with other devices. The memory 1002 stores computer program code. The computer program code includes computer instructions. When the computer instructions are executed by the processor 1001, the vehicle executes the cabin control method of the embodiment of the present application.

[0209] The processor 1001 may be a processor or a controller, such as a central processing unit (CPU), a general-purpose processor, a digital signal processor (DSP), an application-specific integrated circuit (ASIC), a field programmable gate array (FPGA), or other programmable logic devices, transistor logic devices, hardware components, or any combination thereof. It may implement or execute the various exemplary logic blocks, modules, and circuits described in conjunction with the present disclosure. The processor may also be a combination that implements computing functions, such as a combination of one or more microprocessors, a combination of a DSP and a microprocessor, and the like.

[0210] The bus 1004 may be a Peripheral Component Interconnect (PCI) bus or an Extended Industry Standard Architecture (EISA) bus. The bus 1004 may be divided into an address bus, a data bus, a control bus, etc. For ease of illustration, FIG10 shows only one thick line, but this does not mean that there is only one bus or only one type of bus.

[0211] An embodiment of the present application also provides a computer-readable storage medium, which includes computer instructions. When the computer instructions are executed on a vehicle, the vehicle executes the relevant method steps in the above method embodiment.

[0212] An embodiment of the present application further provides a computer program product, which, when executed on a computer, enables the computer to execute the relevant method steps in the above method embodiment.

[0213] Among them, the electronic system, vehicle, computer storage medium or computer program product provided in this application is used to execute the corresponding method provided above. Therefore, the beneficial effects that can be achieved can refer to the beneficial effects in the corresponding method provided above, and will not be repeated here.

[0214] Through the description of the above implementation methods, technical personnel in the relevant field can clearly understand that for the convenience and simplicity of description, only the division of the above-mentioned functional modules is used as an example. In actual applications, the above-mentioned functions can be assigned to different functional modules as needed, that is, the internal structure of the device can be divided into different functional modules to complete all or part of the functions described above.

[0215] In the several embodiments provided in this application, it should be understood that the disclosed devices and methods can be implemented in other ways. For example, the device embodiments described above are merely schematic. For example, the division of the modules or units is only a logical function division. In actual implementation, there may be other division methods, such as multiple units or components can be combined or integrated into another device, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, indirect coupling or communication connection of devices or units, which can be electrical, mechanical or other forms.

[0216] The units described as separate components may or may not be physically separate, and the components shown as units may be one physical unit or multiple physical units, that is, they may be located in one place or distributed in multiple places. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0217] In addition, the functional units in the various embodiments of the present application may be integrated into a single processing unit, or each unit may exist physically separately, or two or more units may be integrated into a single unit. The aforementioned integrated units may be implemented in the form of hardware or software functional units.

[0218] If the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a readable storage medium. Based on this understanding, the technical solution of the embodiment of the present application is essentially or the contributing part or all or part of the technical solution can be embodied in the form of a software product, which is stored in a storage medium and includes several instructions for enabling a device (which can be a single-chip microcomputer, chip, etc.) or a processor to execute all or part of the steps of the method described in each embodiment of the present application. The aforementioned storage medium includes various media that can store program codes, such as a USB flash drive, a mobile hard disk, a read-only memory (ROM), a random access memory (RAM), a magnetic disk or an optical disk.

[0219] The above content is only a specific embodiment of this application, but the scope of protection of this application is not limited to this. Any changes or replacements within the technical scope disclosed in this application should be included in the scope of protection of this application. Therefore, the scope of protection of this application should be based on the scope of protection of the claims.

Claims

1. A cockpit control method, characterized in that: Applied to a vehicle, the vehicle includes at least two speakers respectively arranged at different positions in the cabin, and the vehicle is connected to a head-mounted device; the method includes: Acquire media resources and first position information of the user in the cockpit; the media resources include an image and audio corresponding to the image; Sending an image including a first display effect to the head mounted device according to the first posture information; the first display effect is a three-dimensional effect obtained according to the first posture information; According to the first posture information, each of the speakers is controlled to play the audio corresponding to the speaker; and the audio corresponding to each speaker is obtained according to the audio in the media resource, the attribute information of the speaker and the first posture information; Acquiring second posture information of the user in the cockpit; Sending an image including a second display effect to the head mounted device according to the second posture information; the second display effect is a three-dimensional effect obtained according to the second posture information; According to the second posture information, each of the speakers is controlled to play the audio corresponding to the speaker; and the audio corresponding to each speaker is obtained according to the audio in the media resource, the attribute information of the speaker and the second posture information.

2. The method according to claim 1, characterized in that The obtaining of the first posture information of the user in the cockpit includes: Acquire first information sent by the head-mounted device; the first information is acquired by the head-mounted device through at least one sensor, and the first information includes head posture information of the user; acquiring second information through at least one sensor in the cabin; the second information including one or more of image information of the user, infrared information, seat position information of the seat where the user is seated, seat back angle information, seat angle information, or seat pressure information; The first posture information is determined based on the first information and the second information.

3. The method according to claim 1 or 2, characterized in that The second display effect is a change of the first display effect, and the change of the display effect includes a change in one or more of the image viewing angle, image content, or image size.

4. The method according to any one of claims 1 to 3, characterized in that The audio corresponding to each of the speakers changes according to the change of the user posture information, and the change of the audio includes a change in one or more of the audio frequency, audio amplitude, audio delay, audio phase, or audio spectrum characteristics.

5. The method according to any one of claims 1 to 4, characterized in that The method further comprises: Acquire image information of the image in the media resource; the image information includes one or more of season information, weather information, temperature information, location information, scene information, or event information; Controlling functional components in the cabin to provide functions matching the image information; the functional components include one or more of a seat, an onboard air conditioner, an onboard air humidifier, an onboard fragrance, or a vibration unit on a seat.

6. The method according to claim 5, characterized in that The functional component includes a vibration unit on the seat, the vibration unit includes one or more, and each of the vibration units is arranged at a different position on the seat; The controlling of the functional components in the cockpit to provide functions matching the image information includes: Determining vibration information corresponding to the vibration unit according to the image information; the vibration information includes vibration frequency and / or vibration intensity; The vibration unit is controlled to vibrate according to the vibration information.

7. The method according to claim 5 or 6, characterized in that The functional components include one or more of a vehicle air conditioner, a vehicle air humidifier, or a vehicle fragrance; The controlling of the functional components in the cockpit to provide functions matching the image information includes: Determining setting information of the functional component based on the image information; the setting information includes one or more of a temperature setting value corresponding to the vehicle air conditioner, a humidity setting value corresponding to the vehicle air humidifier, or odor information corresponding to the vehicle fragrance; the odor information includes a target fragrance type and / or odor concentration; According to the setting information, one or more of the temperature of the vehicle air conditioner, the humidity of the vehicle air humidifier, or the smell of the vehicle fragrance is adjusted.

8. The method according to any one of claims 1 to 7, characterized in that The sending, according to the first posture information, an image including a first display effect to the head mounted device, comprises: determining a first display parameter of the image according to the first posture information; the first display parameter is used to characterize a first display effect of the image; Rendering the image according to the first display parameter; The image after the rendering process is sent to the head mounted device.

9. The method according to any one of claims 1 to 8, characterized in that The step of controlling each of the speakers to play audio corresponding to the speaker according to the first posture information includes: For each of the speakers, acquiring attribute information of the speaker; the attribute information includes one or more of the position, sound emission direction, or frequency response of the speaker in the cabin; Determining first playback parameters of the audio corresponding to the speaker based on the audio in the media resource, the first posture information, and the attribute information of the speaker; Rendering the audio corresponding to the speaker according to the first playback parameter; The rendered audio is sent to the speaker, so that the speaker plays the rendered audio.

10. An electronic system, characterized in that: The electronic system comprises at least two speakers respectively arranged at different positions in the cabin, and the electronic system is connected to the head-mounted device; the electronic system also comprises an information acquisition module and a resource processing module; The information acquisition module is used to acquire media resources and the first posture information of the user in the cockpit; the media resources include images and audio corresponding to the images; The resource processing module is configured to send an image including a first display effect to the head mounted device based on the first posture information; the first display effect is a three-dimensional effect obtained based on the first posture information; and control each of the speakers to play audio corresponding to the speaker based on the first posture information; and, obtaining audio corresponding to each of the speakers based on the audio in the media resource, the attribute information of the speaker, and the first posture information; The information acquisition module is further used to obtain second posture information of the user in the cockpit; The resource processing module is further configured to send an image including a second display effect to the head mounted device based on the second posture information; the second display effect is a three-dimensional effect obtained based on the second posture information; and control each of the speakers to play audio corresponding to the speaker based on the second posture information. Furthermore, the audio corresponding to each of the speakers is obtained based on the audio in the media resource, the attribute information of the speaker, and the second posture information.

11. The system according to claim 10, wherein: The information acquisition module is specifically used to obtain the first information sent by the head-mounted device; the first information is obtained by the head-mounted device through at least one sensor, and the first information includes the user's head posture information; the second information is obtained through at least one sensor in the cabin; the second information includes one or more of the user's image information, infrared information, seat position information of the user's seat, seat back angle information, seat angle information, or seat pressure information; the first posture information is determined based on the first information and the second information.

12. The system according to claim 10 or 11, characterized in that The second display effect is a change of the first display effect, and the change of the display effect includes a change in one or more of the image viewing angle, image content, or image size.

13. The system according to any one of claims 10 to 12, characterized in that: The audio corresponding to each of the speakers changes according to the change of the user posture information, and the change of the audio includes a change in one or more of the audio frequency, audio amplitude, audio delay, audio phase, or audio spectrum characteristics.

14. The system according to any one of claims 10 to 13, characterized in that: The resource processing module is further used to obtain image information of images in the media resources; the image information includes one or more of season information, weather information, temperature information, location information, scene information, or event information; and control the functional components in the cabin to provide functions matching the image information; the functional components include one or more of seats, vehicle air conditioners, vehicle air humidifiers, vehicle fragrances, or vibration units on seats.

15. The system according to claim 14, wherein: The functional component includes a vibration unit on the seat, and the vibration unit includes one or more, and each vibration unit is set at a different position on the seat; the resource processing module is specifically used to determine the vibration information corresponding to the vibration unit based on the image information; the vibration information includes vibration frequency and / or vibration intensity; according to the vibration information, the vibration unit is controlled to vibrate.

16. The system according to claim 14 or 15, characterized in that The functional components include one or more of a vehicle air conditioner, a vehicle air humidifier, or a vehicle fragrance; the resource processing module is specifically used to determine setting information of the functional components based on the image information; the setting information includes one or more of a temperature setting value corresponding to the vehicle air conditioner, a humidity setting value corresponding to the vehicle air humidifier, and odor information corresponding to the vehicle fragrance; the odor information includes a target fragrance type and / or odor concentration; according to the setting information, one or more of the temperature of the vehicle air conditioner, the humidity of the vehicle air humidifier, or the odor of the vehicle fragrance is adjusted.

17. The system according to any one of claims 10 to 16, characterized in that: The resource processing module is specifically configured to determine a first display parameter of the image according to the first posture information; the first display parameter is used to represent a first display effect of the image; and render the image according to the first display parameter; The image after the rendering process is sent to the head mounted device.

18. The system according to any one of claims 10 to 17, characterized in that: The resource processing module is specifically configured to obtain attribute information of each speaker for each speaker; the attribute information includes one or more of the position, sound emission direction, or frequency response of the speaker in the cabin; Determining first playback parameters of the audio corresponding to the speaker based on the audio in the media resource, the first posture information, and the attribute information of the speaker; Rendering the audio corresponding to the speaker according to the first playback parameter; The rendered audio is sent to the speaker, so that the speaker plays the rendered audio.

19. A vehicle, characterized in that: The vehicle comprises a memory and one or more processors; the memory is coupled to the processor; wherein computer program code is stored in the memory, and the computer program code includes computer instructions, and when the computer instructions are executed by the processor, the vehicle executes the cockpit control method as described in any one of claims 1 to 9.

20. A computer-readable storage medium, characterized in that The method comprises computer instructions, which, when executed on a vehicle, enable the vehicle to execute the cockpit control method as claimed in any one of claims 1 to 9.

21. A computer program product, characterized in that When the computer program product is run on a vehicle, the vehicle is caused to execute the cockpit control method according to any one of claims 1 to 9.

Images