Cabin control method, electronic system, and vehicle

Abstract

The application provides a cockpit control method, an electronic system and a vehicle, relates to the technical field of electronic control, and can enable a user to more realistically experience media resources in a cockpit from the dimensions of vision and hearing and the like, and meet various experience requirements of the user. In the method, the vehicle can acquire posture information corresponding to different postures of the user, and render images and audio in the media resources according to the currently acquired posture information. When the posture of the user changes, the rendering of the images and the rendering of the audio will change, and then the display effect of the images and the audio will also change. The head-mounted device connected with the vehicle can present dynamic 3D visual effects of the images to the user according to the change of the posture of the user, and the multiple loudspeakers in the cockpit can present dynamic and immersive hearing effects to the user according to the change of the posture of the user. The user can experience the media resources from the aspects of vision and hearing and the like in the cockpit, the user has a better feeling, and various experience requirements of the user are met.

Description

Technical Field

[0001] This application relates to the field of electronic control technology, and in particular to a cockpit control method, electronic system, and vehicle. Background Technology

[0002] In recent years, with the rapid development of intelligent vehicles, users have increasingly higher demands for the functions they provide. For example, users not only expect intelligent vehicles to have excellent driving performance, but also a better entertainment experience.

[0003] Currently, some smart cars' audio-visual systems, display systems, or autonomous driving systems can connect with head-mounted devices such as virtual reality (VR) devices and augmented reality (AR) devices, thereby providing users with a high-definition large-screen viewing experience in the car cabin, or providing users with multi-dimensional virtual driving scene images in autonomous driving scenarios, allowing users to feel as if they are there.

[0004] However, the movie-watching experience or virtual scene experience provided above are relatively simple or basic and cannot meet the diverse experience needs of users. Summary of the Invention

[0005] This application provides a cockpit control method, electronic system, and vehicle that enable users to experience media resources more realistically from visual and auditory dimensions within the cockpit, thus meeting various user experience needs.

[0006] To achieve the above objectives, the embodiments of this application adopt the following technical solutions:

[0007] A first aspect provides a cockpit control method applied to a vehicle, the vehicle including at least two speakers respectively disposed at different locations within the cockpit, and the vehicle connected to a head-mounted device. In this method, the vehicle acquires media resources and a user's first pose information within the cockpit, wherein the media resources include an image and corresponding audio. Then, based on the first pose information, the vehicle sends an image including a first display effect to the head-mounted device, wherein the first display effect is a three-dimensional effect obtained based on the first pose information. Based on the first pose information, the vehicle controls each speaker to play audio corresponding to that speaker, wherein the audio corresponding to each speaker is obtained based on audio from the media resources, speaker attribute information, and the first pose information. The vehicle continues to acquire a second pose information of the user within the cockpit, and based on the second pose information, sends an image including a second display effect to the head-mounted device, wherein the second display effect is a three-dimensional effect obtained based on the second pose information. The vehicle then controls each speaker to play audio corresponding to that speaker based on the second pose information, and the audio corresponding to each speaker is obtained based on audio from the media resources, speaker attribute information, and the second pose information.

[0008] In the above method, the first pose information and the second pose information can represent the information corresponding to the user's different poses in the cockpit. The vehicle can send images corresponding to or matching the user's current pose to the head-mounted device, and control the speakers to play audio corresponding to or matching the user's current pose. Thus, the head-mounted device can present the user with dynamic three-dimensional visual effects that change with the user's pose, and the multiple speakers in the cockpit can present the user with dynamic and immersive auditory effects that change with the user's pose. The user can experience media resources from visual and auditory aspects in the cockpit, giving the user a better experience and meeting the user's various experience needs.

[0009] In one possible implementation of the first aspect, when a vehicle acquires media resources, it can do so either from the vehicle's storage space or from a server. Furthermore, the media resources include: two-dimensional media resources, three-dimensional media resources, panoramic media resources, VR media resources, or AR media resources. Thus, the vehicle can provide users with different media resources for their experience.

[0010] In one possible implementation of the first aspect, when the vehicle acquires the user's first posture information within the cabin, it may acquire first information transmitted by a head-mounted device, wherein the first information is acquired by the head-mounted device through at least one sensor and includes the user's head posture information. Furthermore, the vehicle acquires second information through at least one sensor within the cabin, wherein the second information includes one or more of the following: the user's image information, infrared information, seat position information of the user's seat, seat back angle information, seat angle information, or seat pressure information. The vehicle then determines the first posture information based on the first and second information.

[0011] In the above implementation method, the vehicle can obtain information collected by multiple sensors through multiple information input channels, and combine the information from multiple sensors to more accurately determine the user's position and posture information.

[0012] In one possible implementation of the first aspect, the second display effect is the result of a change in the first display effect, and the change in the display effect includes a change in one or more of the following: image viewing angle, image content, or image size. This results in a richer image display effect presented to the user by the head-mounted device that changes with the user's posture.

[0013] In one possible implementation of the first aspect, the audio corresponding to each speaker changes according to changes in the user's posture information, and the changes in audio include changes in one or more of the following: audio frequency, audio amplitude, audio delay, audio phase, or audio spectral characteristics. This results in a richer sound playback effect presented to the user by the multiple speakers in the cockpit, adapting to changes in the user's posture.

[0014] In one possible implementation of the first aspect, the vehicle acquires image information from images in a media resource, wherein the image information includes one or more of the following: seasonal information, weather information, temperature information, location information, scene information, or event information. Furthermore, functional components within the vehicle's control cockpit provide functions matching the image information, wherein these functional components include one or more of the following: seats, in-vehicle air conditioning, in-vehicle air humidifier, in-vehicle fragrance, or vibration units on the seats. In this way, the vehicle can also combine the image information to allow users to experience sensory, olfactory, and other dimensions corresponding to the content presented in the images, providing users with a richer experience.

[0015] In one possible implementation of the first aspect, the functional component includes a vibration unit on the seat, comprising one or more vibration units, each vibration unit being disposed at a different position on the seat. When the functional component within the vehicle control cockpit provides functionality matching image information, it can determine vibration information corresponding to the vibration unit based on the image information, wherein the vibration information includes vibration frequency and / or vibration intensity. The vehicle then controls the vibration unit to vibrate based on the vibration information. In this way, the user can experience a vibration sensation corresponding to the content presented in the image.

[0016] In one possible implementation of the first aspect, the functional components include one or more of a vehicle air conditioner, a vehicle humidifier, or a vehicle fragrance system. When the functional components within the vehicle control cockpit provide functions matching the image information, setting information for the functional components can be determined based on the image information. This setting information includes one or more of the following: a temperature setting value corresponding to the vehicle air conditioner, a humidity setting value corresponding to the vehicle humidifier, or odor information corresponding to the vehicle fragrance system. The odor information includes a target fragrance type and / or odor concentration. The vehicle adjusts one or more of the following based on the setting information: the temperature of the vehicle air conditioner, the humidity of the vehicle humidifier, or the odor of the vehicle fragrance system. In this way, the user can experience seat posture, temperature, humidity, and odor corresponding to the content presented in the image.

[0017] In one possible implementation of the first aspect, when the vehicle determines, based on the first pose information, to send an image including a first display effect to the head-mounted device, it can determine first display parameters of the image based on the first pose information, wherein the first display parameters characterize the first display effect of the image. Subsequently, the vehicle performs rendering processing on the image based on the first display parameters and sends the rendered image to the head-mounted device.

[0018] In the above implementation, the vehicle can determine the display parameters of the image that matches the user's current posture based on the user's posture information, and render the image according to the display parameters, so that the head-mounted device can display the image that matches the user's current posture, bringing a better visual experience to the user.

[0019] In one possible implementation of the first aspect, when the vehicle controls each speaker to play audio corresponding to the speaker based on the first pose information, it can obtain attribute information for each speaker. The attribute information includes one or more of the following: the speaker's position within the cabin, its sound emission direction, and its frequency response. The vehicle determines first playback parameters for the audio corresponding to the speaker based on the audio in the media resource, the first pose information, and the speaker attribute information. The vehicle then renders the audio corresponding to the speaker according to the first playback parameters and sends the rendered audio to the speaker so that the speaker plays the rendered audio.

[0020] In the above implementation, the vehicle can determine the playback parameters of the audio that match the user's current posture based on the user's posture information and the speaker's attribute information, and render the audio according to the playback parameters, so that each speaker can play the sound that matches the user's current posture, bringing a better listening experience to the user.

[0021] Secondly, a cockpit control method is provided, applied to a vehicle, the vehicle including at least one functional component, and the vehicle connected to a head-mounted device. In this method, the vehicle acquires media resources and a user's first pose information within the cockpit, wherein the media resources include images. Based on the first pose information, the vehicle sends an image including a first display effect to the head-mounted device, wherein the first display effect is a three-dimensional effect obtained based on the first pose information. The vehicle continues to acquire a second pose information of the user within the cockpit, and based on the second pose information, sends an image including a second display effect to the head-mounted device, wherein the second display effect is a three-dimensional effect obtained based on the second pose information. Furthermore, the vehicle controls the functional component within the cockpit to provide functions matching the image information based on the image information.

[0022] In the above method, the first pose information and the second pose information can represent the information corresponding to the user's different poses in the cockpit. The vehicle can send images corresponding to or matching the user's current pose to the head-mounted device. Furthermore, the vehicle can control the functional components in the cockpit to provide functions matching the image information based on the image information. Therefore, not only can the head-mounted device present a dynamic three-dimensional visual effect that changes with the user's pose, but the vehicle can also provide the user with experiences in other dimensions. Users can experience media resources from multiple perspectives within the cockpit, resulting in a better user experience and meeting diverse user needs.

[0023] In one possible implementation of the second aspect, when the vehicle acquires the user's first posture information within the cabin, it may acquire first information transmitted by a head-mounted device. This first information is acquired by the head-mounted device through at least one sensor and includes the user's head posture information. The vehicle acquires second information through at least one sensor within the cabin. This second information includes one or more of the following: the user's image information, infrared information, seat position information of the user's seat, seat back angle information, seat angle information, or seat pressure information. The vehicle determines the first posture information based on the first and second information.

[0024] In the above implementation method, the vehicle can obtain information collected by multiple sensors through multiple information input channels, and combine the information from multiple sensors to more accurately determine the user's position and posture information.

[0025] In one possible implementation of the second aspect, the second display effect is the result of a change in the first display effect, and the change in the display effect includes changes in one or more of the following: image viewing angle, image content, or image size. This results in a richer image display effect presented to the user by the head-mounted device that changes with the user's posture.

[0026] In one possible implementation of the second aspect, the functional component includes a vibration unit on the seat, comprising one or more vibration units, each vibration unit being disposed at a different position on the seat. When the functional component within the vehicle control cockpit provides functionality matching image information, it can determine vibration information corresponding to the vibration unit based on the image information, wherein the vibration information includes vibration frequency and / or vibration intensity. The vehicle then controls the vibration unit to vibrate based on the vibration information. In this way, the user can experience a vibration sensation corresponding to the content presented in the image.

[0027] In one possible implementation of the second aspect, the functional components include one or more of the following: a seat, a vehicle air conditioner, a vehicle humidifier, or a vehicle fragrance system. When the functional components within the vehicle control cockpit provide functions matching the image information, setting information for the functional components can be determined based on the image information. This setting information includes one or more of the following: a temperature setting for the vehicle air conditioner, a humidity setting for the vehicle humidifier, or odor information for the vehicle fragrance system. The odor information includes a target fragrance type and / or odor concentration. The vehicle adjusts one or more of the following based on the setting information: the temperature of the vehicle air conditioner, the humidity of the vehicle humidifier, or the odor of the vehicle fragrance system. In this way, the user can experience seat posture, temperature, humidity, and odor corresponding to the content presented in the image.

[0028] In one possible implementation of the second aspect, when the vehicle sends an image including a first display effect to the head-mounted device based on the first pose information, it can determine first display parameters of the image based on the first pose information, wherein the first display parameters are used to characterize the first display effect of the image. The vehicle then performs rendering processing on the image based on the first display parameters and sends the rendered image to the head-mounted device.

[0029] In the above implementation, the vehicle can determine the display parameters of the image that matches the user's current posture based on the user's posture information, and render the image according to the display parameters, so that the head-mounted device can display the image that matches the user's current posture, bringing a better visual experience to the user.

[0030] In one possible implementation of the second aspect, the vehicle further includes at least two speakers respectively disposed at different locations within the cabin, and the media resource includes audio corresponding to the image. After acquiring the user's first posture information within the cabin, the vehicle can further control each speaker to play the audio corresponding to the speaker based on the first posture information, and the audio corresponding to each speaker is obtained based on the audio in the media resource, the speaker's attribute information, and the first posture information. Subsequently, after acquiring the user's second posture information within the cabin, the vehicle controls each speaker to play the audio corresponding to the speaker based on the second posture information, and the audio corresponding to each speaker is obtained based on the audio in the media resource, the speaker's attribute information, and the second posture information.

[0031] In the above implementation, the vehicle can control the speakers to play audio that corresponds to or matches the user's current posture. As a result, multiple speakers in the cabin can present a dynamic and immersive playback effect that changes with the user's posture. The user can experience media resources from visual and auditory aspects in the cabin, giving the user a better experience and meeting the user's various experience needs.

[0032] In one possible implementation of the second aspect, the audio corresponding to each speaker changes according to changes in the user's posture information, and the changes in audio include changes in one or more of the following: audio frequency, audio amplitude, audio delay, audio phase, or audio spectral characteristics. This results in a richer sound playback effect presented to the user by the multiple speakers in the cockpit, adapting to changes in the user's posture.

[0033] In one possible implementation of the second aspect, when the vehicle controls each speaker to play audio corresponding to the speaker based on the first pose information, it can obtain attribute information for each speaker. This attribute information includes one or more of the following: the speaker's position within the cabin, its sound emission direction, and its frequency response. The vehicle determines first playback parameters for the audio corresponding to the speaker based on the audio in the media resource, the first pose information, and the speaker attribute information. The vehicle then renders the audio corresponding to the speaker based on the first playback parameters and sends the rendered audio to the speaker so that the speaker plays the rendered audio.

[0034] In the above implementation, the vehicle can determine the playback parameters of the audio that match the user's current posture based on the user's posture information and the speaker's attribute information, and render the audio according to the playback parameters, so that each speaker can play the sound that matches the user's current posture, bringing a better listening experience to the user.

[0035] Thirdly, an electronic system is provided, including at least two speakers respectively disposed at different locations within the cockpit, the electronic system being connected to a head-mounted device; the electronic system also includes an information acquisition module and a resource processing module.

[0036] The information acquisition module is used to acquire media resources and the user's first posture information in the cockpit; media resources include images and corresponding audio.

[0037] The resource processing module is used to send an image including a first display effect to the head-mounted device based on the first pose information; the first display effect is a three-dimensional effect obtained based on the first pose information; and to control each speaker to play the audio corresponding to the speaker based on the first pose information; the audio corresponding to each speaker is obtained based on the audio in the media resource, the speaker's attribute information, and the first pose information.

[0038] The information acquisition module is also used to acquire the user's second pose information inside the cockpit.

[0039] The resource processing module is also used to send an image including a second display effect to the head-mounted device according to the second pose information; the second display effect is a three-dimensional effect obtained according to the second pose information; and to control each speaker to play the audio corresponding to the speaker according to the second pose information; the audio corresponding to each speaker is obtained according to the audio in the media resource, the speaker attribute information, and the second pose information.

[0040] In one possible implementation of the third aspect, the information acquisition module is specifically used to acquire first information sent by the head-mounted device; the first information is acquired by the head-mounted device through at least one sensor, and includes the user's head posture information; acquire second information through at least one sensor in the cockpit; the second information includes one or more of the following: the user's image information, infrared information, the seat position information of the user's seat, the seat back angle information, the seat angle information, or the seat pressure information; and determine the first posture information based on the first information and the second information.

[0041] In one possible implementation of the third aspect, the second display effect is the effect after the first display effect has been changed, and the change in the display effect includes a change in one or more of the following: image viewpoint, image content, or image size.

[0042] In one possible implementation of the third aspect, the audio corresponding to each speaker changes according to changes in the user's pose information, and the changes in the audio include changes in one or more of the following: audio frequency, audio amplitude, audio delay, audio phase, or audio spectral characteristics.

[0043] In one possible implementation of the third aspect, the resource processing module is further configured to acquire image information of images in media resources; the image information includes one or more of seasonal information, weather information, temperature information, location information, scene information, or event information; and control the functional components in the cockpit to provide functions that match the image information; the functional components include one or more of seats, vehicle air conditioning, vehicle air humidifier, vehicle fragrance, or vibration units on the seats.

[0044] In one possible implementation of the third aspect, the functional components include a vibration unit on the seat, the vibration unit comprising one or more, and each vibration unit being disposed at a different position on the seat; a resource processing module, specifically configured to determine vibration information corresponding to the vibration unit based on image information; the vibration information including vibration frequency and / or vibration intensity; and to control the vibration unit to vibrate based on the vibration information.

[0045] In one possible implementation of the third aspect, the functional components include one or more of the following: a vehicle air conditioner, a vehicle air humidifier, or a vehicle fragrance; the resource processing module is specifically used to determine the setting information of the functional components based on the image information; the setting information includes one or more of the following: a temperature setting value corresponding to the vehicle air conditioner, a humidity setting value corresponding to the vehicle air humidifier, or odor information corresponding to the vehicle fragrance; the odor information includes a target fragrance type and / or odor concentration; and the setting information is used to adjust one or more of the following: the temperature of the vehicle air conditioner, the humidity of the vehicle air humidifier, or the odor of the vehicle fragrance.

[0046] In one possible implementation of the third aspect, the resource processing module is specifically used to determine the first display parameters of the image based on the first pose information; the first display parameters are used to characterize the first display effect of the image; the image is rendered based on the first display parameters; and the rendered image is sent to the head-mounted device.

[0047] In one possible implementation of the third aspect, the resource processing module is specifically used to acquire the attribute information of each speaker; the attribute information includes one or more of the speaker's position in the cockpit, sound direction, and frequency response; determine the first playback parameters of the audio corresponding to the speaker based on the audio in the media resource, the first pose information, and the speaker's attribute information; perform rendering processing on the audio corresponding to the speaker based on the first playback parameters; and send the rendered audio to the speaker so that the speaker can play the rendered audio.

[0048] Fourthly, an electronic system is provided, which includes at least one functional component and is connected to a head-mounted device. The electronic system also includes an information acquisition module and a resource processing module.

[0049] The information acquisition module is used to acquire media resources and the user's first posture information in the cockpit; media resources include images.

[0050] The resource processing module is used to send an image including a first display effect to the head-mounted device based on the first pose information; the first display effect is a three-dimensional effect obtained based on the first pose information.

[0051] The information acquisition module is also used to acquire the user's second pose information in the cockpit;

[0052] The resource processing module is also used to send an image including a second display effect to the head-mounted device based on the second pose information; the second display effect is a three-dimensional effect obtained based on the second pose information.

[0053] The resource processing module is also used to control the functional components in the cockpit to provide functions that match the image information, based on the image information.

[0054] In one possible implementation of the fourth aspect, the information acquisition module is specifically used to acquire first information sent by the head-mounted device, wherein the first information is acquired by the head-mounted device through at least one sensor, and the first information includes the user's head posture information; acquire second information through at least one sensor in the cockpit, wherein the second information includes one or more of the following: the user's image information, infrared information, the seat position information of the user's seat, the seat back angle information, the seat angle information, or the seat pressure information; and determine the first posture information based on the first information and the second information.

[0055] In one possible implementation of the fourth aspect, the second display effect is the effect after the first display effect has been changed, and the change in the display effect includes a change in one or more of the following: image viewpoint, image content, or image size.

[0056] In one possible implementation of the fourth aspect, the functional component includes a vibration unit on the seat, the vibration unit comprising one or more units, and each vibration unit being disposed at a different position on the seat. The resource processing module, specifically configured to control the functional component within the cockpit to provide functions matching image information, can determine vibration information corresponding to the vibration unit based on the image information, wherein the vibration information includes vibration frequency and / or vibration intensity; and control the vibration unit to vibrate based on the vibration information.

[0057] In one possible implementation of the fourth aspect, the functional components include one or more of the following: a seat, a vehicle air conditioner, a vehicle humidifier, or a vehicle fragrance. The resource processing module is specifically used to determine the setting information of the functional components based on the image information. The setting information includes one or more of the following: a temperature setting value corresponding to the vehicle air conditioner, a humidity setting value corresponding to the vehicle humidifier, or odor information corresponding to the vehicle fragrance. The odor information includes a target fragrance type and / or odor concentration. Based on the setting information, the module adjusts one or more of the following: the temperature of the vehicle air conditioner, the humidity of the vehicle humidifier, or the odor of the vehicle fragrance.

[0058] In one possible implementation of the fourth aspect, the resource processing module is specifically used to determine first display parameters of the image based on the first pose information, wherein the first display parameters are used to characterize a first display effect of the image. The electronic system, based on the first display parameters, performs rendering processing on the image and sends the rendered image to the head-mounted device.

[0059] In one possible implementation of the fourth aspect, the electronic system further includes at least two speakers respectively disposed at different locations within the cockpit, and the media resources include audio corresponding to the images. After acquiring the user's first pose information within the cockpit, the resource processing module is specifically configured to control each speaker to play audio corresponding to the speaker based on the first pose information, wherein the audio corresponding to each speaker is obtained based on the audio in the media resources, the speaker's attribute information, and the first pose information. Subsequently, after acquiring the user's second pose information within the cockpit, the resource processing module is specifically configured to control each speaker to play audio corresponding to the speaker based on the second pose information, wherein the audio corresponding to each speaker is obtained based on the audio in the media resources, the speaker's attribute information, and the second pose information.

[0060] In one possible implementation of the fourth aspect, the audio corresponding to each speaker changes according to changes in the user's pose information, and the changes in the audio include changes in one or more of the following: audio frequency, audio amplitude, audio delay, audio phase, or audio spectral characteristics.

[0061] In one possible implementation of the fourth aspect, the resource processing module is specifically used to acquire, for each speaker, the speaker's attribute information, wherein the attribute information includes one or more of the speaker's position in the cockpit, sound direction, or frequency response; determine the first playback parameters of the audio corresponding to the speaker based on the audio in the media resource, the first pose information, and the speaker's attribute information; render the audio corresponding to the speaker based on the first playback parameters; and send the rendered audio to the speaker so that the speaker plays the rendered audio.

[0062] Fifthly, a vehicle is provided, including a memory and one or more processors; the memory is coupled to the processors; wherein the memory stores computer program code, the computer program code including computer instructions, which, when executed by the processor, cause the vehicle to perform the cockpit control method as described in the first aspect and any implementation thereof, or to perform the cockpit control method as described in the second aspect and any implementation thereof.

[0063] A sixth aspect provides a computer-readable storage medium including computer instructions that, when executed on a vehicle, cause the vehicle to perform the cockpit control method as described in the first aspect and any implementation thereof, or to perform the cockpit control method as described in the second aspect and any implementation thereof.

[0064] In a seventh aspect, a computer program product is provided that, when the computer program product is run on a vehicle, causes the vehicle to execute the cockpit control method as described in the first aspect and any implementation thereof, or to execute the cockpit control method as described in the second aspect and any implementation thereof.

[0065] The beneficial effects that the electronic system provided in the third aspect, the electronic system provided in the fourth aspect, the vehicle provided in the fifth aspect, the computer-readable storage medium provided in the sixth aspect, and the computer program product provided in the seventh aspect can achieve can be referred to the beneficial effects that can be achieved in the first aspect and any of its implementations, or to the beneficial effects that can be achieved in the second aspect and any of its implementations, and will not be repeated here. Attached Figure Description

[0066] Figure 1 A schematic diagram of the vehicle structure shown in the embodiments of this application. Figure 1 ;

[0067] Figure 2 A schematic diagram of the vehicle structure shown in the embodiments of this application. Figure 2 ;

[0068] Figure 3 A schematic diagram of the vehicle structure shown in the embodiments of this application. Figure 3 ;

[0069] Figure 4 A schematic diagram of the vehicle structure shown in the embodiments of this application. Figure 4 ;

[0070] Figure 5 A schematic flowchart illustrating the cockpit control method in the embodiments of this application. Figure 1 ;

[0071] Figure 6 This is a schematic diagram of a vehicle control speaker playing audio, as shown in an embodiment of this application. Figure 1 ;

[0072] Figure 7 This is a schematic diagram of a vehicle control speaker playing audio, as shown in an embodiment of this application. Figure 2 ;

[0073] Figure 8 A schematic flowchart illustrating the cockpit control method in the embodiments of this application. Figure 2 ;

[0074] Figure 9 Here is a schematic diagram of the structure of the electronic system shown in the embodiments of this application:

[0075] Figure 10 A schematic diagram of the vehicle structure shown in the embodiments of this application. Figure 5 . Detailed Implementation

[0076] The technical solutions of the embodiments of this application will be described below with reference to the accompanying drawings. In the description of this application, unless otherwise stated, " / " indicates that the objects before and after are in an "or" relationship. For example, A / B can represent A or B. "And / or" in this application is merely a description of the relationship between related objects, indicating that three relationships can exist. For example, A and / or B can represent: A alone, A and B simultaneously, and B alone, where A and B can be singular or plural. Furthermore, in the description of this application, unless otherwise stated, "multiple" refers to two or more. "At least one of the following" or similar expressions refer to any combination of these items, including any combination of single or plural items. For example, at least one of a, b, or c can represent: a, b, c, ab, ac, bc, or abc, where a, b, and c can be single or multiple. Furthermore, to facilitate a clear description of the technical solutions in the embodiments of this application, the terms "first" and "second" are used in the embodiments of this application to distinguish identical or similar items with substantially the same function and effect. Those skilled in the art will understand that the terms "first" and "second" do not limit the quantity or execution order, and that "first" and "second" are not necessarily different. Meanwhile, in the embodiments of this application, the terms "exemplary" or "for example" are used to indicate that something is being used as an example, illustration, or description. Any embodiment or design scheme described as "exemplary" or "for example" in the embodiments of this application should not be construed as being more preferred or advantageous than other embodiments or design schemes. Specifically, the use of terms such as "exemplary" or "for example" is intended to present related concepts in a concrete manner for ease of understanding.

[0077] Furthermore, the business scenarios described in the embodiments of this application are for the purpose of more clearly illustrating the technical solutions of the embodiments of this application, and do not constitute a limitation on the technical solutions provided in the embodiments of this application. As those skilled in the art will know, with the emergence of new business scenarios, the technical solutions provided in the embodiments of this application are also applicable to similar technical problems.

[0078] The development of intelligent vehicles is currently receiving widespread attention, with more and more video and audio processing technologies being applied to automotive cockpit audio systems to meet users' increasingly higher requirements and growing needs for intelligent vehicle functions.

[0079] Some smart car audio-visual systems can connect to VR / AR glasses. The VR / AR glasses' data cable connects to the USB-C port in the rear of the car, projecting the image from the rear-seat display onto the VR / AR glasses. This provides users with a high-definition, large-screen viewing experience, utilizing the rich video resources on the car's infotainment system (or in-vehicle terminal) to offer a mobile cinema service. Smart cars can also interact with VR / AR head-mounted devices. For example, the smart car can collect images of its surroundings and create VR / AR 3D virtual scenes (such as autonomous driving scenarios) based on these images. Then, responding to user-inputted interactive control commands, it can synthesize dynamic effects with the 3D virtual scene to create a virtual scene image, which is then displayed on VR / AR head-mounted devices.

[0080] However, the above methods only provide users with a visual experience and do not allow users to have a multi-sensory experience through hearing, touch, smell, etc. In other words, users cannot have a more immersive experience.

[0081] In addition, some VR / AR glasses provide users with a fixed 3D visual effect, meaning that the content viewed by the user does not change dynamically with the user's head movements, resulting in a poor user experience.

[0082] In addition, some smart cars are equipped with 3D / 4D sound systems, which can provide users with 3D / 4D stereo and surround sound audio effects. However, due to the limitations of the actual physical environment inside the cabin, the positional relationship between the user and the cabin sound system often cannot meet the user's requirements for standard surround sound. Furthermore, when the user's position and / or posture changes, the user's auditory experience cannot change accordingly, resulting in a poor auditory experience.

[0083] The above solutions offer relatively simple or limited user experience features, failing to meet diverse user needs.

[0084] Based on the above, this application provides a cockpit control method. The vehicle includes at least two speakers respectively disposed at different locations within the cockpit, and is also connected to a head-mounted device. The vehicle acquires media resources and the user's first pose information within the cockpit, wherein the media resources include images and corresponding audio. Then, based on the first pose information, the vehicle sends an image including a first display effect to the head-mounted device, wherein the first display effect is a three-dimensional effect obtained based on the first pose information. Based on the first pose information, the vehicle controls each speaker to play audio corresponding to that speaker, wherein the audio corresponding to each speaker is obtained based on the audio in the media resources, the speaker's attribute information, and the first pose information. The vehicle continues to acquire the user's second pose information within the cockpit, and based on the second pose information, sends an image including a second display effect to the head-mounted device, wherein the second display effect is a three-dimensional effect obtained based on the second pose information. The vehicle then controls each speaker to play audio corresponding to that speaker based on the second pose information, wherein the audio corresponding to each speaker is obtained based on the audio in the media resources, the speaker's attribute information, and the second pose information.

[0085] In the above method, the first pose information and the second pose information can represent the information corresponding to the user's different poses in the cockpit. The vehicle can send images corresponding to or matching the user's current pose to the head-mounted device, and control the speakers to play audio corresponding to or matching the user's current pose. Thus, the head-mounted device can present the user with dynamic 3D visual effects that change with the user's pose, and the multiple speakers in the cockpit can present the user with dynamic and immersive auditory effects that change with the user's pose. The user can experience media resources from visual and auditory aspects in the cockpit, giving the user a better experience and meeting the user's various experience needs.

[0086] Alternatively, this application embodiment also provides another cockpit control method applied to a vehicle, the vehicle including at least one functional component, and the vehicle connected to a head-mounted device. The vehicle acquires media resources and a user's first pose information within the cockpit, wherein the media resources include images. Based on the first pose information, the vehicle sends an image including a first display effect to the head-mounted device, wherein the first display effect is a three-dimensional effect obtained based on the first pose information. The vehicle continues to acquire a second pose information of the user within the cockpit, and based on the second pose information, sends an image including a second display effect to the head-mounted device, wherein the second display effect is a three-dimensional effect obtained based on the second pose information. Furthermore, the vehicle controls the functional component within the cockpit to provide functions matching the image information based on the image information.

[0087] In the above method, the first and second pose information can represent the information corresponding to the user's different poses in the cockpit. The vehicle can send images corresponding to or matching the user's current pose to the head-mounted device. Furthermore, the vehicle can control the functional components in the cockpit to provide functions matching the image information based on the image information. Therefore, not only can the head-mounted device present a dynamic 3D visual effect that changes with the user's pose, but the vehicle can also provide the user with experiences in other dimensions. Users can experience media resources from multiple perspectives within the cockpit, resulting in a better user experience and meeting diverse user needs.

[0088] In some possible implementations, the vehicle described above may include at least one controller for controlling different components or different areas of the vehicle respectively. For example, see... Figure 1 As shown, a vehicle may include a vehicle domain controller, an autonomous driving domain controller, and a smart cockpit domain controller, and the controllers can communicate with each other.

[0089] The vehicle domain controller (VDC) is responsible for overall vehicle control and has high requirements for real-time performance and safety. The autonomous driving domain controller (ADAS / AD domain controller, ADC) is responsible for perception, decision-making, and control functions related to autonomous driving. The cockpit domain controller (CDC) is responsible for cockpit intelligence functions such as human-machine interaction. Alternatively, the autonomous driving domain controller can also be referred to as the mobile data center (MDC).

[0090] The cockpit control method provided in this embodiment can be applied to, for example... Figure 1 The intelligent cockpit domain controller shown.

[0091] In some other possible implementations, the vehicle may also include at least one functional module for providing vehicle functions. For example, see... Figure 2 As shown, the vehicle may include a resource acquisition module, a rendering engine, a pose detection module, etc. Furthermore, the vehicle may include multiple speakers, each of which can be positioned at a different location within the vehicle's cabin. The vehicle can also be connected to a head-mounted device. The vehicle and the head-mounted device can be connected via wired or wireless means.

[0092] The resource acquisition module described above can acquire media resources, which may include images and corresponding audio. Furthermore, the resource acquisition module can also send media resources to the rendering engine.

[0093] The aforementioned pose detection module can acquire the pose information of the user inside the cockpit and send the user's pose information to the rendering engine.

[0094] Since the user's pose (position and attitude) in the cockpit is not fixed, the pose detection module can obtain the user's pose information in real time. This allows the subsequent rendering engine to determine whether the user's pose has changed based on the different pose information received before and after. Furthermore, it can render media resources based on the currently received pose information, ensuring that the display and playback of media resources match the current pose information, i.e., match the user's current pose.

[0095] The aforementioned rendering engine can render images, audio, and other media resources based on the currently received user pose information, so that the display effect of the images and the playback effect of the audio can better match the user's current position and posture, or enable the user to receive better image display effects and audio playback effects in the current position and posture.

[0096] The rendering engine can send the rendered images to the head-mounted device for display, and send the rendered audio to different speakers in the cockpit for playback.

[0097] The aforementioned head-mounted device can be VR / AR glasses or VR / AR helmets, etc.

[0098] When an image is displayed on a head-mounted device, it can present a 3D display effect. The rendering engine receives the user's posture information in real time and renders the image according to the current user's posture. If the user's posture changes, the rendering engine will also change the image rendering, thus changing the 3D display presented on the head-mounted device. This achieves the goal of dynamically adjusting the image display effect according to the user's posture and matching the 3D display characteristics of the head-mounted device, bringing a better visual experience to the user.

[0099] For example, the viewing angle, image size, and image content displayed on the head-mounted device can be adjusted according to changes in the user's posture.

[0100] The aforementioned speakers can be door speakers, ceiling speakers, surround speakers, center speakers, headrest speakers, and subwoofers, etc. This application does not impose specific limitations on these aspects.

[0101] When audio is played through the speakers, since there are multiple speakers in the cockpit, the rendering engine can determine the playback effect for each speaker based on the speaker's attribute information (such as its position, direction of sound, frequency response, and device attributes), the user's pose information, and the audio itself. Based on this playback effect, the engine renders the audio for each speaker separately and sends the rendered audio to the corresponding speaker for playback. Each speaker corresponds to a different audio file.

[0102] It is understandable that, due to the different attributes of each speaker, as well as the different distances and relative positions between them and the user, the same audio obtained by the resource acquisition module will result in different audio after being rendered for different speakers. This allows the user to obtain a more immersive, surround sound experience from the sound emitted by different speakers after it has been propagated to the user, based on the user's current position.

[0103] If the user's posture changes, the rendering engine will also change the audio rendering for each speaker, thus changing the audio corresponding to each speaker. This achieves the goal of dynamically adjusting the audio playback effect according to the user's posture, bringing a better listening experience to the user.

[0104] For example, the aforementioned audio changes include changes in one or more of the following: audio frequency, audio amplitude, audio delay, audio phase, and audio spectral characteristics. Alternatively, since the vehicle inputs an audio signal to the speaker, the audio changes can also be expressed as changes in one or more of the following: audio signal frequency, signal amplitude, signal delay, signal phase, and signal spectral characteristics.

[0105] For example, the user's posture may include head posture, body posture, etc. Head posture further includes the angle of head rotation, the direction of rotation, the angle between the head and the body, etc.; body posture includes the angle of body rotation, the direction of rotation, etc.

[0106] In some possible implementations, the rendering engine may also include multiple rendering modules, each rendering different types of media resources. For example, see... Figure 3 As shown, the rendering engine can include a visual rendering module and an auditory rendering module. The visual rendering module renders images from media resources, while the auditory rendering module renders audio from media resources. Furthermore, the visual rendering module sends the rendered images to the head-mounted device for display, while the auditory rendering module sends the rendered audio for different speakers to their respective speakers for playback.

[0107] In some possible implementations, the pose detection module may also include at least one sensor that can acquire information characterizing the user's position and posture. For example, see... Figure 3 As shown, the pose detection module may include a pressure sensor, a camera, an infrared sensor (or an infrared probe), a seat sensor, etc.

[0108] The pressure sensor can be installed under each seat in the cabin to detect seat information such as the pressure it bears and send the seat pressure information to the rendering engine. The rendering engine then uses the seat pressure information to determine whether a user is sitting in the seat and the user's weight.

[0109] The system can include one or more cameras, positioned anywhere within the cockpit that can capture images of the seats. When a user is seated, the camera captures the user's image and sends this image information to the rendering engine, which then determines the user's posture and other parameters based on this information.

[0110] Alternatively, the camera can capture images of the seat and send the image information to the rendering engine, which can then determine whether a user is sitting in the seat and the user's posture based on the image information.

[0111] Multiple infrared sensors (or infrared probes) can be installed and placed at the corresponding location for each seat in the cockpit. The infrared sensors can detect infrared information on the corresponding seat and send the infrared information to the rendering engine, which then determines whether a user is sitting in that seat based on the infrared information.

[0112] The aforementioned seat sensors can be installed on the seat to acquire other seat information, such as the seat back angle, seat position, seat height, and seat angles (e.g., the angle between the seat cushion and the vehicle's underside plane, or the pitch angle). The seat sensors can send this seat information to the rendering engine, which then determines the user's position and posture based on this information.

[0113] Alternatively, the seat sensor can also be a seat adjustment unit installed on the seat, used to adjust the seat. The seat adjustment unit can send seat information to the rendering engine, which then determines the user's position and posture based on the seat information.

[0114] In some possible implementations, since the head-mounted device includes posture sensors, the pose detection module can also acquire the user's head posture information obtained by the posture sensors in the head-mounted device. Then, the pose detection module sends the user's head posture information to the rendering engine, which determines the user's head posture based on this information.

[0115] Alternatively, in some possible implementations, the head-mounted device can also acquire the user's head posture information through a posture sensor and directly send the user's head posture information to the rendering engine, which then determines the user's head posture based on the head posture information. This application does not impose specific limitations on the method of acquiring head posture information from the head-mounted device.

[0116] The rendering engine can combine information from multiple input channels (such as multiple sensors) to more accurately determine the user's pose. This allows for more accurate rendering of media resources based on the user's pose. When the user experiences media resources using a head-mounted device connected to the vehicle, the head-mounted device can present a better 3D display effect based on the rendered image sent by the rendering engine. This also allows the multiple speakers in the cabin to present a better immersive sound experience based on the rendered audio sent by the rendering engine.

[0117] In some possible implementations, the images in the aforementioned media resources may also include image information, such as seasonal information, weather information, temperature information, location information, scene information, event information, etc. The image information may be identified by the rendering engine from the content displayed in the image, or it may be carried in the media resources. This application embodiment does not impose specific limitations on this.

[0118] The aforementioned rendering engine can also connect with functional components within the vehicle's cabin and, based on image information, control these components to provide functions that match the image information. This allows the functional components to meet the user's needs when viewing media resources, enabling the user to experience a multi-dimensional sensory experience corresponding to the image and providing a richer user experience.

[0119] For example, see Figure 4 As shown, the aforementioned functional components may include seats, in-vehicle air conditioners, in-vehicle humidifiers, in-vehicle fragrance systems, and vibration units on the seats. Furthermore, the rendering engine can perform functional simulations using different components; for example, it can simulate position using the seats, temperature using the in-vehicle air conditioners, humidity using the in-vehicle humidifiers, scent using the in-vehicle fragrance system, and vibration using the vibration units on the seats. Correspondingly, the functions of these components may include, during position simulation, adjusting the seat back angle, seat angle, and seat position; during temperature simulation, adjusting the temperature setting of the in-vehicle air conditioner; during humidity simulation, adjusting the humidity setting of the in-vehicle humidifier; during scent simulation, adjusting the fragrance type and odor concentration of the in-vehicle fragrance system; and during vibration simulation, adjusting the vibration frequency / intensity of the vibration units.

[0120] For example, if the image information contains scene information such as a game (e.g., a roller coaster), the rendering engine can determine the target posture information of a user riding the roller coaster in the game based on this image information, such as the user's height, position, and upper body angle. Then, based on the user's target posture information, it can adjust one or more of the following: the seat back angle, seat angle, and seat position. This provides the user with a more realistic scene effect in terms of tactile perception.

[0121] Furthermore, as described in the foregoing embodiments, since the seat may be equipped with a seat adjustment unit, the rendering engine can further adjust the seat by controlling the seat adjustment unit.

[0122] Understandably, in some possible implementations, in order to ensure the user's experience when watching game screens, the seat adjustment unit can also be set as a low-latency / fast-adjustment unit, so that the rendering engine can adjust the seat more quickly and match the changes in the scene screen.

[0123] For example, if the image information contains seasonal information (such as winter, summer, spring, late autumn), weather information (such as sunny, cloudy, foggy), temperature information (such as hot, cold), and location information (such as desert, rainforest, outdoors, garden), the rendering engine can determine the appropriate temperature settings for the car's air conditioning, humidity settings for the car's humidifier, and scent information for the car's fragrance (such as the target fragrance type and / or scent concentration) based on the image information. Then, based on this setting information, the engine can adjust the temperature of the car's air conditioning, the humidity of the car's humidifier, and the scent of the car's fragrance. This provides users with a more realistic scene experience, allowing them to feel the seat position, temperature, humidity, and scent corresponding to the content presented in the image.

[0124] For example, if the image information contains scene information such as explosion, earthquake, or horror, the rendering engine can determine the vibration information of the vibration unit based on the image information, such as the vibration frequency and intensity that should be applied to the user. Then, based on the vibration information, it can control the vibration unit to vibrate and / or adjust the vibration frequency and intensity of the vibration unit, thereby providing the user with a realistic vibration sensation corresponding to the scene in the image and improving the user experience.

[0125] Furthermore, multiple vibration units can be installed on the seat, which can be located in the seat back, seat cushion, armrests, etc. The rendering engine can determine the vibration information of the corresponding vibration units at different positions on the seat based on image information, and then control different vibration units to vibrate at different frequencies and / or intensities, thus providing users with a more realistic vibration sensation.

[0126] It is understood that the above-described adjustments to various functional components within the cockpit, and the types of image information upon which these adjustments are based, are merely examples. In other possible implementations, one or more of the various functional components may be adjusted separately using one or more of a variety of image information; this application does not impose specific limitations on this approach.

[0127] In some possible implementations, the aforementioned media resources can also include 2D media resources, 3D media resources, panoramic media resources (or 360° panoramic media resources), virtual reality (VR) media resources, or augmented reality (AR) media resources. Regardless of the type of media resource, the rendering engine can render it into a resource with a 3D display effect that can be displayed on a head-mounted device, thereby providing a better viewing experience for users wearing head-mounted devices.

[0128] Furthermore, the media resources can be stored in the vehicle's local storage space, or they can be obtained from the server by the resource acquisition module. Alternatively, when the vehicle is connected to some terminal devices (such as mobile phones, tablets, etc.), the media resources can also be obtained from the terminal devices. This application embodiment does not impose specific limitations on this.

[0129] The aforementioned methods for providing users with visual, auditory, and haptic experiences can be implemented through any two or all combinations. For example, visual experience + auditory experience, visual experience + haptic experience, visual experience + auditory experience + haptic experience, etc. In this way, the rendering engine can adjust the display effect of images on the head-mounted device based on the user's pose information, as well as adjust the audio in the cockpit speakers or the functions provided by the cockpit components. This allows users to experience media resources more realistically from multiple angles or dimensions, creating an immersive experience.

[0130] The aforementioned cockpit control method can provide users with a multi-dimensional experience of media resources, including visual and auditory elements. For example, see [link to example]. Figure 5 As shown, the method may include the following steps S501-S506.

[0131] S501, the vehicle acquires media resources and the user's first position information in the cockpit.

[0132] The media resources include images and corresponding audio. Furthermore, the media resources can be any of the following: 2D media resources, 3D media resources, panoramic media resources, VR media resources, or AR media resources. Additionally, the images can be any of the following: 2D images, 3D images, panoramic images, VR images, or AR images.

[0133] Furthermore, the vehicle can obtain the user's initial posture information within the cabin based on at least one sensor installed in the cabin and / or sensors in the head-mounted device (such as a posture sensor).

[0134] Since users are not always stationary inside the cabin, their posture information changes. The first posture information represents the user's position and attitude at a given moment. The vehicle acquires this posture information in real time.

[0135] In some embodiments, the sensors in the head-mounted device can acquire first information, which may include the user's head posture information. At least one sensor in the cockpit can acquire second information, which may include one or more of the following: the user's image information, infrared information, the seat position information of the user's seat, the seat back angle information, the seat angle information, and the seat pressure information.

[0136] The vehicle can determine the user's first-position information within the cabin based on the first and second information. Since the information comes from multiple sources or channels (such as at least one sensor, a head-mounted device, etc.) and is of various types (such as the user's head posture information, the user's image information, seat angle information, seat position information, seat back angle information, seat pressure information, etc.), the vehicle can combine multiple pieces of information to more accurately determine the user's first-position information. This allows the vehicle to provide the user with better-looking images and more immersive sound based on the first-position information.

[0137] S502, The vehicle sends an image including the first display effect to the head-mounted device based on the first posture information.

[0138] Since the images in the media resources need to be displayed on the head-mounted device, the initial display effect of the images is a 3D effect obtained based on the user's pose information. Furthermore, the initial display effect is matched with the user's current position and pose.

[0139] In some embodiments, the vehicle may further determine first display parameters of the image based on the first pose information. The first display parameters characterize a first display effect of the image, or in other words, the first display parameters correspond to the first display effect. Furthermore, the first display parameters may include color, pixels, refresh rate, brightness, viewing angle, etc., for image display that matches the user's current pose. Subsequently, the vehicle renders the image based on the first display parameters and sends the rendered image to the head-mounted device, thereby causing the head-mounted device to display the rendered image and present the first display effect.

[0140] In this way, the content displayed on the head-mounted device can match the user's current posture, providing a better viewing experience.

[0141] S503: Based on the first position information, the vehicle controls each speaker to play the audio corresponding to the speaker.

[0142] Each speaker has a different audio source, and the audio source for each speaker is obtained based on the audio in the media resource, the speaker's attribute information, and the first-position information.

[0143] The aforementioned speaker attribute information may include one or more of the following: the speaker's own device attributes, the speaker's location in the cockpit, the direction of sound emission, and the frequency response.

[0144] The vehicle can determine how each speaker in the cabin should emit sound, with the user as the sound field center, so that the audio played by all speakers can create an immersive playback effect such as stereo or surround sound at the user's location. For example, the user can receive sounds from different speakers with the same playback effect at their current location. The user's current location can be determined based on the aforementioned first-position information; and how each speaker emits sound can be obtained from the audio in the media resources, the speaker's attribute information, and the first-position information.

[0145] It is understandable that how each speaker produces sound depends not only on the speaker's attribute information, but also on the distance between the speaker and the user, the user's posture, and other factors.

[0146] Furthermore, in some embodiments, for each speaker, the vehicle can acquire the speaker's attribute information and determine the first playback parameters for the audio corresponding to the speaker based on the audio in the media resource, the first pose information, and the speaker's attribute information. The first playback parameters may include the amplitude, frequency, delay, phase, and spectral characteristics of the audio matching the user's current pose. Then, the vehicle renders the audio according to the first playback parameters and sends the rendered audio to the corresponding speaker, causing the speaker to play the rendered audio.

[0147] For example, see Figure 6As shown, the cabin includes speakers 1, 2, 3, and 4. Each speaker is located in a different position within the cabin, such as in the doors, center console, or headrests. After the user sits down, because the attribute information of each speaker may be different, and the distance between each speaker and the user may also be different, the vehicle can determine the first playback parameter corresponding to each speaker. For example, speaker 1 corresponds to playback parameter 1, speaker 2 to playback parameter 2, speaker 3 to playback parameter 3, and speaker 4 to playback parameter 4, and each playback parameter is different. The vehicle renders the audio according to the different playback parameters, obtaining rendered audio 1, audio 2, audio 3, and audio 4. Then, audio 1 is sent to speaker 1 for playback, audio 2 to speaker 2 for playback, audio 3 to speaker 3 for playback, and audio 4 to speaker 4 for playback. After each speaker emits sound, the user, in their current position and posture, experiences an immersive auditory effect such as stereo and surround sound.

[0148] In some embodiments, the vehicle can obtain the corresponding media resource in response to the user's selection of the media resource.

[0149] For example, after the vehicle is connected to the head-mounted device, the in-vehicle terminal in the cabin may display a resource page for the user to select media resources. This resource page may include various media resources, such as video resources and game resources. When the user selects a media resource on the resource page, the vehicle will respond to the user's operation, obtain the corresponding media resource, and perform rendering and other processing on the images and sounds in the media resource. After processing, the rendered image will be sent to the head-mounted device for display, and the rendered audio will be sent to the speakers for playback.

[0150] For another example, after the vehicle connects to the head-mounted device, the head-mounted device may display a resource page for the user to select media resources. This resource page may include various media resources, such as video resources and game resources. After the user wears the head-mounted device, they can select a media resource on the resource page through head movements, etc. The head-mounted device will respond to the user's operation by sending a resource request to the vehicle. The vehicle will obtain the corresponding media resource according to the resource request, and then process the images and sounds in the media resource, such as rendering. After processing, the rendered image will be sent to the head-mounted device for display, and the rendered audio will be sent to the speakers for playback.

[0151] S504, The vehicle acquires the user's second posture information within the cockpit.

[0152] The second pose information can represent the user's position and posture at another moment after the first pose information is acquired. The first pose information and the second pose information can represent the information corresponding to the user's different poses in the cockpit.

[0153] Furthermore, the method for obtaining the second pose information and related content can be found in the aforementioned S501.

[0154] S505. Based on the second pose information, the vehicle determines to send an image including the second display effect to the head-mounted device.

[0155] The second display effect is a 3D effect obtained based on the second pose information.

[0156] Furthermore, the method for determining the second display effect and the determination of the image including the second display effect can be referred to the relevant content of S502 above.

[0157] S506. Based on the second position information, the vehicle controls each speaker to play the audio corresponding to the speaker.

[0158] Each speaker has a different audio source, and the audio source for each speaker is obtained based on the audio source in the media resource, the speaker's attribute information, and the second pose information.

[0159] Furthermore, the determination of the audio corresponding to each speaker can be found in the relevant content of S503 mentioned above.

[0160] Understandably, as a user's position and posture within the cabin change, the vehicle will output images with different display effects to the head-mounted display. Consequently, the image displayed on the head-mounted display will change according to the user's posture. These changes may include alterations to one or more of the following: image viewing angle, image content, and image size. This results in a better visual experience for the user.

[0161] Furthermore, as the user's position and posture change, the audio presented by each speaker will also change accordingly. This change includes variations in one or more of the following: audio frequency, audio amplitude, audio delay, audio phase, and audio spectral characteristics. In this way, by controlling the speaker's audio output based on the user's posture, the vehicle ensures that the user experiences a consistent, immersive, surround sound effect regardless of their position, thus providing a better auditory experience.

[0162] For example, see Figure 7As shown in (a), the user is in the first position, such as the user is in the driver's seat with their face facing directly in front of the user. Based on the user's first position information, the attribute information of each speaker and the audio in the media resources, the vehicle can determine the audio corresponding to each speaker in the cabin and control the speakers to play the corresponding audio. Thus, the virtual sound image M formed by the sound played by multiple speakers is located at a position 45° to the right front of the user's head.

[0163] Afterwards, the user changes position to a second position, such as remaining in the driver's seat but turning their face 45° to the left. (See below) Figure 7 As shown in (b) above. At this time, the user needs to hear the absolutely accurate direction of the virtual sound image M, which should be located at 90° (45°+45°) to the right of the user's head. However, due to the asymmetrical arrangement of the user's position and the speakers in the vehicle, the virtual sound image M may not be at the 90° position. Therefore, the vehicle can adjust the audio corresponding to each speaker (such as audio amplitude, audio delay, audio spectrum, etc.) according to the user's current second pose information, the attribute information of each speaker, and the audio in the media resources, so that after each speaker plays the sound, the virtual sound image M appears in the correct position, that is, at 90° to the right of the user's head.

[0164] Understandably, in addition to changes in facial rotation direction and angle, users can also change their seating position and move their body forward or backward. The vehicle can dynamically track and adaptively adjust the audio played by the speakers in response to these changes in posture, so that when users perceive the virtual sound image, they can break free from the constraints of the in-vehicle speaker attributes and obtain an immersive auditory experience.

[0165] As can be seen from the above, the cockpit control method provided in this application embodiment can represent the information corresponding to the user's different postures in the cockpit. The vehicle can send an image corresponding to or matching the user's current posture to the head-mounted device, and control the speakers to play audio corresponding to or matching the user's current posture. Thus, the head-mounted device can present a dynamic 3D display effect that changes with the user's posture, and the multiple speakers in the cockpit can present a dynamic and immersive playback effect that changes with the user's posture. The user can experience media resources from visual and auditory aspects in the cockpit, giving the user a better experience and meeting the user's various experience needs.

[0166] And, in the above Figure 5 In the cockpit control method shown, the vehicle can also control the functional components in the cockpit to provide functions that match the image information based on the image information, as detailed in the foregoing embodiments. This allows users to experience media resources more realistically from multiple angles or dimensions, creating an immersive experience.

[0167] The aforementioned cockpit control method can provide users with a multi-dimensional experience regarding media resources, including visual and experiential aspects. For example, see [link to example]. Figure 8 As shown, the method may include the following steps S801-S805.

[0168] S801, the vehicle acquires media resources and the user's first position information in the cockpit.

[0169] Media resources include images.

[0170] Furthermore, the content of S801 can be found in the relevant content of S501 in the aforementioned embodiments.

[0171] S802, The vehicle determines to send an image including the first display effect to the head-mounted device based on the first posture information.

[0172] The first display effect is the 3D effect obtained based on the first pose information.

[0173] Furthermore, the content of S802 can be found in the relevant content of S502 in the aforementioned embodiments.

[0174] S803, The vehicle acquires the user's second posture information within the cockpit.

[0175] The second pose information represents the user's position and posture at another moment after the first pose information is acquired. The first and second pose information can represent the information corresponding to the user's different poses within the cockpit.

[0176] Furthermore, the content of S803 can be found in the relevant content of S504 mentioned above.

[0177] S804. Based on the second pose information, the vehicle determines to send an image including the second display effect to the head-mounted device.

[0178] The second display effect is a 3D effect obtained based on the second pose information.

[0179] Furthermore, the content of S804 can be found in the relevant content of S505 mentioned above.

[0180] S805. The vehicle controls the functional components in the cockpit to provide functions that match the image information based on the image information.

[0181] The functional components include one or more of the following: seat, vehicle air conditioner, vehicle air humidifier, vehicle fragrance, and vibration unit on the seat.

[0182] In some embodiments, the vehicle may acquire image information from images in a media resource, wherein the image information includes one or more of seasonal information, weather information, temperature information, location information, scene information, and event information. The vehicle then controls functional components within the cockpit to provide functions matching the image information.

[0183] For example, the functional component includes a vibration unit on the seat, comprising one or more vibration units, each vibration unit being disposed at a different position on the seat. The vehicle determines vibration information corresponding to the vibration unit based on image information, wherein the vibration information includes vibration frequency and / or vibration intensity. Subsequently, the vehicle controls the vibration unit to vibrate based on the vibration information. This allows the user to experience a vibration sensation corresponding to the content presented in the image.

[0184] For another example, the functional components include one or more of the following: a vehicle air conditioner, a vehicle humidifier, and a vehicle fragrance system. Based on the image information, the vehicle determines the setting information for the functional components. This setting information includes one or more of the following: a temperature setting for the vehicle air conditioner, a humidity setting for the vehicle humidifier, and odor information for the vehicle fragrance system. The odor information includes a target fragrance type and / or odor concentration. Then, based on the setting information, the vehicle adjusts one or more of the following: the temperature of the vehicle air conditioner, the humidity of the vehicle humidifier, and the odor of the vehicle fragrance system. Thus, the user experiences seat posture, temperature, humidity, and odor corresponding to the content presented in the image.

[0185] As described above, the first and second pose information represent the information corresponding to the user's different poses within the cockpit. The vehicle can send images corresponding to or matching the user's current pose to the head-mounted device. Furthermore, the vehicle can control the functional components within the cockpit to provide functions matching the image information. Therefore, not only can the head-mounted device present a dynamic 3D display effect that changes with the user's pose, but the vehicle can also provide users with experiences in other dimensions. Users can experience media resources from multiple perspectives within the cockpit, resulting in a better user experience and satisfying diverse experiential needs.

[0186] And, in the above Figure 8 In the cockpit control method shown, the vehicle can also control each speaker to play the corresponding audio based on the audio in the media resource that corresponds to the aforementioned image. Furthermore, the audio corresponding to each speaker will change as the user's posture changes, as detailed in the foregoing embodiments. This allows the user to experience the media resource more realistically from multiple angles or dimensions, creating an immersive experience.

[0187] It is understood that the collection, storage, use, processing, transmission, provision, and disclosure of the aforementioned user information (including user image information, user location, user posture, etc.) all comply with relevant laws and regulations and do not violate public order and good morals. For example, in the embodiments of this application, the processing of user information is carried out with the user's authorization.

[0188] In some solutions, multiple embodiments of this application can be combined, and the combined solution can be implemented. Optionally, some operations in the processes of each method embodiment may be combined, and / or the order of some operations may be changed. Furthermore, the execution order between the steps of each process is merely exemplary and does not constitute a limitation on the execution order between steps; other execution orders are also possible. It is not intended to indicate that the execution order is the only possible order in which these operations can be performed. Those skilled in the art will conceive of various ways to reorder the operations described in the embodiments of this application. In addition, it should be noted that the process details involved in one embodiment of this application are also applicable to other embodiments in a similar manner, or different embodiments may be combined.

[0189] Furthermore, some steps in the method embodiments can be equivalently replaced with other possible steps. Alternatively, some steps in the method embodiments may be optional and can be deleted in certain use cases. Or, other possible steps may be added to the method embodiments.

[0190] Furthermore, the various method embodiments can be implemented individually or in combination.

[0191] It is understood that, in order to achieve the above functions, the aforementioned vehicle includes hardware and / or software modules corresponding to the execution of each function. Based on the algorithmic steps of the various examples described in conjunction with the embodiments disclosed herein, this application can be implemented in hardware or a combination of hardware and computer software. Whether a function is executed in a hardware-driven or software-driven manner depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application in conjunction with the embodiments, but such implementation should not be considered beyond the scope of this application.

[0192] This embodiment can divide the vehicle into functional modules according to the above method example. For example, each function can be divided into its own functional module, or two or more functions can be integrated into one processing module. The integrated module can be implemented in hardware. It should be noted that the module division in this embodiment is illustrative and only represents one logical functional division. In actual implementation, there may be other division methods.

[0193] This application provides an electronic system that may include at least two speakers respectively disposed at different locations within the cockpit, and the electronic system is connected to a head-mounted device. For example... Figure 9 As shown, the electronic system also includes an information acquisition module 901 and a resource processing module 902.

[0194] The information acquisition module 901 is used to acquire media resources and the user's first posture information in the cockpit. For example, it executes the relevant steps of S501 described above.

[0195] The resource processing module 902 is used to determine, based on the first pose information, to send an image including a first display effect to the head-mounted device; and to control each speaker to play audio corresponding to the speaker based on the first pose information. For example, it executes the relevant steps of S502 and S503 described above.

[0196] The information acquisition module 901 is also used to acquire the user's second pose information within the cockpit. For example, it executes the relevant steps in S504 described above.

[0197] The resource processing module 902 is further configured to determine, based on the second pose information, to send an image including the second display effect to the head-mounted device; and to control each speaker to play audio corresponding to the speaker, based on the second pose information. For example, it executes the relevant steps of S505 and S506 described above.

[0198] Optionally, the information acquisition module 901 mentioned above may include the aforementioned Figure 2 The pose detection module and resource acquisition module, etc., are included. Furthermore, the aforementioned resource processing module 902 may include the aforementioned... Figure 2 The rendering engine in [the context].

[0199] This application provides another electronic system, including at least one functional component, which is connected to a head-mounted device. The electronic system may also include an information acquisition module and a resource processing module.

[0200] The information acquisition module is used to acquire media resources and the user's first and second posture information in the cockpit. For example, it executes the relevant steps in S801 described above.

[0201] The resource processing module is used to determine, based on the first pose information, the image including the first display effect to be sent to the head-mounted device. For example, it executes the relevant steps of S802 described above.

[0202] The information acquisition module is also used to acquire the user's second pose information within the cockpit. For example, it executes the relevant steps in S803 described above.

[0203] The resource processing module is also used to determine, based on the second pose information, the image including the second display effect to be sent to the head-mounted device. For example, it performs the relevant steps of S804 described above.

[0204] The resource processing module is also used to control the functional components in the cockpit to provide functions that match the image information, such as performing the relevant steps in S805 described above.

[0205] This application also provides a vehicle, such as... Figure 10 As shown, the vehicle may include one or more processors 1001, memory 1002 and communication interfaces 1003.

[0206] The memory 1002, communication interface 1003, and processor 1001 are coupled together. For example, the memory 1002, communication interface 1003, and processor 1001 can be coupled together via bus 1004.

[0207] The communication interface 1003 is used for data transmission with other devices. The memory 1002 stores computer program code. The computer program code includes computer instructions, which, when executed by the processor 1001, cause the vehicle to perform the cockpit control method described in this embodiment.

[0208] The processor 1001 may be a processor or controller, such as a central processing unit (CPU), a general-purpose processor, a digital signal processor (DSP), an application-specific integrated circuit (ASIC), a field-programmable gate array (FPGA), or other programmable logic devices, transistor logic devices, hardware components, or any combination thereof. It can implement or execute the various exemplary logic blocks, modules, and circuits described in connection with this disclosure. The processor may also be a combination that implements computational functions, such as a combination of one or more microprocessors, a combination of a DSP and a microprocessor, etc.

[0209] Bus 1004 can be a Peripheral Component Interconnect (PCI) bus or an Extended Industry Standard Architecture (EISA) bus, etc. Bus 1004 can be divided into address bus, data bus, control bus, etc. For ease of representation, Figure 10The bus is represented by a single thick line, but this does not mean that there is only one bus or one type of bus.

[0210] This application also provides a computer-readable storage medium that includes computer instructions. When the computer instructions are executed on a vehicle, the vehicle performs the relevant method steps described in the above method embodiments.

[0211] This application also provides a computer program product that, when run on a computer, causes the computer to execute the relevant method steps described in the above method embodiments.

[0212] The electronic systems, vehicles, computer storage media, or computer program products provided in this application are all used to execute the corresponding methods provided above. Therefore, the beneficial effects they can achieve can be referred to the beneficial effects in the corresponding methods provided above, and will not be repeated here.

[0213] Through the above description of the embodiments, those skilled in the art can clearly understand that, for the sake of convenience and brevity, only the division of the above functional modules is used as an example. In actual applications, the above functions can be assigned to different functional modules as needed, that is, the internal structure of the device can be divided into different functional modules to complete all or part of the functions described above.

[0214] In the several embodiments provided in this application, it should be understood that the disclosed apparatus and methods can be implemented in other ways. For example, the apparatus embodiments described above are merely illustrative; for instance, the division of modules or units is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another device, or some features may be ignored or not executed. Furthermore, the mutual coupling or direct coupling or communication connection shown or discussed may be through some interfaces; the indirect coupling or communication connection between devices or units may be electrical, mechanical, or other forms.

[0215] The units described as separate components may or may not be physically separate. A component shown as a unit can be one or more physical units; that is, it can be located in one place or distributed in multiple different locations. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs.

[0216] Furthermore, the functional units in the various embodiments of this application can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated unit can be implemented in hardware or as a software functional unit.

[0217] If the integrated unit is implemented as a software functional unit and sold or used as an independent product, it can be stored in a readable storage medium. Based on this understanding, the technical solution of the embodiments of this application, in essence, or the part that contributes, or all or part of the technical solution, can be embodied in the form of a software product. This software product is stored in a storage medium and includes several instructions to cause a device (which may be a microcontroller, chip, etc.) or processor to execute all or part of the steps of the methods described in the various embodiments of this application. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0218] The above description is merely a specific embodiment of this application, but the scope of protection of this application is not limited thereto. Any variations or substitutions within the technical scope disclosed in this application should be included within the scope of protection of this application. Therefore, the scope of protection of this application should be determined by the scope of the claims.

Claims

1. A cockpit control method, characterized in that, Applied to a vehicle, the vehicle including at least two speakers respectively disposed at different locations within the cockpit, the vehicle being connected to a head-mounted device; the method includes: Acquire media resources and the user's first posture information within the cockpit; the media resources include images and corresponding audio. Based on the first pose information, an image including a first display effect is sent to the head-mounted device; the first display effect is a three-dimensional effect obtained based on the first pose information. Based on the first pose information, each speaker is controlled to play audio corresponding to the speaker; and the audio corresponding to each speaker is obtained based on the audio in the media resource, the attribute information of the speaker, and the first pose information. Obtain the user's second pose information within the cockpit; Based on the second pose information, an image including a second display effect is sent to the head-mounted device; the second display effect is a three-dimensional effect obtained based on the second pose information. Based on the second pose information, each speaker is controlled to play audio corresponding to the speaker; and the audio corresponding to each speaker is obtained based on the audio in the media resource, the attribute information of the speaker, and the second pose information. The method further includes: Obtain image information from the images in the media resources; the image information includes one or more of the following: seasonal information, weather information, temperature information, location information, scene information, or event information; The system controls functional components within the cockpit to provide functions that match the image information; the functional components include one or more of the following: seat, vehicle air conditioner, vehicle air humidifier, vehicle fragrance, or vibration unit on the seat.

2. The method according to claim 1, characterized in that, The acquisition of the user's first posture information within the cockpit includes: Acquire first information sent by the head-mounted device; the first information is acquired by the head-mounted device through at least one sensor, and the first information includes the user's head posture information. The second information is acquired by at least one sensor in the cockpit; the second information includes one or more of the following: user image information, infrared information, seat position information of the user's seat, seat back angle information, seat angle information, or seat pressure information. The first pose information is determined based on the first information and the second information.

3. The method according to claim 1 or 2, characterized in that, The second display effect is the effect after the first display effect has been changed, and the change in display effect includes changes in one or more of the following: image viewpoint, image content, or image size.

4. The method according to claim 1, characterized in that, The audio corresponding to each speaker changes according to the user's pose information, and the change in audio includes changes in one or more of the following: audio frequency, audio amplitude, audio delay, audio phase, or audio spectral characteristics.

5. The method according to claim 1, characterized in that, The functional component includes a vibration unit on the seat, the vibration unit comprising one or more, and each vibration unit being disposed at a different position on the seat; The control system within the cockpit provides functions that match the image information, including: Based on the image information, the vibration information corresponding to the vibration unit is determined; the vibration information includes vibration frequency and / or vibration intensity. The vibration unit is controlled to vibrate based on the vibration information.

6. The method according to claim 1, characterized in that, The functional components include one or more of the following: vehicle air conditioning, vehicle air humidifier, or vehicle fragrance. The control system within the cockpit provides functions that match the image information, including: Based on the image information, the setting information of the functional component is determined; the setting information includes one or more of the following: the temperature setting value corresponding to the vehicle air conditioner, the humidity setting value corresponding to the vehicle air humidifier, or the odor information corresponding to the vehicle fragrance; the odor information includes the target fragrance type and / or odor concentration. Based on the settings, adjust one or more of the following: the temperature of the vehicle air conditioner, the humidity of the vehicle air humidifier, or the scent of the vehicle fragrance.

7. The method according to claim 1, characterized in that, Sending an image including a first display effect to the head-mounted device based on the first pose information includes: Based on the first pose information, a first display parameter of the image is determined; the first display parameter is used to characterize the first display effect of the image. The image is rendered according to the first display parameters; The rendered image is sent to the head-mounted device.

8. The method according to claim 1, characterized in that, The step of controlling each speaker to play audio corresponding to the speaker based on the first pose information includes: For each of the loudspeakers, obtain the loudspeaker's attribute information; the attribute information includes one or more of the loudspeaker's position in the cockpit, sound direction, or frequency response; Based on the audio in the media resource, the first pose information, and the speaker attribute information, determine the first playback parameters of the audio corresponding to the speaker; The audio corresponding to the speaker is rendered according to the first playback parameters; The rendered audio is sent to the speaker so that the speaker plays the rendered audio.

9. An electronic system, characterized in that, The system includes at least two speakers located at different positions within the cockpit, and the electronic system is connected to the head-mounted device; the electronic system also includes an information acquisition module and a resource processing module. The information acquisition module is used to acquire media resources and the user's first posture information in the cockpit; the media resources include images and audio corresponding to the images; The resource processing module is configured to send an image including a first display effect to the head-mounted device based on the first pose information; the first display effect is a three-dimensional effect obtained based on the first pose information; and control each speaker to play audio corresponding to the speaker based on the first pose information. Furthermore, the audio corresponding to each speaker is obtained based on the audio in the media resource, the attribute information of the speaker, and the first pose information; The information acquisition module is also used to acquire the user's second pose information in the cockpit; The resource processing module is further configured to send an image including a second display effect to the head-mounted device according to the second pose information; the second display effect is a three-dimensional effect obtained according to the second pose information; and control each speaker to play audio corresponding to the speaker according to the second pose information. Furthermore, the audio corresponding to each speaker is obtained based on the audio in the media resource, the attribute information of the speaker, and the second pose information; The resource processing module is also used to acquire image information of images in the media resources; the image information includes one or more of seasonal information, weather information, temperature information, location information, scene information, or event information; and to control the functional components in the cockpit to provide functions that match the image information; the functional components include one or more of seats, vehicle air conditioning, vehicle air humidifier, vehicle fragrance, or vibration units on the seats.

10. The system according to claim 9, characterized in that, The information acquisition module is specifically used to acquire first information sent by the head-mounted device; the first information is acquired by the head-mounted device through at least one sensor, and the first information includes the user's head posture information; acquire second information through at least one sensor in the cockpit; the second information includes one or more of the following: the user's image information, infrared information, the seat position information of the user's seat, the seat back angle information, the seat angle information, or the seat pressure information; determine the first posture information based on the first information and the second information.

11. The system according to claim 9 or 10, characterized in that, The second display effect is the effect after the first display effect has been changed, and the change in display effect includes changes in one or more of the following: image viewpoint, image content, or image size.

12. The system according to claim 9, characterized in that, The audio corresponding to each speaker changes according to the user's pose information, and the change in audio includes changes in one or more of the following: audio frequency, audio amplitude, audio delay, audio phase, or audio spectral characteristics.

13. The system according to claim 9, characterized in that, The functional components include a vibration unit on the seat, wherein the vibration unit comprises one or more units, and each vibration unit is disposed at a different position on the seat; the resource processing module is specifically used to determine the vibration information corresponding to the vibration unit based on the image information; the vibration information includes vibration frequency and / or vibration intensity; and to control the vibration unit to vibrate based on the vibration information.

14. The system according to claim 9, characterized in that, The functional components include one or more of a vehicle air conditioner, a vehicle air humidifier, or a vehicle fragrance; the resource processing module is specifically used to determine the setting information of the functional components based on the image information; the setting information includes one or more of the following: the temperature setting value corresponding to the vehicle air conditioner, the humidity setting value corresponding to the vehicle air humidifier, and the odor information corresponding to the vehicle fragrance; the odor information includes a target fragrance type and / or odor concentration; and based on the setting information, adjust one or more of the following: the temperature of the vehicle air conditioner, the humidity of the vehicle air humidifier, or the odor of the vehicle fragrance.

15. The system according to claim 9, characterized in that, The resource processing module is specifically used to determine the first display parameters of the image based on the first pose information; the first display parameters are used to characterize the first display effect of the image; and to perform rendering processing on the image based on the first display parameters. The rendered image is sent to the head-mounted device.

16. The system according to claim 9, characterized in that, The resource processing module is specifically used to acquire the attribute information of each speaker; the attribute information includes one or more of the speaker's position in the cockpit, sound direction, or frequency response. Based on the audio in the media resource, the first pose information, and the speaker attribute information, determine the first playback parameters of the audio corresponding to the speaker; The audio corresponding to the speaker is rendered according to the first playback parameters; The rendered audio is sent to the speaker so that the speaker plays the rendered audio.

17. A vehicle, characterized in that, The system includes a memory and one or more processors; the memory is coupled to the processors; wherein the memory stores computer program code, the computer program code including computer instructions, which, when executed by the processor, cause the vehicle to perform the cockpit control method as described in any one of claims 1-8.

18. A computer-readable storage medium, characterized in that, Includes computer instructions that, when executed on a vehicle, cause the vehicle to perform the cockpit control method as described in any one of claims 1-8.

19. A computer program product, characterized in that, When the computer program product is run on a vehicle, it causes the vehicle to perform the cockpit control method as described in any one of claims 1-8.

Images