Method and apparatus for adjusting display device

Abstract

The present application provides a method and apparatus for adjusting a display device, the method comprising: obtaining a first facial image of a first user; determining first spatial coordinates of facial feature points of the first user based on the first facial image; and adjusting the orientation of a first side of an image displayed by the display device based on the first spatial coordinates, wherein the first side of the image contains information to be conveyed to the first user. The method for adjusting a display device of the present application can accurately adjust the orientation of an image displayed by the display device based on the user's position, thereby allowing the user to experience interaction with the display device, thereby helping to improve the user's user experience at the interactive level.

Description

Technical Field

[0001] The present application relates to the field of human-computer interaction, and more specifically, to a method and apparatus for adjusting a display device. Background Art

[0002] With the development of smart cars, more and more imaging display devices are being integrated into the cockpit, such as large screens, heads-up displays (HUDs), and holographic displays. However, existing display devices have a limited display range. When the user is in different positions, these displays may not be at the optimal display angle or position for the user, and may even result in incomplete imaging or even no display.

[0003] Therefore, a method and apparatus capable of accurately adjusting a display device according to a user's position is in urgent need of development. Summary of the Invention

[0004] The present application provides a method and apparatus for adjusting a display device, which can accurately adjust the orientation of an image displayed by the display device according to the user's position, thereby helping to improve the user experience at the human-computer interaction level.

[0005] In a first aspect, a method for adjusting a display device is provided. This method can be performed by a vehicle; alternatively, it can be performed by an onboard terminal of the vehicle, such as a vehicle computer; alternatively, it can be performed by a chip or circuit used in the vehicle, although this application does not limit this. For ease of description, the following description uses the vehicle as an example.

[0006] The method may include: obtaining a first facial image of a first user; determining first spatial coordinates of facial feature points of the first user based on the first facial image; and adjusting the orientation of a first side of an image displayed by a display device based on the first spatial coordinates, wherein the first side of the image contains information to be conveyed to the first user.

[0007] In the above technical solution, the orientation of the image displayed by the display device is accurately adjusted according to the user's position, which can improve the user experience at the interaction level.

[0008] In some possible implementations, the first spatial coordinates are the three-dimensional coordinates of the user's facial feature points in the vehicle coordinate system.

[0009] For example, the facial feature points may be located at any one or more of the first user's brow center, eyes, eyebrows, and nose; or, the facial feature points may also be points at other locations on the first user's face, which is not specifically limited in this embodiment of the present application.

[0010] In some possible implementations, the aforementioned image may be a three-dimensional image, such as a holographic projection. It should be understood that different angles of a holographic projection contain different information. Alternatively, the aforementioned image may be a two-dimensional image with three-dimensional features, such as a digital human, that interacts with the first user. It should be understood that different orientations of the digital human's face can achieve different interaction effects. For example, when the digital human's face is facing the first user, the first user can experience the effect of face-to-face communication with the digital human.

[0011] In one example, if the image is a 3D image of a virtual assistant, the first side of the image may be the side that includes the front face of the virtual assistant. In another example, if the image is a 3D image of a vehicle and the first user wants to show the vehicle's front cowl, the first side of the image may be the side that includes the vehicle's front cowl.

[0012] Exemplarily, adjusting the orientation of the first side of the image may include adjusting the posture of the image. For example, when the image is a three-dimensional image of a virtual assistant, the head posture of the image may be adjusted so that its face faces the first user.

[0013] In combination with the first aspect, in certain implementations of the first aspect, the method further includes: obtaining a second facial image of the second user after adjusting the orientation of the first side of the image displayed by the display device according to the first spatial coordinates; determining the second spatial coordinates of the facial feature points of the second user based on the second facial image; and adjusting the orientation of the first side of the image displayed by the display device according to the second spatial coordinates.

[0014] In some possible implementations, the determination of the second spatial coordinates can be actively triggered by the second user. For example, the second user may trigger the determination through a voice command, such as issuing the voice command "Hologram, please face me." Alternatively, the second user may trigger the determination through a button, such as clicking the "Open Hologram" button. Furthermore, the vehicle controls the camera to capture a second facial image of the second user, thereby determining the second spatial coordinates of the second user's facial feature points.

[0015] In some possible implementations, a second facial image of the second user is acquired after a preset time period has elapsed since the orientation of the first side of the image displayed on the display device was adjusted according to the first spatial coordinates. For example, the preset time period may be 3 seconds, 5 seconds, or another preset time period, which is not specifically limited in this embodiment of the present application.

[0016] In the above technical solution, when there are two or more users, the orientation of the image displayed by the display device can be adjusted according to the three-dimensional coordinates of the facial feature points of each user among the multiple users, thereby improving the interactive experience in the multi-user scenario.

[0017] In combination with the first aspect, in certain implementations of the first aspect, the method further includes: obtaining a third facial image of the first user after adjusting the orientation of the first side of the image displayed by the display device according to the first spatial coordinates; determining the third spatial coordinates of the facial feature points of the first user based on the third facial image; and adjusting the orientation of the first side of the image displayed by the display device according to the third spatial coordinates when the distance between the first spatial coordinate and the third spatial coordinate is greater than or equal to a preset threshold.

[0018] In some possible implementations, when the first user interacts with the image, the position of the first user's head changes. At this time, a second facial image of the first user can be obtained, and the second coordinate position of the facial feature point can be determined based on the second facial image.

[0019] Furthermore, when the distance the first user's head moves exceeds a preset threshold, the orientation of the first side of the image is adjusted according to the second spatial coordinates.

[0020] For example, the preset threshold may be 15 centimeters, or 20 centimeters, or other distances, which is not specifically limited in the embodiments of the present application.

[0021] In the above technical solution, when the distance the user's head moves exceeds a preset distance, the direction of the image is adjusted according to the position after the movement, which can avoid frequent adjustment of the image direction and help improve the user's driving experience.

[0022] In combination with the first aspect, in certain implementations of the first aspect, the method further includes: before obtaining the first facial image of the first user, obtaining audio information, the audio information including the first user's voice instructions, the voice instructions being used to instruct to turn on and / or adjust the display device; determining the sound source position of the audio information; and determining the position of the first user based on the sound source position.

[0023] In the above technical solution, in a multi-user scenario, when a user issues a voice command, the position of the sound source can be used to determine whether the user issuing the command is the driver, co-driver or rear passenger, and then the corresponding display device can be adjusted so that the side of the image displayed by the display device that contains the information to be conveyed to the user is facing the user, thereby improving the interactivity between the user and the image displayed by the display device.

[0024] In combination with the first aspect, in certain implementations of the first aspect, the audio information also includes voice instructions of the second user, and the method also includes: controlling the display device to display the first image and the second image; adjusting the display device so that the first side of the first image is facing the first user, and the first side of the second image is facing the second user, wherein the image displayed by the display device includes the first image and / or the second image, the first side of the first image contains information to be conveyed to the first user, and the first side of the second image contains information to be conveyed to the second user.

[0025] In the above technical solution, in a multi-user scenario, the display device can also copy images, so that each image faces one of the multiple users, further enhancing the user's interactive experience and driving fun.

[0026] In combination with the first aspect, in some implementations of the first aspect, the method further includes: adjusting the position of the image displayed by the display device according to the first spatial coordinates.

[0027] In some possible implementations, the position of the image along the y-axis in the vehicle coordinate system can be adjusted based on the first spatial coordinate. For example, after determining the user's position, the display device can be adjusted to move the image projected by the display device to a position corresponding to the user's position. For example, if the user is in the passenger seat, the display device can be adjusted to move the image to the passenger seat.

[0028] In some possible implementations, the position of the image in the z-axis direction in the vehicle coordinate system can be adjusted according to the first spatial coordinate. For example, after determining the user's position, the display device can be adjusted to move the height of the image projected by the display device to a height suitable for the user to view.

[0029] It should be understood that the y-axis direction in the vehicle coordinate system is perpendicular to the longitudinal symmetry plane of the vehicle; and the z-axis direction in the vehicle coordinate system is parallel to the longitudinal symmetry plane of the vehicle.

[0030] In the above technical solution, the display device is adjusted to control the image to move to the user's position, or the image is controlled to move to an appropriate height, so that the user does not have to turn his head to interact with the image displayed on the display device, which helps to improve the user's driving experience.

[0031] In combination with the first aspect, in certain implementations of the first aspect, the method further includes: before adjusting the orientation of the first side of the image displayed by the display device according to the first spatial coordinates, obtaining initial posture information of the display device, and the initial posture information is used to indicate the posture angle and position of the display device; adjusting the orientation of the first side of the image displayed by the display device according to the first spatial coordinates includes: determining the rotation angle and rotation direction of the display device according to the first spatial coordinates and the initial posture information; and adjusting the orientation of the first side of the image displayed by the display device according to the rotation angle and the rotation direction.

[0032] In some possible implementations, before adjusting the orientation of the first side of the image displayed by the display device according to the first spatial coordinates, the initial posture information of the image is obtained, and the initial posture information of the image is used to indicate the posture angle and position of the image; the rotation angle and rotation direction of the image are determined based on the first spatial coordinates and the initial posture information of the image. Furthermore, based on the mapping relationship between the angle transformation of the display device and the angle transformation of the image, the rotation angle and rotation direction of the display device are determined, and then the display device is adjusted. Exemplarily, if the above-mentioned image is a virtual assistant, the posture angle of the image can be the head posture angle of the virtual assistant.

[0033] In combination with the first aspect, in certain implementations of the first aspect, the first spatial coordinates of the facial feature points of the first user are used to indicate the three-dimensional position of the facial feature points of the first user in a vehicle, and the vehicle includes the display device.

[0034] In the above technical solution, the display device can be adjusted according to the three-dimensional coordinates of the user's facial feature points in the vehicle coordinate system, so that the orientation of the image displayed by the display device can be accurately adjusted, providing the user with a better interactive effect and prompting the user's interactive experience.

[0035] In combination with the first aspect, in some implementations of the first aspect, the display device includes a holographic projection device, and the image includes a three-dimensional 3D image.

[0036] In a second aspect, a device for adjusting a display device is provided, the device comprising: an acquisition unit for acquiring a first facial image of a first user; a processing unit for determining first spatial coordinates of facial feature points of the first user based on the first facial image; and adjusting the orientation of a first side of an image displayed by the display device based on the first spatial coordinates.

[0037] In combination with the second aspect, in certain implementations of the second aspect, the acquisition unit is further used to: acquire a second facial image of the second user after adjusting the orientation of the first side of the image displayed by the display device according to the first spatial coordinates; the processing unit is further used to: determine the second spatial coordinates of the facial feature points of the second user based on the second facial image; and adjust the orientation of the first side of the image displayed by the display device according to the second spatial coordinates.

[0038] In combination with the second aspect, in certain implementations of the second aspect, the acquisition unit is further used to: acquire a third facial image of the first user after adjusting the orientation of the first side of the image displayed by the display device according to the first spatial coordinates; the processing unit is further used to: determine the third spatial coordinates of the facial feature points of the first user based on the third facial image; and adjust the orientation of the first side of the image displayed by the display device according to the third spatial coordinates when the distance between the first spatial coordinate and the third spatial coordinate is greater than or equal to a preset threshold.

[0039] In combination with the second aspect, in certain implementations of the second aspect, the acquisition unit is further used to: before acquiring the first facial image of the first user, acquire audio information, the audio information including the voice instructions of the first user, the voice instructions being used to instruct to turn on and / or adjust the display device; the processing unit is further used to: determine the sound source position of the audio information; and determine the position of the first user based on the sound source position.

[0040] In combination with the second aspect, in certain implementations of the second aspect, the audio information also includes voice instructions of the second user, and the processing unit is further used to: control the display device to display the first image and the second image; adjust the display device so that the first side of the first image is facing the first user, and the first side of the second image is facing the second user, wherein the image displayed by the display device includes the first image and / or the second image, the first side of the first image contains information to be conveyed to the first user, and the first side of the second image contains information to be conveyed to the second user.

[0041] In combination with the second aspect, in some implementations of the second aspect, the processing unit is further used to: adjust the position of the image displayed by the display device according to the first spatial coordinates.

[0042] In combination with the second aspect, in certain implementations of the second aspect, the acquisition unit is further used to: obtain initial posture information of the display device before adjusting the orientation of the first side of the image displayed by the display device according to the first spatial coordinates, and the initial posture information is used to indicate the posture angle and position of the display device; the processing unit is also used to: determine the rotation angle and rotation direction of the display device based on the first spatial coordinates and the initial posture information; and adjust the orientation of the first side of the image displayed by the display device according to the rotation angle and the rotation direction.

[0043] In combination with the second aspect, in certain implementations of the second aspect, the first spatial coordinates of the facial feature points of the first user are used to indicate the three-dimensional position of the facial feature points of the first user in a vehicle, and the vehicle includes the display device.

[0044] In combination with the second aspect, in some implementations of the second aspect, the display device includes a holographic projection device, and the image includes a three-dimensional 3D image.

[0045] In a third aspect, a device for adjusting a display device is provided, the device comprising: a memory for storing a program; a processor for executing the program stored in the memory, and when the program stored in the memory is executed, the processor is used to execute the method in any possible implementation of the first aspect above.

[0046] In a fourth aspect, a vehicle is provided, comprising the apparatus of any possible implementation of the second aspect or the apparatus of any implementation of the third aspect, and the display device. The display device may be a holographic projection device, displaying a three-dimensional image; or an onboard display screen, displaying a digital human image; or other display devices, which are not specifically limited in this application.

[0047] In a fifth aspect, a computer program product is provided, comprising: a computer program code, which, when executed on a computer, enables the computer to execute the method in any one of the possible implementations of the first aspect.

[0048] It should be noted that the above-mentioned computer program code can be stored in whole or in part on the first storage medium, wherein the first storage medium can be packaged together with the processor or separately packaged with the processor, and the embodiments of the present application do not specifically limit this.

[0049] In a sixth aspect, a computer-readable medium is provided, wherein the computer-readable medium stores a program code, and when the computer program code runs on a computer, the computer executes the method in any possible implementation of the first aspect.

[0050] In a seventh aspect, a chip is provided, comprising a processor for calling a computer program or computer instruction stored in a memory so that the processor executes the method in any possible implementation of the first aspect.

[0051] In combination with the seventh aspect, in a possible implementation, the processor is coupled to the memory through an interface.

[0052] In combination with the seventh aspect, in a possible implementation, the chip system also includes a memory, in which a computer program or computer instructions are stored.

[0053] The present invention provides a method for adjusting a display device that can accurately adjust the orientation of an image displayed on the display device based on the user's position, thereby improving the user's experience at the interactive level. The display device can be adjusted based on the position of the user's head movement. When the distance the user's head moves exceeds a preset distance, the orientation of the image is adjusted based on the position after the head moves. This can avoid frequent adjustments to the image orientation and help improve the user's driving experience. When there are two or more users, the orientation of the image displayed on the display device can be adjusted based on the three-dimensional coordinates of each user's facial feature points, improving the interactive experience in multi-user scenarios. The user's position can also be determined based on a voice command issued by the user, and the display device can be adjusted accordingly. In a multi-user scenario, when a user issues a voice command, the sound source location can be used to determine whether the user issuing the command is the driver, front passenger, or rear passenger, and the corresponding display device can be adjusted so that the side of the image displayed on the display device containing the information intended to be conveyed to the user faces the user, improving the interactivity between the user and the image displayed on the display device. In multi-user scenarios, the display device can replicate images, allowing each image to face one of the multiple users, further enhancing the user interaction experience and driving pleasure. The display device can also be adjusted to control the image to move to the user's location or to an appropriate height, eliminating the need for users to turn their heads to interact with the image displayed on the display device, further improving the user's driving experience. BRIEF DESCRIPTION OF THE DRAWINGS

[0054] Figure 1 This is a schematic diagram of an application scenario of a method for adjusting a display device provided in an embodiment of the present application.

[0055] Figure 2 It is a schematic block diagram of a vehicle 100 provided in an embodiment of the present application.

[0056] Figure 3 This is a schematic block diagram of a system for adjusting a display device provided in an embodiment of the present application.

[0057] Figure 4 This is a schematic flowchart of a method for adjusting a display device provided in an embodiment of the present application.

[0058] Figure 5 It is a schematic diagram of the transformation relationship between the image coordinate system and the camera coordinate system provided in an embodiment of the present application.

[0059] Figure 6 This is a schematic diagram of an application scenario for adjusting an image displayed by a display device provided in an embodiment of the present application.

[0060] Figure 7 This is another schematic flowchart of a method for adjusting a display device provided in an embodiment of the present application.

[0061] Figure 8 This is another schematic diagram of an application scenario for adjusting the image displayed by a display device provided in an embodiment of the present application.

[0062] Figure 9 This is another schematic flowchart of a method for adjusting a display device provided in an embodiment of the present application.

[0063] Figure 10 This is a schematic block diagram of an apparatus for adjusting a display device provided in an embodiment of the present application.

[0064] Figure 11 This is a schematic block diagram of an apparatus for adjusting a display device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0065] The technical solutions in the embodiments of the present application will be described below with reference to the accompanying drawings.

[0066] Figure 1 A schematic diagram of a vehicle cockpit scenario provided in an embodiment of the present application. One or more cameras may be installed inside the smart cockpit to capture images inside or outside the cabin, such as a driver monitoring system (DMS) camera, a cabin monitor system (CMS) camera, and a dashcam camera. Figure 1 In (a), the camera installed on the A-pillar is used as an example. The camera used to capture the inside and outside of the cabin can be the same camera or different cameras. In addition, there is also a car display screen in the cabin. Figure 1 In (a) of the figure, the display screen set in the central control area is used as an example. The vehicle can use the vehicle display screen, HUD ( Figure 1 (a) in the figure) and the vehicle-mounted holographic projection equipment ( Figure 1For example, a 3D image of a virtual assistant can be displayed to the user through an in-vehicle holographic projection device, specifically as follows: Figure 1 In some possible implementations, the vehicle-mounted holographic projection device can also perform imaging on the back of the front seat headrest, as shown in (a). Figure 1 As shown in (b) in FIG. It should be understood that Figure 1 The three-dimensional image shown in (a) is only an example. The specific content of the three-dimensional image can also be other three-dimensional objects, for example, it can also be a three-dimensional image of the vehicle. It should be understood that the position of the camera that collects image information in the cabin in the embodiment of the application is not specifically limited. The camera can be located Figure 1 The support structure may be located on the A-pillar shown in (a) above, on the B-pillar, below the steering wheel, or near a rearview mirror.

[0067] Figure 2 1 is a functional block diagram of the vehicle 100 provided in an embodiment of the present application. The vehicle 100 may include a perception system 120, a display device 130 and a computing platform 150. The perception system 120 may include several sensors for sensing information about the environment surrounding the vehicle 100 and the internal environment of the vehicle cabin. For example, the perception system 120 may include a positioning system, which may be a global positioning system (GPS), a BeiDou system or other positioning systems, an inertial measurement unit (IMU), a lidar, a millimeter-wave radar, an ultrasonic radar, a visual sensor, a sound sensor and a camera device. The above-mentioned camera device may include a red, green and blue / infrared (RGB / IR) camera, or a depth camera, such as a time of flight (TOF) camera, a binocular camera, a structured light camera, etc.

[0068] Some or all functions of vehicle 100 may be controlled by computing platform 150. Computing platform 150 may include processors 151 to 15n (n is a positive integer). A processor is a circuit capable of processing signals. In one implementation, the processor may be a circuit capable of reading and executing instructions, such as a central processing unit (CPU), a microprocessor, a graphics processing unit (GPU) (which can be understood as a type of microprocessor), or a digital signal processor (DSP). In another implementation, the processor may implement certain functions through the logical relationships of hardware circuits. The logical relationships of the hardware circuits may be fixed or reconfigurable. For example, the processor may be a hardware circuit implemented as an application-specific integrated circuit (ASIC) or a programmable logic device (PLD), such as a field programmable gate array (FPGA). In a reconfigurable hardware circuit, the process of the processor loading a configuration file to implement the hardware circuit configuration can be understood as the process of the processor loading instructions to implement the functions of some or all of the above units. In addition, it can also be a hardware circuit designed for artificial intelligence, which can be understood as an ASIC, such as a neural network processing unit (NPU), a tensor processing unit (TPU), a deep learning processing unit (DPU), etc. In addition, the computing platform 150 can also include a memory for storing instructions. Some or all of the processors 151 to 15n can call the instructions in the memory and execute the instructions to achieve corresponding functions.

[0069] The display device 130 in the cockpit is mainly divided into three categories. The first category is the vehicle-mounted display screen; the second category is the projection display screen, such as the HUD; and the third category is the holographic projection device. The vehicle-mounted display screen is a physical display screen and is an important part of the vehicle infotainment system. There can be multiple display screens in the cockpit, such as the digital instrument display screen, the central control screen, the display screen in front of the passenger on the front passenger seat (also called the front passenger), the display screen in front of the left rear passenger and the display screen in front of the right rear passenger. Even the car window can be used as a display screen. Head-up display, also known as head-up display system, includes a combined head-up display (C-HUD) system, a windshield head-up display (W-HUD) system, and an augmented reality head-up display system (AR-HUD). A panoramic imaging system is a display device that can display three-dimensional (3D) images of objects. In some possible implementations, the holographic projection device can record and reproduce the 3D image of the object based on the interference and diffraction principles of light, or the 3D effect can be achieved through edge blanking, Pepper's ghost and other methods. In the embodiment of the present application, the image of the 3D virtual assistant or the 3D image of the vehicle can be displayed by the holographic projection device, or other 3D images can be displayed, and the embodiment of the present application does not specifically limit this. In some possible implementations, the display device 130 can also display a digital human. It should be understood that a digital human can refer to a virtual image with human features such as appearance and personality, with the ability to express language, facial expressions and body movements, and can communicate and interact with humans. The image can be displayed through an in-vehicle display screen, etc.

[0070] It should be understood that the image displayed by the HUD is a two-dimensional image projected on the HUD screen; the digital human displayed on the vehicle display is also included in the two-dimensional image with stereoscopic effect and depth of field; and the image displayed by the holographic projection device is a three-dimensional image, that is, when observing the holographic image from different angles, you can see different sides of the object, with parallax effect and depth of field.

[0071] In embodiments of the present application, the processor may obtain user image information collected by the perception system 120 and determine the user's eye position and the user's gaze focus point in conjunction with the three-dimensional coordinate system of the vehicle 100. Upon determining that the user's gaze focus point is on the display device and / or the image displayed on the display device, the processor may adjust the display device based on the user's actual position within the vehicle 100. In some possible implementations, the processor may also determine the user's location based on sound information collected by the sound sensor in the perception system 120, and further adjust the angle of the image projected by the holographic projection device based on the user's location. In some possible implementations, the user image information and sound information may also be stored as data in a memory within the computing platform 150. In some possible implementations, the processor may process the user image information and sound information to obtain parameterized indicators, and further adjust the display device based on the parameterized indicators. It should be understood that the above operations may be performed by the same processor, or by one or more processors, and this embodiment of the present application is not specifically limited in this regard.

[0072] The following combination Figure 3 Describe in detail the workflow of the system for adjusting the display device. Figure 3 The system architecture diagram of a display device adjustment system provided by an embodiment of the present application is shown as follows. The system 200 includes a sensing module, a calibration module, a display device adjustment module, and a display module. The sensing module may include Figure 2 One or more camera devices and one or more sensors in the perception system 120 shown in FIG; the display module may include Figure 2 One or more display devices in the display device 130 shown in FIG; the calibration module and the display device adjustment module may include Figure 2 One or more processors in the computing platform 150 shown in . Specifically, the perception module can detect the user's facial information and / or voice information, and the display device adjustment module can determine the adjustment parameters based on the facial image information obtained from the perception module and the external parameters in the whole vehicle coordinate system obtained from the calibration module, and then determine the actual three-dimensional coordinates of any position of the user's face in the vehicle in combination with the external parameters in the whole vehicle coordinate system and the facial image information. Furthermore, the display device adjustment module can adjust the display device in the display module according to the three-dimensional coordinates of one or several feature points on the user's face. Exemplarily, the above-mentioned feature point on the user's face can be an eye, or it can be a point between the two eyes, or it can be other points, and the embodiments of the present application do not specifically limit this.

[0073] It should be understood that the above modules and devices are only examples. In actual applications, the above modules and devices may be added or deleted according to actual needs. Figure 3The central display module and the display device adjustment module can also be combined into one module, that is, the functions of the two are implemented by one module.

[0074] As mentioned above, the display range of display devices under the current technical background is limited. When the user is in different positions, these display devices are not necessarily in the best display angle or position for the user, and even incomplete imaging and invisible display may occur. In particular, for holographic projection equipment, the direction of the three-dimensional image it projects is fixed, and users in different positions see different images, which may cause users who want to obtain the information conveyed by the image to be unable to obtain relevant information in a timely and comprehensive manner. In view of this, the embodiments of the present application provide a method and device for adjusting a display device, which can adjust the holographic projection device according to the user's position so that the three-dimensional image it projects can be accurately adjusted according to the user's position, thereby enhancing the interactivity with the user and improving the user's driving experience.

[0075] Figure 4 A method for adjusting a display device provided in an embodiment of the present application is shown. The method 400 can be applied to Figure 1 The scenario shown can also be applied to Figure 2 In the vehicle 100 shown, the method may also be performed by Figure 3 The system shown in FIG. 400 is executed. The method 400 includes:

[0076] S401: Acquire a first facial image of a user.

[0077] For example, after the display device is turned on, a camera located inside the vehicle can be used to capture a first facial image of the user, where the first facial image includes an image of the user's eyes. For example, to facilitate capturing the driver's facial image, the camera can be located on the A-pillar so that the camera can capture both eyes of the user. It should be understood that the user can be any user in the vehicle cabin, for example, the driver, the passenger seat, or a user in the back seat.

[0078] S402: Determine whether the user's sight focus is on the displayed image.

[0079] Specifically, when it is determined that the user's visual focus is the displayed image, execute S403; otherwise, execute S405.

[0080] In some possible implementations, the display device is a holographic projection device, and the image projected by the display device is a virtual 3D image. It is then determined whether the user's visual focus is on the virtual 3D image.

[0081] For example, the user's gaze focus point may be determined by gaze estimation technology or gaze tracking technology.

[0082] S403: Determine first position information of facial feature points according to the first facial image.

[0083] Exemplarily, the first position information is used to indicate the first position of the facial feature point in the vehicle coordinate system.

[0084] In some possible implementations, one facial feature point may be determined based on the first facial image, or two or more facial feature points may be determined based on the first facial image. This embodiment of the present application does not specifically limit this.

[0085] For example, the facial feature points may be located at any one or more of the center of the eyebrows, eyes, eyebrows, and nose; or, the facial feature points may also be points at other locations on the user's face, which is not specifically limited in the embodiments of the present application.

[0086] Furthermore, the three-dimensional coordinates of the facial feature point inside the vehicle cabin, ie, the first position information, are determined based on the image coordinates of the facial feature point on the first facial image.

[0087] In a specific implementation, a facial feature region may be determined based on the first facial image, and the first position information may be determined based on the facial feature region. Alternatively, the first position information may be determined based on other feature points or regions determined from the first facial image, which is not specifically limited in this application.

[0088] The detailed method of determining the first position information according to the image coordinates can be referred to the following description:

[0089] The image coordinate system is a coordinate system established on the image captured by the camera. The image coordinate system can take the center of the image captured by the camera as the origin, and the above image coordinate system is a two-dimensional coordinate system. The camera coordinate system takes the optical center of the camera as the origin of the coordinate system, the Xc and Yc axes are parallel to the x and y axes of the image coordinate system, the optical axis of the camera is the Zc axis, and the coordinate system satisfies the right-hand rule. The optical center of the camera can be understood as the geometric center of the camera lens, and the above camera coordinate system is a three-dimensional coordinate system. The transformation relationship between the image coordinate system and the camera coordinate system can be as follows Figure 5 shown. Figure 5In the figure, the two-dimensional coordinate system o-xy is the image coordinate system of the image q captured by the camera. The intersection of the x-axis and the y-axis is o, that is, o is the origin of the image coordinate system, and the coordinates of any point on the image plane q can be expressed by the image coordinates (x, y). The three-dimensional coordinate system O-XYZ is the camera coordinate system, and any plane in the camera coordinate system can be called a spatial plane. The spatial plane Q is any plane in the spatial coordinate system (that is, the spatial plane Q is any plane in O-XYZ), and the coordinates of any point on the spatial plane Q can be expressed by the three-dimensional coordinates (X, Y, Z). The image plane q and the spatial plane Q have the following transformation relationship: q = sHQ. q represents the image coordinates of a point in the image plane, Q represents the three-dimensional coordinates of a point in the spatial plane, s is the scale factor, and the homography matrix H = MW.

[0090] in, M is the intrinsic parameter matrix of the camera, f x It represents the product of the physical focal length of the lens in the camera and the size of each unit of the imaging in the x direction. y Represents the product of the physical focal length of the camera lens and the size of each imaging unit in the y direction. x Indicates the offset of the projection coordinate center relative to the optical axis in the x direction, c y Indicates the offset of the projection coordinate center relative to the optical axis in the y direction. The size of each imaging unit can be a pixel in the image captured by the camera.

[0091] Where W = [R, t], where W is the camera's extrinsic parameter matrix. R is the 3x3 orthogonal identity matrix, also known as the rotation matrix, and t is the three-dimensional translation vector. If the image plane q and the spatial plane Q have the above transformation relationship, and the image coordinates, scale factor, and homography matrix H of a point on q are known, the spatial coordinates of that point on Q can be calculated based on the transformation relationship between the image plane and the spatial plane.

[0092] For example, Figure 5 In the image coordinates of a point on plane q, (x1, y1), if the camera's intrinsic parameter matrix is M, the camera's extrinsic parameter matrix is W, and the scale factor is s, then according to the transformation relationship between the image plane and the space plane, we can get the equation with X1, Y1, and Z1 as unknowns:

[0093]

[0094] Where H=MW. Solving this equation, we can obtain the coordinates (X1, Y1, Z1) of the point on the spatial plane Q.

[0095] Furthermore, the coordinates (X1, Y1, Z1) of the point on the spatial plane Q are converted to the vehicle coordinate system to obtain the three-dimensional coordinates (X2, Y2, Z2) of the point in the vehicle coordinate system.

[0096] For example, the three-dimensional coordinates (X2, Y2, Z2) can be determined using a perspective-n-point (PnP) algorithm based on a priori head model. The priori head model includes the relationship between the vehicle coordinate system and the image coordinate system of a specified point. Specifically, the coordinates of a point in the vehicle coordinate system and the coordinates of the same point in the camera coordinate system have the following relationship:

[0097]

[0098] Among them, p is the coordinate of the point in the pixel coordinate system, P C is the coordinate of the point in the camera coordinate system, P W is the coordinate of the point in the vehicle coordinate system, ω is the depth of the point, K is the intrinsic parameter matrix of the camera, R CW and is the pose transformation from the vehicle coordinate system to the camera coordinate system. More specifically, R CW is the rotation matrix from the vehicle coordinate system to the camera coordinate system (converting the same vector in the vehicle coordinate system to the camera coordinate system). is the corresponding translation vector (i.e., the vector from the origin of the camera coordinate system to the origin of the vehicle coordinate system, expressed in the camera coordinate system). Furthermore, according to the coordinates of n points in the prior head model in the vehicle coordinate system and the coordinates of these n points in the image coordinate system, R can be solved. CW and Furthermore, the three-dimensional coordinates (X2, Y2, Z2) of the facial feature point in the vehicle coordinate system can be obtained based on the coordinates (X1, Y1, Z1) of the facial feature point in the camera coordinate system.

[0099] In some possible implementations, the 3D position of the corresponding position of the face can be obtained through the facial key point detection algorithm, and then the three-dimensional coordinates (X2, Y2, Z2) of the facial feature points in the whole vehicle coordinate system can be determined; alternatively, the head reconstruction algorithm can be used to directly reconstruct the user's head in real time, and then the three-dimensional coordinates (X2, Y2, Z2) of the facial feature points in the whole vehicle coordinate system can be determined.

[0100] It should be understood that the three-dimensional coordinates of facial feature points in the cockpit can provide a more accurate reference for adjusting the display device.

[0101] S404: Adjust the display direction of the display device according to the first position information.

[0102] For example, the display device in the embodiment of the present application is a holographic projection device, and the image displayed by the display device is a 3D image. For example, the direction of the holographic image displayed by the display device can be adjusted according to the first position information so that the side of the holographic image containing the content to be conveyed to the user faces the user. For example, when the holographic image is a 3D image of a virtual assistant, the face, or the body and face of the virtual assistant can be adjusted to face the user, such as Figure 6 As shown in (a) in FIG, when the user is in the passenger seat, the holographic image of the virtual assistant can be adjusted to face the passenger seat user. In some possible implementations, the facial orientation of the digital person displayed on the display device can be adjusted based on the first position information.

[0103] In some possible implementations, the preset direction of the holographic image displayed by the display device can be as follows: Figure 6 As shown in (b), for example, when the holographic image is a virtual assistant, the preset direction can be the direction of the virtual assistant's face; when the holographic image is other images, the preset direction can be the direction of the side containing the content that you want to convey to the user.

[0104] Furthermore, adjusting the display direction of the display device according to the first position information may include: adjusting the posture of the holographic image so that the side containing the content to be conveyed to the user is turned toward the first position of the user's facial feature point, that is, Figure 6 The preset direction shown in (b) points to the first position of the facial feature point, as shown in FIG. Figure 6 As shown in (c) in .

[0105] In some possible implementations, the position of the holographic image can be adjusted by adjusting the angle of the holographic projection device. For example, the holographic projection device includes a projection device and an imaging device, wherein the projection device projects light to the imaging device so that the imaging device can display a three-dimensional stereoscopic image. In some possible implementations, when the projection device rotates, the three-dimensional stereoscopic image rotates, and the rotation angle of the projection device in a plane perpendicular to the longitudinal symmetry plane of the vehicle can be adjusted according to the first position information, so that the side of the holographic image containing the content to be conveyed to the user is facing the user. In some possible implementations, there is a mapping relationship between the angle transformation of the holographic projection device and the angle transformation of the holographic image. For example, the holographic projection device rotates α degrees (degree, °) clockwise in a plane perpendicular to the longitudinal symmetry plane of the vehicle, and the holographic image will rotate β degrees clockwise in a plane perpendicular to the longitudinal symmetry plane of the vehicle, wherein α and β may be equal or unequal, and the embodiments of the present application are not limited to this.

[0106] For example, Figure 6The initial position of the preset direction indicating holographic image shown in (b) is recorded as 0 degrees, and the coordinates of the holographic image in the vehicle coordinate system are (X3, Y3, Z3). Figure 6 The coordinates of the first position shown in (c) in the vehicle coordinate system are (X4, Y4, Z4). Based on the first position information and the initial position of the holographic image, the first rotation angle θ1 can be determined to be arctan((X4-X3) / (Y4-Y3)), and the first rotation direction is clockwise rotation in a plane perpendicular to the longitudinal symmetry plane of the vehicle. Furthermore, based on the mapping relationship between the angle transformation of the holographic projection device and the angle transformation of the holographic image, the rotation angle θ2 of the holographic projection device is determined, and then the holographic projection device is controlled to rotate clockwise by θ2 in a plane perpendicular to the longitudinal symmetry plane of the vehicle, so that the side of the holographic image containing the content to be conveyed to the user faces the user.

[0107] For example, the projection device can realize digital high-definition multi-channel hard disk synchronous playback; the imaging device can include a spectroscope. Alternatively, the projection device and imaging device can also include other devices, which are not specifically limited in the embodiments of the present application.

[0108] In some possible implementations, the position of the three-dimensional image can also be adjusted based on the first position information. For example, when the three-dimensional image is a virtual assistant, the orientation of the virtual assistant's head and / or body can be adjusted so that the virtual assistant's head is oriented toward the first position.

[0109] In some possible implementations, the height of the image displayed by the holographic projection device can also be adjusted based on the first position information. For example, the height of the image displayed by the holographic projection device can be adjusted based on the height information in the first position information, thereby controlling the holographic projection device to display the holographic image at a corresponding height. For example, the rotation angle of the projection device within a plane parallel to the longitudinal symmetry plane of the vehicle, and thus the height of the holographic image, can be controlled based on the height information in the first position information.

[0110] S405: Acquire a second facial image of the user, and determine second position information of facial feature points based on the second facial image.

[0111] Exemplarily, the second position information is used to indicate the second position of the facial feature point in the vehicle coordinate system.

[0112] In some possible implementations, when the user interacts with the holographic image, the user's head position changes. In this case, a second facial image of the user can be obtained, and second position information of facial feature points can be determined based on the second facial image. Specifically, the method for determining the second position information of facial feature points can be referred to the description in S403 and will not be repeated here.

[0113] S406: Determine whether the distance difference between the first position and the second position is greater than or equal to a first threshold.

[0114] Specifically, when it is determined that the distance difference between the first position and the second position is greater than or equal to the first threshold, S407 is executed; otherwise, S408 is executed.

[0115] It should be understood that triggering the display device to adjust when the value is greater than or equal to a preset threshold can avoid frequent adjustment of the display device.

[0116] For example, the first threshold may be 15 centimeters (cm), or 20 cm, or other distances, which is not specifically limited in the embodiments of the present application.

[0117] S407: Adjust the display direction of the display device according to the second position information.

[0118] In some possible implementations, the first position and the second position are both the positions of the facial feature points of the first user (such as the main driver) in the vehicle coordinate system. Then, the display direction of the display device can be adjusted according to the second position information as follows: Figure 6 As shown in (d) in .

[0119] In some possible implementations, the first position is the position of the facial feature points of the first user (such as the user in the main driver's seat) in the vehicle coordinate system, and the second position is the position of the facial feature points of the second user (such as the user in the co-pilot seat) in the vehicle coordinate system. The display direction of the display device can be adjusted according to the second position information as follows: Figure 6 As shown in (e) in the figure. For example, when the second position and the first position indicate different users, the determination of the second position may be actively triggered by the second user. For example, the second user triggers the determination through a voice command, such as "Hologram, please face me"; or the second user triggers the determination through a related button, such as the second user clicking a "Turn on hologram" button.

[0120] For example, the angle and direction in which the display device needs to be rotated can be determined based on the first position information and the second position information, and the display device can be controlled to rotate so that the display direction of the display device is toward the second position. In some possible implementations, the display device rotates within a plane perpendicular to the longitudinal symmetry plane of the vehicle.

[0121] S408, end.

[0122] It should be understood that the above “end” represents the end of the process of adjusting the display device.

[0123] Figure 4The steps or operations of the method for controlling the display device shown are only exemplary. The embodiment of the present application may also perform other operations or Figure 4 In addition, Figure 4 The steps in Figure 4 are executed in a different order than the ones presented, and may not be executed Figure 4 All operations in . Exemplarily, S405 to S407 may not be executed. Exemplarily, S402 may be skipped and S403 may be executed directly.

[0124] A method for adjusting a display device provided in an embodiment of the present application can adjust the display direction of the display device according to the three-dimensional coordinates of the feature points of the user's head in the vehicle coordinate system, thereby accurately adjusting the orientation of the image displayed by the display device, providing the user with a better interactive effect and improving the user's interactive experience.

[0125] Figure 7 A method for adjusting a display device provided by an embodiment of the present application is shown. The method 700 can be applied to Figure 1 The scenario shown can also be applied to Figure 2 In the vehicle 100 shown, the method may also be performed by Figure 3 The system shown is executed. Figure 7 The steps or operations of the method for controlling the display device shown are only exemplary. The embodiment of the present application may also perform other operations or Figure 4 The method 700 includes:

[0126] S701: Acquire audio data inside the vehicle and determine the sound source position of the audio data.

[0127] For example, the audio information may be obtained by excluding various invalid audio information from the collected audio information inside the vehicle. Invalid audio information may be audio information with an excessively low volume. The sound source location may be the location of the sound source corresponding to the audio information. The sound source location may be relative to a light display device based on sound source positioning, or may be a specific location coordinate. This embodiment of the present application does not impose any specific limitations on this.

[0128] For example, the location of a sound source can be determined based on the time difference of arrival (TDOA) principle using audio information collected by multiple sound sensors. For example, sound sensors A and B each detect audio emanating from a sound source S. The sound signal from source S arrives at sensor A at time t1 and at sensor B at time t2. The time difference dt = |t1 t2|. Assuming the distance between source S and sensor A is AS, the distance between source S and sensor B is BS, and the speed of sound is c, we can obtain dt = t1 t2 = AS / c BS / c. Based on the distance a between the two sound sensors, one of the sensors is selected as the reference point to determine the location of the sound source.

[0129] In some possible implementations, the audio information may be a voice message containing a specific wake-up word, such as "turn on the display device"; or "virtual assistant, please face me"; or other voice information.

[0130] S702: Determine the user's location based on the sound source location.

[0131] In some possible implementations, within a preset time period, two sound source locations are determined based on the acquired audio data. For example, if the sound source locations are determined to be at the driver's seat and the co-pilot seat, the display device can be controlled to "copy" the image, and the two images obtained after the copying are controlled to face the user at the driver's seat and the user at the co-pilot seat respectively, such as Figure 8 As shown in (a) in FIG. For example, the image is a 3D image of a virtual assistant, and the two copied images are controlled by the same command.

[0132] In some possible implementations, within a preset time period, two sound source locations are determined based on the acquired audio data. For example, if the sound source locations are determined to be the main driver's seat and the co-pilot's seat, the display device can be controlled to display two images, and the two images can be controlled to face the user in the main driver's seat and the user in the co-pilot's seat, respectively.

[0133] Exemplarily, the preset time length may be 3 seconds, or 5 seconds, or other preset time lengths.

[0134] In some possible implementations, the display device is set at the front center armrest. When the sound source position is determined to be at the left rear seat and the right rear seat, the display device can be controlled to project two images, and the two images are respectively facing the user at the left rear seat and the user at the right rear seat.

[0135] In some possible implementations, when the number of users is greater than 2, the display device may be controlled to project multiple images, and the multiple images may be adjusted to face the multiple users respectively.

[0136] In some possible implementations, after determining the user's position based on the sound source position, the display device can be adjusted so that the image projected by the display device moves to the sound source position. For example, if the sound source position is the co-pilot seat, the display device can be adjusted so that the image position moves to the co-pilot seat, as shown in the following example: Figure 8 As shown in (b) in .

[0137] In some possible implementations, a holographic projection device includes a projection device and an imaging device, wherein the projection device projects light onto the imaging device so that the imaging device can display a three-dimensional image. In some possible implementations, the imaging device is positioned to the passenger seat, and the projection angle of the projection device is adaptively adjusted to form a holographic image at the passenger seat.

[0138] An embodiment of the present application provides a method for adjusting a display device. In a multi-user scenario, when a user issues a voice command, the method can determine whether the user issuing the command is the driver, front passenger, or rear passenger based on the sound source location. The method then adjusts the corresponding display device so that the projected 3D image faces that user, improving interactivity between the user and the image projected by the display device. In a multi-user scenario, the display device can also replicate the image, so that each image faces one of the multiple users, further enhancing the user's interactive experience and driving pleasure.

[0139] Figure 9 A method for adjusting a display device provided by an embodiment of the present application is shown. The method 900 can be applied to Figure 1 The scenario shown can also be applied to Figure 2 In the vehicle 100 shown, the method may also be performed by Figure 3 The system shown is executed. Figure 9 The steps or operations of the method for controlling the display device shown are only exemplary. The embodiment of the present application may also perform other operations or Figure 4 The method 900 includes:

[0140] S901: Acquire a first facial image of a first user.

[0141] Specifically, the method for obtaining the first facial image of the first user can refer to the description in the above embodiment and will not be repeated here.

[0142] S902: Determine first spatial coordinates of facial feature points of the first user according to the first facial image.

[0143] In some possible implementations, the first spatial coordinates may be the three-dimensional coordinates of the facial feature points of the first user in the vehicle coordinate system.

[0144] For example, the facial feature points of the first user may be the points in the above embodiment. Further, the method for determining the first spatial coordinates may refer to the description in the above embodiment, which will not be repeated here.

[0145] S903: Adjust the orientation of a first side of an image displayed by the display device according to the first spatial coordinates, where the first side of the image includes information to be conveyed to the first user.

[0146] Exemplarily, the image may be a holographic image, or a two-dimensional image with a stereoscopic effect and depth of field, which is not specifically limited in the embodiments of the present application.

[0147] Specifically, the method of adjusting the orientation of the image displayed by the display device according to the first spatial coordinates can be described in the above embodiment and will not be repeated here.

[0148] A method for adjusting a display device provided in an embodiment of the present application enables a user to experience interaction with the display device, thereby helping to improve the user's usage experience at the interaction level.

[0149] In the various embodiments of the present application, unless otherwise specified or there is a logical conflict, the terms and / or descriptions between the various embodiments are consistent and can be referenced by each other. The technical features in different embodiments can be combined to form new embodiments according to their inherent logical relationships.

[0150] Combined with the above Figures 4 to 9 The method provided in the embodiment of the present application is described in detail. Figure 10 and Figure 11 The apparatus provided in the embodiments of the present application will be described in detail. It should be understood that the description of the apparatus embodiment corresponds to the description of the method embodiment, and therefore, for matters not described in detail, reference can be made to the method embodiment above, and for the sake of brevity, no further description will be given here.

[0151] Figure 10 A schematic block diagram of an apparatus 2000 for adjusting a display device according to an embodiment of the present application is shown, wherein the apparatus 2000 includes an acquisition unit 2010 and a processing unit 2020. The acquisition unit 2010 can implement corresponding communication functions, and the processing unit 2020 is used for data processing.

[0152] Optionally, the device 2000 may further include a storage unit, which may be used to store instructions and / or data. The processing unit 2020 may read the instructions and / or data in the storage unit so that the device implements the aforementioned method embodiment.

[0153] The apparatus 2000 may include a Figure 4 、 Figure 7 、 Figure 9Furthermore, each unit in the apparatus 2000 and the above-mentioned other operations and / or functions are respectively for realizing Figure 4 、 Figure 7 、 Figure 9 The corresponding process of the method embodiment in FIG.

[0154] Wherein, when the device 2000 is used to perform Figure 9 When the method 900 is performed, the acquisition unit 2010 can be used to execute S901 in the method 900, and the processing unit 2020 can be used to execute S902 and S903 in the method 900.

[0155] The device 2000 includes: an acquisition unit 2010, used to acquire a first facial image of a first user; a processing unit 2020, used to determine the first spatial coordinates of the facial feature points of the first user based on the first facial image; and adjust the orientation of the first side of the image displayed by the display device according to the first spatial coordinates, where the first side of the image contains information to be conveyed to the first user.

[0156] Optionally, the acquisition unit 2010 is also used to: acquire a second facial image of the second user after adjusting the orientation of the first side of the image displayed by the display device according to the first spatial coordinates; the processing unit 2020 is also used to: determine the second spatial coordinates of the facial feature points of the second user based on the second facial image; and adjust the orientation of the first side of the image displayed by the display device according to the second spatial coordinates.

[0157] Optionally, the acquisition unit 2010 is also used to: acquire a third facial image of the first user after adjusting the orientation of the first side of the image displayed by the display device according to the first spatial coordinates; the processing unit 2020 is also used to: determine the third spatial coordinates of the facial feature points of the first user based on the third facial image; when the distance between the first spatial coordinate and the third spatial coordinate is greater than or equal to a preset threshold, adjust the orientation of the first side of the image displayed by the display device according to the third spatial coordinates.

[0158] Optionally, the acquisition unit 2010 is also used to: before acquiring the first facial image of the first user, acquire audio information, the audio information including the voice instructions of the first user, the voice instructions being used to instruct to turn on and / or adjust the display device; the processing unit 2020 is also used to: determine the sound source position of the audio information; and determine the position of the first user based on the sound source position.

[0159] Optionally, the audio information also includes voice commands of the second user, and the processing unit 2020 is further used to: control the display device to display the first image and the second image; adjust the display device so that the first side of the first image is facing the first user, and the first side of the second image is facing the second user, wherein the image displayed by the display device includes the first image and / or the second image, the first side of the first image contains information to be conveyed to the first user, and the first side of the second image contains information to be conveyed to the second user.

[0160] Optionally, the processing unit 2020 is further configured to adjust a position of the image displayed by the display device according to the first spatial coordinates.

[0161] Optionally, the acquisition unit 2010 is also used to: obtain initial posture information of the display device before adjusting the orientation of the first side of the image displayed by the display device according to the first spatial coordinates, and the initial posture information is used to indicate the posture angle and position of the display device; the processing unit 2020 is also used to: determine the rotation angle and rotation direction of the display device based on the first spatial coordinates and the initial posture information; and adjust the orientation of the first side of the image displayed by the display device according to the rotation angle and the rotation direction.

[0162] Optionally, the first spatial coordinates of the facial feature points of the first user are used to indicate the three-dimensional position of the facial feature points of the first user in a vehicle, and the vehicle includes the display device.

[0163] Optionally, the display device includes a holographic projection device, and the image includes a three-dimensional 3D image.

[0164] It should be understood that the division of the various units in the above device is merely a division of logical functions. In actual implementation, they may be fully or partially integrated into a single physical entity, or they may be physically separated. Furthermore, the units in the device may be implemented in the form of a processor calling software; for example, the device includes a processor connected to a memory storing instructions, and the processor calls the instructions stored in the memory to implement any of the above methods or the functions of the various units in the device. The processor may be a general-purpose processor, such as a CPU or a microprocessor, and the memory may be a memory within the device or external to the device. Alternatively, the units in the device may be implemented in the form of hardware circuits, and the functions of some or all of the units may be implemented through the design of the hardware circuits. The hardware circuits may be understood as one or more processors. For example, in one implementation, the hardware circuit is an ASIC, and the functions of some or all of the above units may be implemented through the design of the logical relationships between the components within the circuits. In another implementation, the hardware circuit may be implemented using a PLD, such as an FPGA, which may include a large number of logic gate circuits. The connections between the logic gate circuits are configured using a configuration file to implement the functions of some or all of the above units. All units of the above apparatus may be implemented entirely in the form of software called by a processor, or entirely in the form of hardware circuits, or partially in the form of software called by a processor and the rest in the form of hardware circuits.

[0165] In an embodiment of the present application, a processor is a circuit with the ability to process signals. In one implementation, the processor may be a circuit with the ability to read and execute instructions, such as a CPU, a microprocessor, a GPU, or a DSP. In another implementation, the processor may implement certain functions through the logical relationship of a hardware circuit, and the logical relationship of the hardware circuit may be fixed or reconfigurable, such as a hardware circuit implemented by an ASIC or PLD, such as an FPGA. In a reconfigurable hardware circuit, the process of the processor loading a configuration document to implement the configuration of the hardware circuit can be understood as the process of the processor loading instructions to implement the functions of some or all of the above units. In addition, it can also be a hardware circuit designed for artificial intelligence, which can be understood as an ASIC, such as an NPU, TPU, DPU, etc.

[0166] It can be seen that each unit in the above device can be one or more processors (or processing circuits) configured to implement the above method, such as: CPU, GPU, NPU, TPU, DPU, microprocessor, DSP, ASIC, FPGA, or a combination of at least two of these processor forms.

[0167] In addition, the various units in the above devices can be fully or partially integrated together, or can be implemented independently. In one implementation, these units are integrated together and implemented in the form of a system-on-a-chip (SOC). The SOC may include at least one processor for implementing any of the above methods or implementing the functions of the various units of the device. The type of the at least one processor can be different, for example, including a CPU and FPGA, a CPU and an artificial intelligence processor, a CPU and a GPU, etc.

[0168] In a specific implementation, the operations performed by the acquisition unit 2010 and the processing unit 2020 may be performed by the same processor, or may be performed by different processors, for example, by multiple processors. Figure 2 The processors may be connected to one or more sensors in the perception system 120, and the facial image of the user may be obtained from the one or more sensors and processed; in another example, the one or more processors may also be connected to one or more display devices in the display device 130, and then control the direction of the image displayed by the display device. For example, in the specific implementation process, the one or more processors may be provided in the vehicle computer, or may be provided in other vehicle-mounted terminals. For example, in the specific implementation process, the device 2000 may be a chip provided in the vehicle computer or other vehicle-mounted terminals. For example, in the specific implementation process, the device 2000 may be provided in the vehicle computer or other vehicle-mounted terminals. Figure 2 Computing platform 150 is shown.

[0169] Figure 11 This is a schematic block diagram of an apparatus for adjusting a display device according to an embodiment of the present application. Figure 11 The apparatus 2100 for adjusting a display device shown may include a processor 2110, a transceiver 2120, and a memory 2130. The processor 2110, the transceiver 2120, and the memory 2130 are connected via an internal connection path. The memory 2130 is used to store instructions, and the processor 2110 is used to execute the instructions stored in the memory 2130 to receive / send some parameters via the transceiver 2120. Optionally, the memory 2130 may be coupled to the processor 2110 via an interface or may be integrated with the processor 2110.

[0170] It should be noted that the transceiver 2120 may include but is not limited to a transceiver device such as an input / output interface to achieve communication between the device 2100 and other devices or communication networks.

[0171] The processor 2110 can be a general-purpose CPU, microprocessor, ASIC, GPU, or one or more integrated circuits, configured to execute relevant programs to implement the method for adjusting a display device according to the embodiment of the present application. The processor 2110 can also be an integrated circuit chip with signal processing capabilities. In specific implementations, the various steps of the method for adjusting a display device according to the present application can be performed by hardware integrated logic circuits or software instructions within the processor 2110. The processor 2110 can also be a general-purpose processor, DSP, ASIC, FPGA, or other programmable logic device, discrete gate or transistor logic device, or discrete hardware component. It can implement or execute the various methods, steps, and logic block diagrams disclosed in the embodiments of the present application. The general-purpose processor can be a microprocessor or any conventional processor. The steps of the method disclosed in conjunction with the embodiments of the present application can be directly implemented and executed by a hardware decoding processor, or by a combination of hardware and software modules within the decoding processor. The software modules can be located in storage media well-known in the art, such as random access memory, flash memory, read-only memory, programmable read-only memory, electrically erasable programmable memory, registers, or the like. The storage medium is located in the memory 2130 , and the processor 2110 reads the information in the memory 2130 and executes the method for adjusting the display device according to the embodiment of the method of the present application in combination with its hardware.

[0172] The memory 2130 may be a read-only memory (ROM), a static storage device, a dynamic storage device, or a random access memory (RAM).

[0173] The transceiver 2120 uses a transceiver device such as, but not limited to, a transceiver to implement communication between the device 2100 and other devices or a communication network. For example, the transceiver 2120 can be used to obtain a user's facial image and / or audio information.

[0174] The present application also provides a vehicle, which may include the aforementioned apparatus 2000 or apparatus 2100, and the aforementioned display device. The display device may be a holographic projection device, displaying a three-dimensional image; or an in-vehicle display screen, displaying a digital human image; or other display devices, which are not specifically limited in this application.

[0175] An embodiment of the present application further provides a computer program product, which includes: computer program code, which enables the computer to execute the above method when the computer program code is run on a computer.

[0176] The embodiment of the present application also provides a computer-readable storage medium, which stores a program code. When the computer program code is run on a computer, the computer executes the above Figure 4 、 Figure 7 、 Figure 9 Any of the methods in .

[0177] The embodiment of the present application also provides a chip, comprising: at least one processor and a memory, wherein the at least one processor is coupled to the memory and is configured to read and execute instructions in the memory to perform the above Figure 4 、 Figure 7 、 Figure 9 Any of the methods in .

[0178] This application will present various aspects, embodiments, or features around systems including multiple devices, components, modules, etc. It should be understood and appreciated that each system may include additional devices, components, modules, etc., and / or may not include all of the devices, components, modules, etc. discussed in conjunction with the figures. Furthermore, combinations of these aspects may also be used.

[0179] Additionally, in the embodiments of this application, words such as "exemplary" and "for example" are used to indicate examples, illustrations, or descriptions. Any embodiment or design described in this application as "exemplary" should not be construed as being preferred or advantageous over other embodiments or designs. Rather, the use of the word "exemplary" is intended to present concepts in a concrete manner.

[0180] In the embodiments of the present application, “corresponding” and “relevant” may sometimes be used interchangeably. It should be noted that when the distinction between them is not emphasized, the meanings they intend to express are consistent.

[0181] The network architecture and business scenarios described in the embodiments of the present application are intended to more clearly illustrate the technical solutions of the embodiments of the present application, and do not constitute a limitation on the technical solutions provided in the embodiments of the present application. A person skilled in the art will appreciate that, with the evolution of the network architecture and the emergence of new business scenarios, the technical solutions provided in the embodiments of the present application are equally applicable to similar technical problems.

[0182] References to "one embodiment" or "some embodiments" in this specification mean that a particular feature, structure, or characteristic described in conjunction with that embodiment is included in one or more embodiments of the present application. Thus, phrases such as "in one embodiment," "in some embodiments," "in other embodiments," and "in yet other embodiments" appearing in various places in this specification do not necessarily refer to the same embodiment, but rather mean "one or more but not all embodiments," unless otherwise specifically emphasized. The terms "including," "comprising," "having," and variations thereof mean "including but not limited to," unless otherwise specifically emphasized.

[0183] In this application, "at least one" means one or more, and "more" means two or more. "And / or" describes the association relationship of associated objects, indicating that three relationships may exist. For example, A and / or B can mean: including the existence of A alone, the existence of A and B at the same time, and the existence of B alone, where A and B can be singular or plural. The character " / " generally indicates that the previous and next associated objects are in an "or" relationship. "At least one of the following items" or similar expressions refers to any combination of these items, including any combination of single or plural items. For example, at least one of a, b, or c can mean: a, b, c, ab, ac, bc, or abc, where a, b, c can be single or multiple.

[0184] Those skilled in the art will appreciate that the units and algorithm steps of each example described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Professional and technical personnel can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of this application.

[0185] Those skilled in the art will clearly understand that, for the convenience and brevity of description, the specific working processes of the systems, devices and units described above can refer to the corresponding processes in the aforementioned method embodiments and will not be repeated here.

[0186] In the several embodiments provided in this application, it should be understood that the disclosed systems, devices and methods can be implemented in other ways. For example, the device embodiments described above are merely schematic. For example, the division of the units is merely a logical function division. In actual implementation, there may be other division methods, such as multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, indirect coupling or communication connection of devices or units, which can be electrical, mechanical or other forms.

[0187] The units described as separate components may or may not be physically separate, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed across multiple network units. Some or all of these units may be selected to achieve the purpose of this embodiment according to actual needs.

[0188] In addition, each functional unit in each embodiment of the present application may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit.

[0189] If the functions are implemented in the form of software functional units and sold or used as independent products, they can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present application can essentially be embodied in the form of a software product, which is stored in a storage medium and includes a number of instructions for enabling a computer device (which can be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods described in each embodiment of the present application. The aforementioned storage medium includes various media that can store program code, such as a USB flash drive, a mobile hard drive, ROM, RAM, a magnetic disk, or an optical disk.

[0190] The above description is merely a specific embodiment of the present application, but the scope of protection of the present application is not limited thereto. Any changes or substitutions that can be easily conceived by a person skilled in the art within the technical scope disclosed in this application should be included in the scope of protection of this application. Therefore, the scope of protection of this application should be based on the scope of protection of the claims.

Claims

1. A method for adjusting a display device, characterized in that: include: Acquiring audio information, where the audio information includes a voice command of a first user and a voice command of a second user, where the voice command is used to instruct to turn on and / or adjust the display device; determining a position of the first user and a position of the second user according to a sound source position of the audio information; Acquire a first facial image of the first user; determining first spatial coordinates of facial feature points of the first user based on the first facial image; obtaining a second facial image of the second user; determining second spatial coordinates of facial feature points of the second user based on the second facial image; controlling the display device to display a first image and a second image, where the second image is copied based on the first image; Based on the first spatial coordinate and the second spatial coordinate, the display device is adjusted so that the first side of the first image faces the first user and the first side of the second image faces the second user, wherein the first side of the first image contains information to be conveyed to the first user and the first side of the second image contains information to be conveyed to the second user.

2. The method according to claim 1, characterized in that After adjusting the orientation of the first side of the image displayed by the display device according to the first spatial coordinates, the method further includes: obtaining a third facial image of the first user; determining third spatial coordinates of facial feature points of the first user based on the third facial image; When the distance between the first space coordinate and the third space coordinate is greater than or equal to a preset threshold, the orientation of the first side of the image displayed by the display device is adjusted according to the third space coordinate.

3. The method according to claim 1 or 2, characterized in that The method further comprises: The position of the image displayed by the display device is adjusted according to the first spatial coordinates.

4. The method according to any one of claims 1 to 3, characterized in that Before adjusting the orientation of the first side of the image displayed by the display device according to the first spatial coordinates, the method further includes: Acquire initial posture information of the display device, where the initial posture information is used to indicate the posture angle and position of the display device; The adjusting the orientation of the first side of the image displayed by the display device according to the first spatial coordinates includes: Determine a rotation angle and a rotation direction of the display device according to the first spatial coordinates and the initial posture information; The orientation of the first side of the image displayed by the display device is adjusted according to the rotation angle and the rotation direction.

5. The method according to any one of claims 1 to 4, characterized in that The first spatial coordinates of the facial feature points of the first user are used to indicate the three-dimensional position of the facial feature points of the first user in a vehicle, and the vehicle includes the display device.

6. The method according to any one of claims 1 to 5, characterized in that The display device includes a holographic projection device, and the image includes a three-dimensional 3D image.

7. A device for adjusting a display device, characterized in that: include: An acquiring unit, configured to acquire audio information, the audio information including a voice instruction of a first user and a voice instruction of a second user, the voice instruction being used to instruct to turn on and / or adjust the display device; a processing unit, configured to determine a position of the first user and a position of the second user based on a sound source position of the audio information; The acquisition unit is further configured to acquire a first facial image of a first user; The processing unit is further configured to determine first spatial coordinates of facial feature points of the first user based on the first facial image; The acquisition unit is further configured to acquire a second facial image of a second user; The processing unit is further configured to determine second spatial coordinates of facial feature points of the second user based on the second facial image; The processing unit is further configured to control the display device to display a first image and a second image, where the second image is copied based on the first image; Adjust the display device so that the first side of the first image faces the first user, and the first side of the second image faces the second user, wherein the image displayed by the display device includes the first image and / or the second image, the first side of the first image contains information to be conveyed to the first user, and the first side of the second image contains information to be conveyed to the second user.

8. The device according to claim 7, characterized in that The acquisition unit is further configured to: After adjusting the orientation of the first side of the image displayed by the display device according to the first spatial coordinates, acquiring a third facial image of the first user; The processing unit is also used to: determine the third spatial coordinates of the facial feature points of the first user based on the third facial image; when the distance between the first spatial coordinates and the third spatial coordinates is greater than or equal to a preset threshold, adjust the orientation of the first side of the image displayed by the display device according to the third spatial coordinates.

9. The device according to claim 7 or 8, characterized in that The processing unit is further configured to: The position of the image displayed by the display device is adjusted according to the first spatial coordinates.

10. The device according to any one of claims 7 to 9, characterized in that The first spatial coordinates of the facial feature points of the first user are used to indicate the three-dimensional position of the facial feature points of the first user in a vehicle, and the vehicle includes the display device.

11. The device according to any one of claims 7 to 10, characterized in that The acquisition unit is further configured to: before adjusting the orientation of the first side of the image displayed by the display device according to the first spatial coordinates, acquire initial posture information of the display device, wherein the initial posture information is used to indicate the posture angle and position of the display device; The processing unit is further configured to: determine a rotation angle and a rotation direction of the display device according to the first spatial coordinates and the initial posture information; and adjust the orientation of the first side of the image displayed by the display device according to the rotation angle and the rotation direction.

12. The device according to any one of claims 7 to 11, characterized in that The display device includes a holographic projection device, and the image includes a three-dimensional 3D image.

13. A device for adjusting a display device, characterized in that: include: memory for storing computer programs; A processor, configured to execute the computer program stored in the memory, so that the apparatus performs the method according to any one of claims 1 to 6.

14. A vehicle, characterized in that: The device comprises the apparatus according to any one of claims 7 to 12 and the display device.

15. A computer-readable storage medium, characterized in that A computer program is stored thereon, and when the computer program is executed by a computer, the method according to any one of claims 1 to 6 is implemented.

16. A chip, characterized in that: The chip includes a processor and a data interface, and the processor reads instructions stored in a memory through the data interface to execute the method according to any one of claims 1 to 6.

Images