Head-up display device, display control method, device and projection device

Abstract

The present disclosure provides a head-up display device, a display control method, a device, a projection device, and a computer-readable storage medium, belonging to the field of assisted driving technology. The method includes: obtaining dynamic information of the vehicle itself; in the case of predicting that the vehicle itself shakes, determining a display setting of the projection device based on the dynamic information; in the case of the vehicle itself shaking, controlling the projection device to display an image according to the display setting, so that the shaking displacement of the image displayed by the projection device is less than a displacement threshold value.

Description

Technical Field

[0001] The present disclosure relates to the field of assisted driving technologies, and particularly to a head-up display device, a display control method, a device, a projection device, and a computer-readable storage medium. Background Art

[0002] A head-up display (HUD) is a device that directly projects key information onto the windshield or a transparent display screen in front of the driver's line of sight, enabling the driver to obtain important information such as the vehicle's speed, navigation, traffic sign recognition, and incoming call information in real time without looking down at the dashboard or navigation system. With the development of technology, HUDs have more and more functions and are more intelligent. For example, an augmented reality head-up display (AR-HUD) combines virtual information with the actual road environment through augmented reality technology to provide more intuitive navigation guidance.

[0003] However, due to the diverse driving environments, during driving, affected by the external environment, the vehicle may vibrate. And because the vibrations of the vehicle and the driver are not in the same frequency, the image displayed by the HUD observed by the driver vibrates, resulting in a blurred display image, causing the driver to feel dizzy and potentially posing a safety problem. Summary of the Invention

[0004] The present disclosure provides a head-up display device, a display control method, a device, a projection device, and a computer-readable storage medium, which can adjust the display position of the display image on a vibrating road surface to avoid the display image vibrating up and down and resulting in a blurred display.

[0005] In the solution provided by the present disclosure, the display control method can be executed by the projection device or some components in the projection device. Among them, the projection device has a projection function. For example, it can be an AR-HUD, a HUD, or other devices with a projection function. Some components in the projection device can be a processing chip, a processing circuit, a processor, etc.

[0006] The technical solution of the present disclosure is implemented as follows:

[0007] In a first aspect, the present disclosure provides a display control method, which includes: obtaining the dynamic information of the vehicle; when it is predicted that the vehicle vibrates, determining the display settings of the projection device based on the dynamic information; when the vehicle vibrates, controlling the projection device to display an image according to the display settings, so that the vibration displacement of the image displayed by the projection device is less than the displacement threshold.

[0008] Second aspect, the present disclosure provides a display control device, which includes: an acquisition part, a determination part, and a control part; the acquisition part is configured to acquire the dynamic information of the vehicle; the determination part is configured to, when it is predicted that the vehicle shakes, determine the display settings of the projection device based on the dynamic information; the control part is configured to, when the vehicle shakes, control the projection device to display an image according to the display settings, so that the shaking displacement of the image displayed by the projection device is less than the displacement threshold.

[0009] Third aspect, the present disclosure provides a projection device, which includes: a processor and a memory; the processor is used to execute the instructions stored in the memory to implement the display control method as described in the first aspect.

[0010] Fourth aspect, the present disclosure provides a computer-readable storage medium, which stores at least one instruction, and the at least one instruction is used to be executed by a processor to implement the display control method as described in the first aspect.

[0011] Fifth aspect, the present disclosure provides a head-up display device, which includes a display control part and a display part; wherein, the display control part is configured to acquire the dynamic information of the vehicle; and, when it is predicted that the vehicle shakes, determine the display settings based on the dynamic information; and, when the vehicle shakes, display an image according to the display settings, so that the shaking displacement of the displayed image is less than the displacement threshold; the display part is configured to display the displayed image on the front windshield of the vehicle.

[0012] Sixth aspect, the present disclosure provides a vehicle, which includes the head-up display device as described in the fifth aspect.

[0013] The present disclosure provides a display control method, which includes: acquiring the dynamic information of the vehicle; when it is predicted that the vehicle shakes, determining the display settings of the projection device based on the dynamic information; when the vehicle shakes, controlling the projection device to display an image according to the display settings, so that the shaking displacement of the image displayed by the projection device is less than the displacement threshold. That is, as the vehicle body shakes, the image displayed by the projection device remains fixed relative to the driver's line of sight, or the change value is so small that the human eye cannot distinguish it. Sensually, it is felt that the displayed image does not shake as the vehicle shakes, avoiding the blurring of the displayed image caused by the vehicle shaking. Thus, on a bumpy road, the driver can clearly and intuitively obtain the required auxiliary information from the projection device. BRIEF DESCRIPTION OF THE DRAWINGS

[0014] Figure 1 It is a schematic diagram of the composition of an in-vehicle system provided by the present disclosure.

[0015] Figure 2 An exemplary top view of the vehicle provided by the present disclosure.

[0016] Figure 3 An exemplary perspective view from the driver's seat of the vehicle provided by the present disclosure.

[0017] Figure 4 A schematic diagram of the architecture of the head-up display device provided by the present disclosure.

[0018] Figure 5a A perspective view of the windshield when the vehicle provided by the present disclosure is driving on a non-jittery road surface.

[0019] Figure 5b A perspective view of the windshield when the vehicle provided by the present disclosure is driving on an upward jittery road surface.

[0020] Figure 5c A perspective view of the windshield when the vehicle provided by the present disclosure is driving on a downward jittery road surface.

[0021] Figure 6 One of the schematic flow diagrams of the display control method provided by the present disclosure.

[0022] Figure 7 A schematic diagram of a vehicle coordinate system provided by the present disclosure.

[0023] Figure 8 A schematic diagram of the road condition information collected by the vehicle provided by the present disclosure.

[0024] Figure 9a A schematic diagram of predicting that jitter is about to occur provided by the present disclosure.

[0025] Figure 9b A schematic diagram of predicting that jitter will not occur provided by the present disclosure.

[0026] Figure 10 A perspective view of the windshield when the vehicle with height compensation is driving on an upward jittery road surface provided by the present disclosure.

[0027] Figure 11 The second schematic flow diagram of the display control method provided by the present disclosure.

[0028] Figure 12a A schematic diagram of the display position of the virtual image displayed when the vehicle provided by the present disclosure is driving on a non-jittery road surface.

[0029] Figure 12b A schematic diagram of the display position of the virtual image displayed when the vehicle with height compensation is driving on an upward jittery road surface provided by the present disclosure.

[0030] Figure 13It is the third flowchart of the display control method provided by the present disclosure.

[0031] Figure 14 It is a schematic diagram of the composition of a display control device provided by the present disclosure.

[0032] Figure 15 It is a schematic diagram of the structure of a projection device provided by the present disclosure. Detailed implementation manners

[0033] Next, the technical solutions in the present disclosure will be clearly and completely described in conjunction with the accompanying drawings in the present disclosure.

[0034] Refer to Figure 1 , which shows an example of an in-vehicle system 100 applicable to the technical solutions of the present disclosure. In some examples, the vehicle equipped with the system 100 can be an internal combustion engine vehicle powered by an engine, a hybrid vehicle powered by an engine and an electric motor, an electric vehicle powered by an electric motor, and other types of vehicles. In the subsequent content of this specification, the vehicle equipped with the in-vehicle system 100 is referred to as the present vehicle.

[0035] As Figure 1 shown, the in-vehicle system 100 includes: a navigation subsystem 110, an environmental detection device group 120 for obtaining the environment where the vehicle is located during vehicle driving, a vehicle driving state detection device group 130, a data processing unit 140, a display control unit 150, a display unit 160, a traveling system 180, and a suspension system 190. The above components or device groups are coupled together through a communication bus 12. In some examples, the communication bus 12 is used for connection communication between the above components or device groups. It should be noted that Figure 1 only a part of the in-vehicle system 100 is shown, rather than all of the components of the in-vehicle system 100.

[0036] In Figure 1 , the navigation subsystem 110 includes: a positioning device 111 and a map information storage device 112. Among them, the positioning device 111 can locate the position of the present vehicle based on positioning systems such as the global positioning system (GPS), China's Beidou system, Russia's GLONASS system, Europe's Galileo system, Japan's Quasi-Zenith Satellite System (QZSS), and India's Indian Regional Navigation Satellite System (IRNSS) to obtain the position information of the present vehicle. The map information storage device 112 stores map information, can obtain a navigation path leading to the destination according to the position information obtained from the positioning device 111, and display the position information and the navigation path in a map application.

[0037] In Figure 1 Figure 1 , the environmental detection device group 120 may include a vehicle-mounted communication device 121, a radar 122, a laser rangefinder 123, a camera 124, and a light sensor 125. These devices can acquire environmental information representing the inside or outside of the vehicle.

[0038] The vehicle-mounted communication device 121 can wirelessly communicate with one or more devices directly or via a communication network. These devices that can communicate with the vehicle-mounted communication device 121 can be other vehicles, roadside units or roadside platforms, or mobile terminal devices used by the vehicle occupants inside the vehicle, etc. In some examples, the vehicle-mounted communication device 121 can use 3G cellular communication, such as code division multiple access (CDMA), EVDO, global system for mobile communications (GSM) / general packet radio service (GPRS), or 4G cellular communication, such as long term evolution (LTE), or 5G cellular communication. In some examples, the vehicle-mounted communication device 121 can also communicate with a wireless local area network (WLAN) using WiFi. In some embodiments, the vehicle-mounted communication device 121 can also directly communicate with devices using an infrared link, Bluetooth, or ZigBee. In some examples, the vehicle-mounted communication device 121 can also communicate with devices using other wireless protocols.

[0039] The radar 122 is used to sense objects in the surrounding environment of the vehicle and can also be used to sense the speed and / or forward direction of these objects. In some examples, the radar 122 can use electromagnetic waves or lasers as a medium and detect objects based on the time of flight (TOF) method or the phase shift method, and detect the position of the detected object, the distance to the detected object, and the relative speed. In some examples, in order to be able to detect objects located in front of, behind, or to the side of the vehicle, the radar 122 can be configured at an appropriate position outside the vehicle.

[0040] The laser rangefinder 123 can use lasers to sense objects in the environment where the vehicle is located. In some embodiments, the laser rangefinder 123 may include one or more laser sources, a laser scanner, and one or more detectors, as well as other system components.

[0041] The camera 124 can be used to capture multiple images of the surrounding environment of the vehicle. The camera 124 can be a static camera or a video camera. In some examples, in order to obtain an external image of the vehicle, the camera 124 can be located at an appropriate position outside the vehicle. For example, in order to obtain an image in front of the vehicle, the camera 124 can be arranged close to the front windshield inside the vehicle. Alternatively, the camera 124 can be arranged around the front bumper or radiator grille. In some examples, in order to obtain an image behind the vehicle, the camera 124 can be arranged close to the rear window glass inside the vehicle. Alternatively, the camera 124 can be arranged around the rear bumper, trunk or tailgate. In some examples, in order to obtain an image on the side of the vehicle, the camera 124 can be arranged close to at least one of the side windows inside the vehicle. Alternatively, the camera 124 can be arranged around the side mirror, fender or door. In some examples, in order to obtain a foreground image of the vehicle that is the same as the driver's field of view, the camera 124 can be around the steering wheel.

[0042] The light sensor 125 can be used to detect the ambient illuminance of the vehicle. Specifically, it can be a sensor that only has the function of detecting ambient illuminance, or it can be a sensor that not only has the function of detecting ambient illuminance but also has other functions (such as a rain and light sensor). In some examples, in order to detect the illuminance of the ambient light inside the vehicle for controlling the interior lighting, the light sensor 125 can be arranged at a position close to the reading lamp on the roof console. In some examples, in order to detect the illuminance of the ambient light inside the vehicle for controlling the brightness of the interior lighting and the dashboard backlight, the light sensor 125 can be behind or inside the dashboard. In some examples, in order to detect the ambient light entering the vehicle interior, the light sensor 125 can be arranged on the inner side of the front windshield, close to the rearview mirror. In some examples, in order to detect the ambient light entering the vehicle interior for determining the degree of influence on the driver, the light sensor 125 can be arranged inside or outside the steering wheel. In some examples, in order to detect the illuminance of the ambient light outside the vehicle for automatically adjusting the anti-glare function of the rearview mirror, the light sensor 125 can be arranged on the back or near the rearview mirror. In order to detect the illuminance of the ambient light outside the vehicle for automatically controlling the headlights, the light sensor 125 can be arranged near the front bumper or front grille. In order to detect the illuminance of the ambient light outside the vehicle, avoid the interference of direct light or reflected light, and provide a more accurate light reading, the light sensor 125 can be arranged on the side or corner of the vehicle. In some advanced driver assistance systems, the light sensor 125 may be integrated with a front view camera or other sensors to provide ambient light information.

[0043] In Figure 1Among them, the vehicle driving state detection device group 130 may include: a steering angle sensor 131 for detecting the steering angle of the vehicle itself, a vehicle speed sensor 132 for detecting the driving speed of the vehicle itself, and an acceleration sensor 133 for detecting the acceleration applied to the vehicle itself. In some examples, as shown by the dashed box, it may further include an inertial sensor 134 for detecting the position and orientation changes of the vehicle itself based on inertial acceleration. In a specific implementation process, the inertial sensor 134 may be a combination of the acceleration sensor 133 and a gyroscope.

[0044] In Figure 1 Among them, the data processing unit 140 may be implemented as a computing system having a memory, a processor, an input / output interface, and a bus connecting these components. In some examples, the data processing unit 140 causes the processor to execute multiple commands through program instructions stored in the memory to process the data obtained by the navigation subsystem 110, the environment detection device group 120, and the vehicle driving state detection device group 130. In some examples, the data processing unit 140 may also control the driving of the vehicle itself based on some or all of the processed data.

[0045] In Figure 1 Among them, as shown by the dashed box, the display control unit 150 and the display unit 160 may be the main body of a Head Up Display (HUD) device 170. The display control unit 150 may, after receiving the data processed by the data processing unit 140, or after receiving the data obtained by the navigation subsystem 110, the environment detection device group 120, and the vehicle driving state detection device group 130, process the received data to obtain a display setting that needs to be displayed, and project the display setting onto the windshield of the vehicle itself through the display unit 160 for display.

[0046] In Figure 1 Among them, the propulsion system 180 may include an engine 181, an energy source 182, a transmission 183, and wheels / tires 184. The engine 181 may be an internal combustion engine, an electric motor, an air compression engine, or other types of engine combinations, such as a hybrid engine composed of a gasoline engine and an electric motor, or a hybrid engine composed of an internal combustion engine and an air compression engine. The engine 181 converts the energy source 182 into mechanical energy. Examples of the energy source 182 include gasoline, diesel, other petroleum-based fuels, propane, other compressed gas-based fuels, ethanol, solar panels, batteries, and other power sources. The energy source 182 may also supply energy to other systems of the driving device 100. The transmission 183 may transmit the mechanical power from the engine 181 to the wheels 184. The transmission 183 may include a gearbox, a differential, and a drive shaft. In one embodiment, the transmission 183 may further include other components, such as a clutch. Among them, the drive shaft may include one or more shafts that can be coupled to one or more wheels 184.

[0047] In Figure 1 , the suspension system 190 connects the wheels 184 to the vehicle body and is a mechanical system that absorbs shocks caused by road irregularities, maintains vehicle stability and comfort. The main components of the suspension system may include: a spring for supporting the weight of the vehicle, absorbing and reducing shocks caused by road irregularities; a shock absorber for controlling the vibration of the spring and quickly calming the spring's rebound; upper and lower arms, which are linkages for connecting the wheels 184 to the vehicle body, transmitting force and torque; a steering knuckle, which is a fixed point of the wheel 184, connecting the wheel 184 and the suspension system 190 and allowing the wheel 184 to rotate; a stabilizer bar for reducing the roll of the vehicle during turning; an axle for connecting the left and right wheels, transmitting power and supporting the wheels; ball joints for allowing the linkages of the suspension system 190 to move in multiple directions; suspension bushings for reducing friction and vibration between the components of the suspension system 190; and suspension brackets, which are components for connecting the suspension system 190 and the vehicle body, absorbing and isolating vibrations.

[0048] Combined with Figure 2 the exemplary top view of the present vehicle shown in Figure 3 and the exemplary perspective view from the driver's seat of the present vehicle shown in

[0049] In Figure 3 , the windshield 204 is visually located above the vehicle instrument panel 206. The driver can turn the steering wheel 210 in the passenger compartment 208 to steer the vehicle, such as changing lanes, merging, and parking the vehicle. In some embodiments, the steering wheel 210 can be retracted or omitted.

[0050] The head-up display device 170 (see Figure 4 ) projects display images (such as a vehicle speed icon 301, a speed limit icon 302, navigation information 303, a pedestrian warning icon 304, etc., and the pedestrian warning icon 304 fits the actual pedestrian) onto a part of the windshield 204 through one or more holes in the instrument panel 206. Although Figure 3 only a partial display image is shown, more information can be displayed on the windshield 204, such as the current gear of the vehicle transmission, engine speed, direction of the vehicle, current infotainment system settings, and / or other vehicle information. The head-up display device 170 provides information to the vehicle driver without the driver having to shift their line of sight away from the objects in front of the vehicle.

[0051] See Figure 4Exemplary implementation architecture of the head-up display device 170 shown. The display control unit 150 generates a signal 412 based on the data processed by the data processing unit 140, or the data 420 transmitted by the navigation subsystem 110, the environmental detection device group 120, and the vehicle driving state detection device group 130. The display unit 160 may include: a light source 161 and an optical path component 162. The light source 161 outputs light based on the signal 412 from the display control unit 150 to be displayed on the windshield 204. For example, the light source 161 may include one or more lasers and output red, green, and blue light.

[0052] The optical path component 162 can reflect the output of the light source 161 to the windshield 204 through the hole 216. A viewer (e.g., a driver) can view the display image 212 in the display area where the display image 212 is projected onto the windshield 204. In some examples, the optical path component 162 may include one or more mirrors (plane mirrors) and concave mirrors (magnifying glasses). The output of the light source 161 is folded back by the mirror and magnified by the concave mirror and then reflected to the windshield 204 to form a virtual image 40 that can be visually observed by the driver. The visual effect presented by the virtual image 40 is that the virtual image 40 is projected onto a projection plane 41 at a set distance in front of the vehicle, but passes through the projection plane 41 and the real environment remains visible. In some examples, the optical path component 162 can also be omitted, and the light source 161 can directly project the display image 212 onto the windshield 204 to form a virtual image 40 on the projection plane 41.

[0053] Combined with the foregoing Figure 3 shown, taking the images displayed in the windshield 204 including: a speed limit icon and a pedestrian warning icon as an example. In the world coordinate system, as Figure 5a shown, when driving on a smooth road surface, the windshield 204 is located at position 1, the speed limit icon displayed in the windshield 204 is located at position 11, and the pedestrian warning icon is located at position 21. As Figure 5b shown, when the vehicle passes over the raised obstacle 501, the windshield 204 shakes upward from position 1 to position 2, the speed limit icon displayed in the windshield 204 moves up from position 11 to position 21, and the pedestrian warning icon moves up from position 12 to position 22. As Figure 5c shown, when the vehicle passes over the sunken road surface 502, the windshield 204 shakes upward from position 1 to position 3, the speed limit icon displayed in the windshield 204 moves down from position 11 to position 31, and the pedestrian warning icon moves down from position 12 to position 32. Therefore, when the vehicle is driving on an uneven road surface, the up-and-down shaking of the vehicle causes the display image to shake up and down. Since the driver's shaking is not in the same frequency as the vehicle's shaking, the image displayed in the HUD seen by the driver is blurred due to the shaking, which makes the driver feel dizzy and affects safe driving.

[0054] Based on the above description, the present disclosure expects to provide a display control method that can keep the position of the displayed image from moving with jitter when the vehicle passes over an uneven road surface. As Figure 6 shown, it shows an example of a display control method provided by the present disclosure. This method can be executed by the aforementioned head-up display device 170, and in particular, can be executed by the display control unit 150 in the aforementioned head-up display device 170. Figure 6 The method shown includes steps S601 to S603.

[0055] In step S601, dynamic information of the vehicle itself is acquired.

[0056] Among them, the dynamic information includes: road condition information during the driving of the vehicle itself (such as: the flatness of the road surface, whether there are obstacles, the type of obstacles, the size of obstacles, etc.); the dynamic information may also include: driving information during the driving of the vehicle itself (such as vehicle speed, acceleration, etc.); the dynamic information may further include: vehicle configuration parameters (such as the elastic coefficient of the suspension system, the elastic coefficient of the driver's seat, the distance between the driver's seat and the wheels in the longitudinal axis direction in the vehicle coordinate system, etc.). Figure 7 shows the vehicle coordinate system. The longitudinal axis X is the driving direction of the vehicle itself, the transverse axis Y is perpendicular to the X axis in the horizontal direction, and the vertical axis Z is perpendicular to the plane formed by the X axis and the Y axis.

[0057] In step S602, when it is predicted that the vehicle itself jitters, based on the dynamic information, the display settings of the projection device are determined.

[0058] Among them, the display settings include: the display height of the display image projected by the projection device, the refresh frequency of the projection device, etc. The display height is used to indicate the specified height at which the image is projected on the windshield, and the refresh frequency of the projection device is used to indicate the update frequency of the display image, that is, the number of times the content displayed by the projection device is updated per second.

[0059] During the driving of the vehicle, sensors for collecting road condition information will collect road condition information within a preset range in front of the vehicle, or according to navigation information, road condition information at a certain distance from the vehicle that has not been traveled can also be determined. Among the pre-collected road condition information of the road section in front of the vehicle that has not been traveled, there are obstacles, such as: speed bumps, gravel roads, depressions, bricks and other protrusions. When the vehicle runs over these obstacles, it will cause the vehicle to jitter, that is, it is predicted that the vehicle jitters.

[0060] In some implementable ways, sensors for collecting road condition information near the wheels (such as ultrasonic sensors, radar sensors, etc.) collect road condition information. If it is determined that there is an obstacle within a preset distance ahead, it can be determined that the vehicle is about to vibrate. By reasonably setting the position of the sensor (such as the set position can exactly collect the road condition information within a preset distance directly in front of the wheel), the corresponding driving duration of the preset distance is relatively short (such as 1 s), and the probability of the wheel turning is small. Therefore, the accuracy of predicting possible vibration is relatively high.

[0061] In some implementable ways, to further improve the accuracy of vibration prediction. The road condition information in front of the vehicle can be collected by a camera to predict whether the vehicle is about to vibrate. Specifically, analyze the pictures of the road condition information within a preset range of the vehicle. When it is determined that there is an obstacle ahead, determine whether the vehicle will run over the obstacle according to the driving route of the vehicle. Specifically, the driving route of the vehicle can be determined according to the steering angle of the wheel, the orientation angle of the vehicle head, the steering angle of the steering wheel, etc.

[0062] Exemplarily, the driving route is determined based on the steering angle of the wheel, and then it is predicted whether the wheel will run over the obstacle. As Figure 8 shown, it is an image of the obstacle collected. The image includes the obstacle 801. Analyze the image to determine the edge points P1 to P4 of the obstacle 801 in the image coordinate system. The four points are only examples, and more edge points can be analyzed. The edge points at least include the point with the largest abscissa and the smallest abscissa, the point with the largest ordinate and the smallest ordinate. Convert P1 to P4 in the image coordinate system to points M1 to M4 in the vehicle coordinate system. As Figure 9a shown, determine the obstacle coverage area 901 according to points M1 to M4. The driving route determined according to the steering angle of the wheel relative to the X-axis direction is as shown in the figure. If the driving route of the right wheel overlaps with the obstacle coverage area 901, it is predicted that the vehicle is about to vibrate. As Figure 9b shown, when the wheel rotates a certain angle, the driving route determined according to the steering angle of the wheel relative to the X-axis direction does not overlap with the obstacle coverage area 901, and it is predicted that the vehicle will not vibrate.

[0063] It should be noted that the obtained dynamic information is the dynamic information when the vehicle vibrates. That is, the road condition information is the information of the actual vibrating road surface. However, since the time interval between predicting that the vehicle will vibrate and the vehicle actually vibrating is relatively short, generally at the second level, the current driving information can be used as the driving information when vibrating. The configuration parameters of the vehicle are fixed, and the configuration parameters of the vehicle when vibrating are the current configuration parameters of the vehicle.

[0064] In step S603, when the vehicle vibrates, according to the display settings, the projection device is controlled to display an image such that the jitter displacement of the image displayed by the projection device is less than the displacement threshold.

[0065] Combined with Figure 5b , when the vehicle vibrates upward, the display image of the projection device is controlled to move downward. As Figure 10 shown, the windshield 204 vibrates upward to position 2. According to the display settings, the speed limit icon is located at position 11 and the pedestrian warning icon is located at position 12. That is, as the vehicle body vibrates, the image displayed by the projection device remains fixed relative to the driver's line of sight, or the change value is so small that the human eye cannot distinguish it. Sensually, it is felt that the displayed image does not vibrate as the vehicle vibrates. In this way, on a bumpy road, it does not affect the driver's acquisition of information from the projection device.

[0066] In some embodiments of the present disclosure, combined with Figure 6 , as Figure 11 shown, the display settings include: compensation height; the dynamic information includes: road condition information; in the above step S602, when it is predicted that the vehicle vibrates, based on the dynamic information, the display settings of the projection device are determined, which can be specifically implemented through the following steps S602a and S602b.

[0067] In step S602a, when it is predicted that the vehicle vibrates, based on the road condition information, the first vibration amplitude of the left wheel and the second vibration amplitude of the right wheel are determined.

[0068] Among them, the left wheel and the right wheel are a set of wheels, which can be the left front wheel and the right front wheel, or the left rear wheel and the right rear wheel. The vibration amplitude refers to the distance moved upward or downward when the vehicle passes over an obstacle.

[0069] In step S602b, according to the first vibration amplitude and the second vibration amplitude, the compensation height of the image is determined as the first compensation height.

[0070] The result of weighted summation of the first vibration amplitude and the second vibration amplitude is used as the first compensation height. That is, when the vehicle vibrates, the windshield moves up or down a certain distance. By moving the image (i.e., the virtual image) displayed by the projection device in the opposite direction of the vibration by the first compensation height, the position of the displayed image in the world coordinate system remains unchanged relative to that before the vibration.

[0071] In one case, if the left wheel and the right wheel pass over the same obstacle simultaneously, the weights of the left wheel and the right wheel are the same. For example, it is predicted that the vehicle is about to pass over a speed bump. The left wheel and the right wheel pass over the speed bump at the same time. The height of the speed bump is h1, and the weights are both 0.5. Then the result of weighted summation is h1. Or, it is predicted that the left wheel and the right wheel pass over the same depression at the same time. The height of the depression is -h2, and the weights are both 0.5. Then the result of weighted summation is -h2.

[0072] In another case, the left wheel and the right wheel pass over different obstacles simultaneously, and the directions of the jitters caused by the obstacles are the same, either upward or downward; or one wheel passes over an obstacle while the other wheel does not. At this time, the weights of the left wheel and the right wheel can be set to be the same; however, since both the driver and the displayed image are on the left side of the vehicle, the lifting or lowering of the left wheel has a greater impact on the image jitter than that of the right wheel. The weight of the left wheel can also be set to be greater than the weight of the right wheel. For example, on a gravel road, it is predicted that the height of the small gravel that the left wheel is about to pass over is h3. At the same time when the left wheel passes over the small gravel, the height of the large gravel that the right wheel passes over is h4, and h3 is less than h4. The weight of the left wheel is set to 0.7, and the weight of the right wheel is set to 0.3. Then the result of weighted summation is (0.7h3 + 0.3h4).

[0073] In still another case, the left wheel and the right wheel pass over different obstacles simultaneously, and the directions of the jitters caused by the obstacles are different, one upward and the other downward. At this time, the weights of the left wheel and the right wheel can be set to be the same. For the same reason as in the above-mentioned another case, the weight of the left wheel can also be set to be greater than the weight of the right wheel. For example, it is predicted that the height of the gravel that the left wheel is about to pass over is h5. At the same time when the left wheel passes over the gravel, the height of the depression that the right wheel falls into is -h6. The weights of the left wheel and the right wheel are both set to 1. Then the result of weighted summation is (h5 - h6).

[0074] In summary, different weights can be set for weighted summation according to the actual situation, so that the position of the finally obtained adjusted displayed image is fixed and will not jitter with the jitter of the vehicle.

[0075] It should be noted that after compensating the height of the displayed image according to the first compensation height, the height of the virtual image is compensated here, and correspondingly, the height of the image displayed on the windshield also changes. As Figure 12a shown, it is a schematic diagram of the display height of the virtual image 1201 on a non-jittery road for example. In the world coordinate system, the virtual image 1201 is displayed at the position of height 1. As Figure 12bAs shown, as the vehicle jitters when passing over the obstacle 501, if no adjustment is made, the virtual image 1201 will move upward and be displayed at the position of height 2. After adjustment, the virtual image 1201 remains displayed at the position of height 1.

[0076] During the driving process of the vehicle, the degree of jitter is related to internal factors of the vehicle such as the speed, acceleration, elastic coefficient of the suspension system, and elastic coefficient of the wheels, in addition to the influence of road condition information, i.e., external factors. The greater the speed or acceleration, the greater the amplitude of jitter caused by the vehicle passing over the obstacle; the better the elastic coefficient, the smaller the amplitude of jitter caused by the vehicle passing over the obstacle.

[0077] Therefore, in order to more accurately determine the position where the image is to be displayed, optionally, the dynamic information further includes: vehicle information. Wherein, the vehicle information is the factor of the vehicle itself that affects the amplitude of jitter of the vehicle when passing over the obstacle. For example, the vehicle information includes at least one of the following: speed, acceleration, elastic coefficient of the suspension system, and elastic coefficient of the wheels; the method further includes the following steps S602c and S602d.

[0078] In step S602c, according to the vehicle information, a correction coefficient is determined.

[0079] In step S602d, according to the correction coefficient, the first compensation height is corrected, and the obtained second compensation height is determined as the compensation height of the image.

[0080] Among them, the result of weighted summation only considers the external road condition information to determine the amplitude of jitter of the vehicle when passing over the obstacle. However, due to the shock absorption effect of the vehicle's suspension system and tires, the amplitude of jitter of the vehicle will be smaller than the actually determined result of weighted summation. Moreover, due to the different speeds and accelerations of the vehicle, when passing over the same obstacle, the amplitude of jitter is also different. Based on this, in order to make the position of the displayed image more accurate, the vehicle information is comprehensively considered to determine a correction coefficient for correcting the result of weighted summation, so that the obtained second compensation height after correction is more accurate.

[0081] In some implementable ways, according to the vehicle information, a correction coefficient is determined. Specifically, before the vehicle leaves the factory, different vehicle information is configured for testing, and the corresponding correction coefficients under different configurations are determined, and the corresponding relationship between the vehicle information and the correction coefficients is stored. For the un-stored vehicle information, the corresponding correction coefficient can be determined by linear interpolation.

[0082] In some other implementable ways, the correction coefficient is predicted by a correction coefficient prediction model. A model pre-trained with different vehicle information and corresponding correction coefficients is learned to obtain a correction coefficient prediction model capable of predicting the correction coefficient according to the vehicle information, and this correction coefficient prediction model is set in the vehicle.

[0083] Optionally, the display settings include: a compensation height. In the case where the vehicle vibrates, according to the display settings, the projection device is controlled to display an image, which can be specifically implemented through the following step S603a.

[0084] In step S603a, in the case where the vehicle vibrates, according to the compensation height of the image, the display position of the image is controlled.

[0085] When the vehicle is driving on a non-bumpy road, the position of the usually displayed image does not change, and each frame is displayed at a fixed position. If the vehicle vibrates, as the vehicle vibrates, the windshield will also vibrate accordingly. Correspondingly, the position of the image displayed on the windshield is fixed relative to the windshield and will also vibrate. Therefore, in the case where the vehicle vibrates, a frame of the image to be displayed is moved by the compensation height on the basis of the display height of the previous frame of the image. The compensation height can be positive (raising the display position of the image) or negative (lowering the display position of the image), and the frame of the image to be displayed is displayed according to the compensated height.

[0086] During the process of changing the display position of the image, if the distance to be moved is large, for example: the display height of the previous frame of the image is 10 cm, and the display height of the next frame of the image is 12 cm; in this case, image ghosting and afterimages may occur, resulting in an unclear displayed image.

[0087] To avoid image ghosting and afterimages, the refresh frequency of the projection device can be changed so that the image is translated from one position to another through multiple frames to achieve smooth transition of the image. Therefore, optionally, the display settings further include: a target refresh frequency; as Figure 13 shown, in the case where the vehicle vibrates, according to the compensation height, the above step S603a controls the display position of the displayed image, which can be specifically implemented through the following step S603b.

[0088] In step S603b, in the case where the vehicle vibrates, during the process of controlling the display position of the image according to the compensation height of the image, the refresh frequency of the projection device is controlled to be the target refresh frequency.

[0089] The refresh frequency of the projection device refers to the number of times the projection device updates the displayed content per unit time. A high refresh rate can provide a smoother display effect, reducing screen flicker and latency.

[0090] In some feasible ways, according to the processing capabilities of different processors, a fixed target refresh frequency is set when the vehicle jitters. For example, on a non-bumpy road surface, the refresh frequency of the HUD is the first refresh frequency, and when jitter occurs, the refresh frequency of the HUD is changed to the second refresh frequency; wherein, the first refresh frequency is less than the second refresh frequency.

[0091] In another feasible way, when the vehicle jitters, the target refresh frequency is determined according to the compensation height. Specifically, when the compensation height belongs to the first range, the target refresh frequency is the first refresh frequency; when the compensation height belongs to the second range, the target refresh frequency is the second refresh frequency; when the compensation height belongs to the third range, the target refresh frequency is the third refresh frequency. Wherein, any value in the first range is less than any value in the second range, any value in the second range is less than any value in the third range, the first refresh frequency is less than the second refresh frequency, and the second refresh frequency is less than the third refresh frequency. It should be noted that the first range, the second range, and the third range are only examples and do not limit the present disclosure. In practical applications, more or fewer ranges can be set as needed. The principle of setting is: when the compensation height is small, the refresh frequency is low, and when the compensation height is large, the refresh frequency is high. The specific boundaries are set according to different actual vehicles.

[0092] In still other embodiments, when the vehicle jitters, the target refresh frequency is determined according to the occupancy rate of the processor. Specifically, when the occupancy rate of the processor is less than or equal to the first occupancy rate threshold, the target refresh frequency is the fourth refresh frequency; when the occupancy rate of the processor is greater than the first occupancy rate threshold and less than or equal to the second occupancy rate threshold, the target refresh frequency is the fifth refresh frequency; when the occupancy rate of the processor is greater than the second occupancy rate threshold, the target refresh frequency is the sixth refresh frequency. Wherein, the first occupancy rate threshold is less than the second occupancy rate threshold, the second occupancy rate threshold is less than the third occupancy rate threshold, the fourth refresh frequency is less than the fifth refresh frequency, and the fifth refresh frequency is less than the sixth refresh frequency. It should be noted that the first occupancy rate threshold, the second occupancy rate threshold, and the third occupancy rate threshold are only examples and do not limit the present disclosure. In practical applications, finer or coarser divisions can be set as needed. The principle of setting is: when the occupancy rate is small, the refresh frequency is high, and when the occupancy rate is large, the refresh frequency is low. The specific boundaries are set according to different actual vehicles.

[0093] Furthermore, when the vehicle jitters, the occupancy rate of the processor and the compensation height can also be comprehensively considered. The determination of the specific refresh frequency refers to the above strategies for occupancy rate and compensation height, which will not be elaborated here.

[0094] In some embodiments of the present disclosure, the dynamic information includes: road condition information; in step S602 above, when it is predicted that the vehicle shakes, based on the dynamic information, the display settings of the projection device are determined, which can be specifically implemented through the following step S602e and step S602f.

[0095] In step S602e, when it is predicted that the vehicle shakes, based on the road condition information, it is determined whether a target road condition identifier corresponding to the road condition information is stored in the vehicle.

[0096] In step S602f, when the target road condition identifier is stored, based on the target road condition identifier, the display settings of the projection device are determined.

[0097] Optionally, during the driving process of the driver, there is usually a historical road section that has been traveled. The corresponding relationship between the road section identifier of the historical road section and the display settings can be stored. The specific storage strategy can be based on the number of times the vehicle travels on a road section. When the number of travel times is greater than the threshold number of times, the corresponding relationship between the road section identifier of the road section and the display settings is stored, or the driver stores it manually.

[0098] The road section identifier (such as an image of the road section) is used to uniquely identify a road section. When it is predicted that the vehicle is about to shake, first, it is determined whether the road section identifier of the road section where the vehicle is about to shake is stored in the vehicle. If so, when the vehicle shakes, the HUD is controlled to be displayed according to the display settings corresponding to the road section identifier of the road section; if not, the scene type corresponding to the road section is analyzed and determined, and the display settings of the HUD are determined according to the scene type.

[0099] For the corresponding relationship between the stored road section identifier and the display settings, the driver is allowed to adjust it as needed. For example, when passing through a shaking road section, after compensating the height of the image, if the driver still feels the image shakes, the driver can manually adjust the compensation height, and use the display settings obtained by the adjustment where the driver feels no shaking as the display settings of the HUD corresponding to the road section.

[0100] Exemplarily, when the driver is on a certain jitter section of the fixed route from the company to home, it is found that the displayed image jitters up and down with the jitter of the vehicle. One reason for this jitter is insufficient height compensation of the displayed image, and the other reason is that the height compensation of the displayed image exceeds the limit (for example, it should be compensated upward by 1 cm, but actually compensated upward by 1.5 cm). The driver determines whether the existing height compensation is insufficient or exceeds the limit based on whether the jitter of the displayed image during the vehicle jitter is in the same direction as the vehicle (insufficient compensation) or in the opposite direction (excessive compensation). If the compensation is insufficient, increase the current compensation height; if the compensation exceeds the limit, decrease the current compensation height until the displayed image of the HUD no longer jitters when the vehicle passes through this jitter section. Mark this section (such as the image of this section) and the corresponding display settings and store them in the vehicle for direct use of the display settings when driving to this section again.

[0101] Optionally, in the case where the target road condition identifier is not stored, based on the road condition information, determine the scene type; then, according to the scene type, determine the display settings of the projection device. The scene type is used to indicate the jitter situation and historical road conditions when the vehicle passes through the road surface.

[0102] Exemplarily, classified by jitter frequency and jitter amplitude, the scene types include: high frequency and high amplitude, high frequency and low amplitude, low frequency and high amplitude, and low frequency and low amplitude. This scene classification is only for illustrative purposes, and the actual granularity that can be divided can be finer or coarser. Moreover, the specific ranges for determining the high and low frequencies and the high and low jitter amplitudes can be determined according to the actual situation.

[0103] Collect the road condition information in front of the vehicle through sensors, analyze the collected road condition information to determine the corresponding scene type, and different display settings corresponding to different scene types are pre-stored in the vehicle, so as to determine the corresponding display settings according to the scene type. This method of determining the display settings according to the scene type requires lower computing power for the vehicle.

[0104] Since the jitter of the HUD image caused by vehicle jitter is affected by many factors and is not a simple linear relationship, therefore, the powerful learning ability of the neural network model can be utilized to solve the non-linear problem and predict the display settings of the HUD during the jitter process. In some embodiments of the present disclosure, in step S602, when it is predicted that the vehicle jitters, based on the dynamic information, determine the display settings of the head-up display device HUD, which can be specifically implemented through the following step S602h.

[0105] In step S602h, when it is predicted that the vehicle jitters, input the dynamic information into the display setting prediction model to obtain the display settings.

[0106] In an exemplary implementation manner of the present disclosure, the above display setting prediction model is obtained through training. In this exemplary implementation manner, the initial model may be a convolutional neural network model, an object detection convolutional neural network model, a recurrent neural network model, a generative adversarial network model, but is not limited thereto, and other neural network models well-known to those skilled in the art may also be adopted. No specific limitation is made in this exemplary implementation manner.

[0107] The display setting prediction model is mainly a neural network model based on deep learning. For example, the display setting prediction model may be based on a feedforward neural network. A feedforward network can be implemented as an acyclic graph, where nodes are arranged in layers. Generally, the feedforward network topology includes an input layer and an output layer, and the input layer and the output layer are separated by at least one hidden layer. The hidden layer transforms the input received by the input layer into a representation useful for generating an output in the output layer. The network nodes are fully connected via edges to the nodes in the adjacent layer, but there are no edges between the nodes within each layer. The data received at the nodes of the input layer of the feedforward network is propagated (i.e., "fed forward") to the nodes of the output layer via an activation function, and the activation function calculates the state of the nodes in each successive layer of the network based on coefficients ("weights"), and the coefficients are respectively associated with each of the edges connecting these layers. The output of the display setting prediction model can take various forms, and the present disclosure does not limit this. The display setting prediction model may also include other neural network models, for example, a convolutional neural network (CNN) model, a recurrent neural network (RNN) model, a generative adversarial network (GAN) model, but is not limited thereto, and other neural network models well-known to those skilled in the art may also be adopted.

[0108] The display setting prediction model is obtained by training a pre-trained model, and specifically may include the following steps: obtaining a pre-trained neural network model; constructing a training sample set, and each training sample includes: dynamic information and display settings; training the pre-trained neural network model with the training samples until the value of the loss function meets a preset condition, and then obtaining the display setting prediction model. For example, during the supervised learning training process for a neural network, the output generated by the network in response to an input representing an instance in the training sample set is compared with the "correct" labeled output of the instance; an error signal representing the difference between the output and the labeled output is calculated; and when the error signal is propagated backward through the layers of the network, the weights associated with the connections are adjusted to minimize the error. The model when the error of each output generated from the instances in the training data set is minimized is defined as the display setting prediction model.

[0109] Optionally, the loss function used in the model training process includes a first parameter for measuring the display effect of the compensated display image, a second parameter for measuring the display image quality, and a third parameter for measuring the resource consumption.

[0110] To measure the display effect of the compensated display image, multiple consecutive frames of images during the jitter process can be collected by a camera, and the display effect can be measured according to the position change amount of the multiple frames of images during the jitter process. The larger the position change amount, the worse the display effect, otherwise the better. For example, the mean value, variance, etc. of the position change amount of each frame of image are taken, and the obtained values are used to measure the quality of the display effect.

[0111] During the model training process, in addition to the need for the compensated height predicted by the model to be accurate, the model also needs to be able to predict the refresh frequency of the HUD during the compensation process, and the refresh frequency is not the higher the better. The resource consumption of the processor also needs to be comprehensively considered. Therefore, the loss function also includes a second parameter for measuring the quality of the display image and a third parameter for measuring the resource consumption.

[0112] To measure the quality of the display image, multiple consecutive frames of images during the compensation process can be collected by a camera. For the target elements in an image frame, determine the change in their pixel positions in the consecutive images. If the pixel position changes of the target elements in adjacent frames are not continuous, it is determined that the display effect is poor. Specifically, it includes the following steps: Denoise and sharpen the image to reduce the influence of noise on the analysis and enhance the image edges, making the trailing easier to detect; Register the consecutive image sequences to ensure that the same elements or feature points in each frame of image are in the same position; Select some feature points in the first frame, and then track the movement of these points in the consecutive frames; For the tracked feature points, calculate their displacement vectors in the consecutive frames, and the displacement vectors represent the movement of the feature points on the image plane; Predict the positions of the feature points in the next frame; For each frame, calculate the difference between the actually observed position of the feature points and the predicted position, that is, the residual; Analyze the magnitude and distribution of the residuals. If the residuals are large or concentrated in certain directions, it may indicate the existence of trailing; Conduct statistical analysis on the residuals, such as calculating the mean value, variance, standard deviation, etc., to evaluate the degree of trailing. Usually, professional image analysis tools are used to analyze multiple consecutive frames of images to obtain a scoring value, and the scoring value is used to measure the display effect, which will not be elaborated here.

[0113] To measure the resource consumption, the occupancy rate of the processor during the compensation process can be statistically analyzed, and the mean value of the occupancy rate is used as the measurement value. It is also possible to statistically analyze the duration during which the occupancy rate exceeds the occupancy rate threshold, and use the total duration of a jitter process as the measurement value.

[0114] Exemplarily, during the model training process, after compensating according to the determined compensation height, if the first parameter meets the standard, but the second parameter does not meet the standard, and the third parameter meets the standard, then increase the refresh frequency of the HUD; if the first parameter meets the standard, the second parameter meets the standard, but the third parameter does not meet the standard, then decrease the refresh frequency of the HUD; if the first parameter meets the standard, but the second parameter does not meet the standard, and the third parameter does not meet the standard, then decrease the compensation height. If the first parameter does not meet the standard, directly adjust the compensation height.

[0115] During the model training process, it is also possible to train the fuzzy control weight corresponding to measuring the display image quality and the performance control weight corresponding to measuring the resource consumption. The fuzzy control weight is used to control the smoothness during the image height compensation process to prevent blurring, that is, to control the refresh frequency of the HUD so that the second parameter can meet the standard; the performance control weight is used to control the resource consumption of the processor to prevent excessive resource consumption resulting in lag, so that the third parameter can meet the standard. The larger the value of the fuzzy control weight, the smoother the image transition during the compensation process, but the more resource consumption; the larger the value of the performance control weight, the lower the resource consumption, but it may cause the image during the compensation process not to transition smoothly and appear blurred. Therefore, during the training process, these two parameters need to be trained to balance the refresh frequency and resource consumption.

[0116] Specifically, before starting the training, the fuzzy control weight and the performance control weight will have an initial value. During the training process, the fuzzy control weight and the performance control weight are dynamically adjusted. The fuzzy control weight is adjusted according to the second parameter. If the value of the second parameter is small, then increase the fuzzy control weight; the performance control weight is adjusted according to the third parameter. If the value of the third parameter is small, then increase the performance control weight.

[0117] Adapt the trained display setting prediction model to the vehicle. According to the driver's vehicle usage habits, the frequently used routes can be stored. For the routes that the vehicle has traveled multiple times, the routes and display settings can be stored. The next time it is used, there is no need to input the dynamic information into the model to obtain the output, and the stored display settings can be directly used.

[0118] In addition, if the driver feels that the display settings obtained according to the display setting prediction model cannot completely improve the jitter, the driver can also use the historical driving data in the vehicle as input to let the model re-learn. Until the value of the loss function meets the preset conditions, a display setting prediction model suitable for the driving habits of this vehicle is obtained to achieve personalized adaptation.

[0119] Based on the same inventive concept as the foregoing technical solution, refer to Figure 14 , which shows a display control device 1400 provided by the present disclosure. This device 1400 can be Figure 1 、 Figure 4The display control unit shown in , the apparatus 1400 includes: an acquisition part 1401, a determination part 1402, and an update part 1403;

[0120] The acquisition part 1401 is configured to acquire the dynamic information of the vehicle itself;

[0121] The determination part 1402 is configured to, when it is predicted that the vehicle itself shakes, determine the display settings of the projection device based on the dynamic information;

[0122] The control part 1403 is configured to, when the vehicle itself shakes, control the projection device to display an image according to the display settings, so that the shake displacement of the image displayed by the projection device is less than the displacement threshold.

[0123] In some embodiments of the present disclosure, the display settings include: a compensation height; the dynamic information includes: road condition information; the determination part 1402 is specifically configured to, when it is predicted that the vehicle itself shakes, determine a first shake amplitude of the left wheel and a second shake amplitude of the right wheel based on the road condition information; and determine the compensation height of the image as a first compensation height according to the first shake amplitude and the second shake amplitude.

[0124] In some embodiments of the present disclosure, the dynamic information further includes: vehicle information; the apparatus further includes: a correction part; the determination part 1402 is further configured to determine a correction coefficient according to the vehicle information; the correction part is configured to correct the first compensation height according to the correction coefficient, and determine the obtained second compensation height as the compensation height of the image.

[0125] In some embodiments of the present disclosure, the control part 1403 is specifically configured to, when the vehicle itself shakes, control the display position of the image according to the compensation height of the image.

[0126] In some embodiments of the present disclosure, the display settings further include: a refresh frequency; the control part 1403 is specifically configured to, when the vehicle itself shakes, during the process of controlling the display position of the image according to the compensation height of the image, control the refresh frequency of the projection device to be a target refresh frequency.

[0127] In some embodiments of the present disclosure, the dynamic information includes: road condition information; the determination part 1402 is specifically configured to, when it is predicted that the vehicle itself shakes, determine whether a target road condition identifier corresponding to the road condition information is stored in the vehicle itself based on the road condition information; and when the target road condition identifier is stored, determine the display settings of the projection device based on the target road condition identifier.

[0128] In some embodiments of the present disclosure, before determining the scene type based on the road condition information, the determining part 1402 is further configured to determine the scene type based on the road condition information when the target road condition identifier is not stored, where the scene type is used to indicate the jitter condition of the vehicle when passing through the road surface; and determine the display setting of the projection device according to the scene type.

[0129] In some embodiments of the present disclosure, the determining part 1402 is specifically configured to input the dynamic information into the display setting prediction model when it is predicted that the vehicle shakes, and obtain the display setting.

[0130] In some embodiments of the present disclosure, the device further includes: a building part, a training part; the obtaining part 1401 is further configured to obtain a pre-trained initial model; the building part is configured to build a training sample set, and each training sample includes: dynamic information and a display setting; the training part is configured to train the pre-trained initial model through the training samples until the value of the loss function meets a preset condition, and then obtain the display setting prediction model.

[0131] In some embodiments of the present disclosure, the loss function includes a first parameter for measuring the display effect of the compensated display image, a second parameter for measuring the display image quality, and a third parameter for measuring the resource consumption.

[0132] It should be noted that the display control device provided in the present disclosure can achieve the same effect as the above display control method, and details are not described herein again.

[0133] Reference Figure 15 , which shows a structural block diagram of a projection device provided by an exemplary embodiment of the present disclosure. In some examples, the projection device has a communication function and can access a wired network or a wireless network. In some examples, the projection device can receive data based on the accessed wired network or wireless network. It can be understood that the projection device undertakes the calculation and processing work of the technical solution of the present disclosure, and the present disclosure does not limit this.

[0134] As Figure 15 shown, the projection device in the present disclosure may include one or more of the following components: a processor 1510 and a memory 1520.

[0135] Optionally, the processor 1510 is connected to various parts within the entire projection device using various interfaces and circuits. By running or executing instructions, programs, code sets, or instruction sets stored in the memory 1520, and by invoking the data stored in the memory 1520, it performs various functions of the projection device and processes data. Optionally, the processor 1510 may be implemented in at least one hardware form of digital signal processing (DSP), field-programmable gate array (FPGA), or programmable logic array (PLA). The processor 1510 may integrate one or a combination of several of a central processing unit (CPU), a graphics processing unit (GPU), a neural-network processing unit (NPU), and a baseband chip, etc. Among them, the CPU mainly processes the operating system, user interface, application programs, etc.; the GPU is responsible for rendering and drawing the content to be displayed on the touch display screen; the NPU is used to implement artificial intelligence (AI) functions; the baseband chip is used to process wireless communication. It can be understood that the above baseband chip may not be integrated into the processor 1510 and may be implemented separately by a single chip.

[0136] The memory 1520 may include random access memory (RAM) and may also include read-only memory (ROM). Optionally, the memory 1520 includes a non-transitory computer-readable storage medium. The memory 1520 can be used to store instructions, programs, code, code sets, or instruction sets. The memory 1520 may include a program storage area and a data storage area. Among them, the program storage area may store instructions for implementing the operating system, instructions for at least one function (such as touch control function, sound playback function, image playback function, etc.), instructions for implementing each of the above method embodiments, etc.; the data storage area may store data created according to the use of the projection device, etc.

[0137] In addition, those skilled in the art can understand that the structure of the projection device shown in the above drawings does not limit the projection device. The projection device may include more or fewer components than shown in the drawings, or combine certain components, or have different component arrangements. For example, the projection device further includes components such as a display screen, a camera assembly, a microphone, a speaker, a radio frequency circuit, an input unit, sensors (such as an acceleration sensor, an angular velocity sensor, a light sensor, etc.), an audio circuit, a WiFi module, a power supply, a Bluetooth module, etc., which will not be elaborated here.

[0138] The present disclosure also provides a computer-readable storage medium storing at least one instruction for being executed by a processor to implement the display control method described in each of the above embodiments.

[0139] The present disclosure also provides a head-up display device including a display control unit and a display unit. The display control unit is configured to obtain dynamic information of the vehicle itself; and, in the case where it is predicted that the vehicle shakes, determine a display setting based on the dynamic information; and, in the case where the vehicle shakes, display an image according to the display setting so that the shaking displacement of the displayed image is less than a displacement threshold. The display unit is configured to display the display image on the front windshield of the vehicle.

[0140] The present disclosure also provides a vehicle including the above head-up display device.

[0141] The present disclosure also provides a computer program product including computer instructions stored in a computer-readable storage medium. A processor of a computing device reads the computer instructions from the computer-readable storage medium, and the processor executes the computer instructions so that the computing device executes to implement the display control method described in each of the above embodiments.

[0142] In addition, those skilled in the art can understand that the structure of the computing device shown in the above drawings does not limit the computing device. The computing device may include more or fewer components than shown in the drawings, or combine certain components, or have different component arrangements. For example, the computing device further includes components such as a display screen, a camera assembly, a microphone, a speaker, a radio frequency circuit, an input unit, sensors (such as an acceleration sensor, an angular velocity sensor, a light sensor, etc.), an audio circuit, a WiFi module, a power supply, a Bluetooth module, etc., which will not be elaborated here.

[0143] The present disclosure also provides a computer-readable storage medium storing at least one instruction for being executed by a processor to implement the display control method described in each of the above embodiments.

[0144] The present disclosure also provides a computer program product, which includes computer instructions stored in a computer-readable storage medium; a processor of a computing device reads the computer instructions from the computer-readable storage medium, and the processor executes the computer instructions to cause the computing device to execute and implement the display control method described in each of the above embodiments.

[0145] Those skilled in the art should be able to realize that in one or more of the above examples, the functions described in the present disclosure can be implemented by hardware, software, firmware, or any combination thereof. When implemented using software, these functions can be stored in a computer-readable medium or transmitted as one or more instructions or codes on a computer-readable medium. The computer-readable medium includes computer storage media and communication media, where the communication media includes any medium that facilitates the transmission of a computer program from one place to another. The storage media can be any available medium accessible by a general-purpose or special-purpose computer.

[0146] It should be noted that: among the technical solutions described in the present disclosure, they can be arbitrarily combined without conflict.

[0147] As described above, the above are only specific embodiments of the present invention, but the protection scope of the present invention is not limited thereto. Any person skilled in the art can easily think of changes or substitutions within the technical scope disclosed by the present invention, and all should be covered by the protection scope of the present invention. Therefore, the protection scope of the present invention should be subject to the protection scope of the claims.

Claims

1. A display control method, characterized in that: The method comprises: Get the dynamic information of the vehicle; In the case where the host vehicle is predicted to vibrate, determining a display setting of a projection device based on the dynamic information; In the case where the vehicle shakes, controlling the projection device to display an image according to the display setting so that the shaking displacement of the image displayed by the projection device is less than a displacement threshold; Wherein, the display setting includes: compensation height; the dynamic information includes: road condition information and vehicle information; In the case where the host vehicle is predicted to shake, determining the display setting of the projection device based on the dynamic information includes: In the case where it is predicted that the host vehicle shakes, determining a first shaking amplitude of the left wheel and a second shaking amplitude of the right wheel based on the road condition information; Determining, according to the first jitter amplitude and the second jitter amplitude, a compensation height of the image as a first compensation height; Determine a correction coefficient according to vehicle information, wherein the vehicle information includes at least one of the following: an elastic coefficient of a suspension system, an elastic coefficient of a wheel, an elastic coefficient of a driver's seat, and a distance between the driver's seat and the wheel in a longitudinal axis direction in a vehicle coordinate system; The first compensation height is corrected according to the correction coefficient, and the obtained second compensation height is determined as the compensation height of the image.

2. The method according to claim 1, characterized in that When the vehicle shakes, controlling the projection device to display an image according to the display setting includes: When the host vehicle shakes, the display position of the image is controlled according to the compensation height of the image.

3. The method according to claim 2, characterized in that The display setting also includes: a refresh frequency; when the vehicle shakes, controlling the display position of the image according to the compensation height of the image, including: In the case where the host vehicle shakes, in the process of controlling the display position of the image according to the compensation height of the image, the refresh frequency of the projection device is controlled to be the target refresh frequency.

4. The method according to claim 1, characterized in that: The dynamic information includes: road condition information; when the vehicle is predicted to shake, determining the display setting of the projection device based on the dynamic information includes: In the case where it is predicted that the host vehicle shakes, determining whether a target road condition identifier corresponding to the road condition information is stored in the host vehicle based on the road condition information; In the case where the target road condition identifier is stored, the display setting of the projection device is determined based on the target road condition identifier.

5. The method according to claim 4, characterized in that The method further comprises: In the case where the target road condition identifier is not stored, determining a scene type based on the road condition information, the scene type being used to indicate a shaking condition of the vehicle when passing through the road surface; According to the scene type, display settings of the projection device are determined.

6. The method according to any one of claims 1 to 5, characterized in that: In the case where the host vehicle is predicted to shake, determining the display setting of the projection device based on the dynamic information includes: In the case where it is predicted that the host vehicle shakes, the dynamic information is input into a display setting prediction model to obtain the display setting.

7. The method according to claim 6, characterized in that The method further comprises: Get the pre-trained initial model; Constructing a training sample set, each training sample includes: dynamic information and display settings; The pre-trained initial model is trained using the training samples until the value of the loss function meets a preset condition, thereby obtaining the display setting prediction model.

8. The method according to claim 7, characterized in that The loss function includes a first parameter for measuring the display effect of the compensated display image, a second parameter for measuring the display image quality, and a third parameter for measuring resource consumption.

9. A display control device, characterized in that: The device comprises: an acquisition part, a determination part, and a control part; The acquisition part is configured to acquire dynamic information of the vehicle; The determining section is configured to determine a display setting of a projection device based on the dynamic information when the host vehicle is predicted to vibrate; The control part is configured to control the projection device to display an image according to the display setting when the host vehicle shakes, so that the shaking displacement of the image displayed by the projection device is less than a displacement threshold; Wherein, the display setting includes: compensation height; the dynamic information includes: road condition information and vehicle information; The determining part is specifically configured to determine a first shaking amplitude of the left wheel and a second shaking amplitude of the right wheel based on the road condition information when it is predicted that the host vehicle shakes; Determining, according to the first jitter amplitude and the second jitter amplitude, a compensation height of the image as a first compensation height; The determining part is further configured to determine the correction coefficient according to vehicle information, wherein the vehicle information includes at least one of the following: an elastic coefficient of a suspension system, an elastic coefficient of a wheel, an elastic coefficient of a driver's seat, and a distance between the driver's seat and the wheel in a longitudinal axis direction in a vehicle coordinate system; The first compensation height is corrected according to the correction coefficient, and the obtained second compensation height is determined as the compensation height of the image.

10. A head-up display device, characterized in that: The head-up display device includes a display control unit and a display unit; wherein, The display control unit is configured to obtain dynamic information of the vehicle; and, in the event that the host vehicle is predicted to vibrate, determining a display setting based on the dynamic information; and, in the event of a shake of the host vehicle, displaying an image according to the display setting so that a shake displacement of the displayed image is less than a displacement threshold; Wherein, the display setting includes: compensation height; the dynamic information includes: road condition information and vehicle information; In the case where the host vehicle is predicted to shake, determining the display setting of the projection device based on the dynamic information includes: In the case where it is predicted that the host vehicle shakes, determining a first shaking amplitude of the left wheel and a second shaking amplitude of the right wheel based on the road condition information; Determining, according to the first jitter amplitude and the second jitter amplitude, a compensation height of the image as a first compensation height; Determine a correction coefficient according to vehicle information, wherein the vehicle information includes at least one of the following: an elastic coefficient of a suspension system, an elastic coefficient of a wheel, an elastic coefficient of a driver's seat, and a distance between the driver's seat and the wheel in a longitudinal axis direction in a vehicle coordinate system; Correcting the first compensation height according to the correction coefficient, and determining the obtained second compensation height as the compensation height of the image; The display unit is configured to display an image on a front windshield of the vehicle.

11. A projection device, characterized in that: include: A processor, and a memory, wherein the memory stores program instructions, and when the program instructions are executed by the processor, the processor executes the display control method according to any one of claims 1 to 8.

12. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores at least one instruction, and the at least one instruction is used to be executed by a processor to implement the display control method according to any one of claims 1 to 8.

Images