AR-HUD, fault detection method, storage medium and program product

Abstract

The embodiment of the invention provides an AR-HUD, a fault detection method, a storage medium and a program product, and the method comprises the steps: obtaining first display brightness determined according to a current display image and second display brightness collected by a first light sensor in real time in an image display process; wherein the first light sensor is located in a light guide device of the AR-HUD; and judging whether the AR-HUD has a display fault or not based on the first display brightness, the second display brightness, the change rate of the first display brightness and the change rate of the second display brightness.

Description

Technical Field

[0001] The present invention relates to the technical field of vehicle-mounted display equipment, and in particular to an AR-HUD and a fault detection method, a storage medium, and a program product. Background Art

[0002] A Head Up Display (HUD) device is an optical display device that directly displays driving information such as vehicle speed and navigation on the vehicle windshield through optical projection. Augmented Reality Head Up Display (AR-HUD) devices add functions such as dynamic arrow indications based on ordinary HUD devices, so that they can provide users with more intuitive and safe navigation information based on current road conditions. However, during the operation of the AR-HUD device, the brightness may decrease and the user may not be able to view the prompt information normally. There is currently no good detection solution for AR-HUD display failures. Summary of the invention

[0003] The embodiments of the present invention provide an AR-HUD and a fault detection method, a storage medium, and a program product, so as to provide a solution for detecting AR-HUD display faults.

[0004] In a first aspect, an embodiment of the present invention provides a fault detection method for an AR-HUD, comprising:

[0005] During the image display process, a first display brightness determined according to the current display image and a second display brightness acquired by a first light sensor are acquired in real time; wherein the first light sensor is located in the display module of the AR-HUD;

[0006] Based on the first display brightness and the second display brightness, and the change rate of the first display brightness and the change rate of the second display brightness, it is determined whether a display failure occurs in the AR-HUD.

[0007] Further, judging whether a display failure occurs in the AR-HUD based on the first display brightness and the second display brightness, and the change rate of the first display brightness and the change rate of the second display brightness, specifically includes:

[0008] Determine whether a first difference between the first display brightness and the second display brightness is greater than a preset first threshold;

[0009] If the first difference is greater than the preset first threshold, determining whether a second difference between the change rate of the first display brightness and the change rate of the second target display brightness is greater than a preset second threshold;

[0010] If the second difference is greater than the preset second threshold, it is determined that a first type of display failure occurs in the AR-HUD; if the second difference is less than or equal to the preset second threshold, it is determined that a second type of display failure occurs in the AR-HUD.

[0011] Further, after determining that the AR-HUD has a display failure, the method further includes at least one of the following:

[0012] Send alarm information to users;

[0013] Generate fault record information and send it to a fault recording device connected to the AR-HUD;

[0014] If it is determined that the AR-HUD has a first type of display failure, restarting the AR-HUD;

[0015] If it is determined that the AR-HUD has a second type of display failure, a third display brightness is determined according to the first display brightness and the second display brightness, and the current display image is displayed with the third display brightness as the target.

[0016] Optionally, determining a third display brightness according to the first display brightness and the second display brightness specifically includes:

[0017] The third target display brightness is determined according to the current ambient light brightness, the first display brightness, and the second display brightness.

[0018] Optionally, the first display brightness is determined according to a current display image and current ambient light brightness.

[0019] In a second aspect, based on the same inventive concept, an embodiment of the present invention further provides an AR-HUD, including:

[0020] A display module, connected to the processor, for luminously displaying a display image sent by the processor;

[0021] A first light sensor, installed inside the display module and connected to the processor, for collecting a second display brightness emitted by the display module and sending the second display brightness to the processor;

[0022] The processor is used to implement the following steps:

[0023] Determine the current display image in real time and send it to the display module;

[0024] In the process of displaying an image, obtaining in real time a first display brightness and a second display brightness determined according to a currently displayed image;

[0025] Based on the first display brightness and the second display brightness, and the change rate of the first display brightness and the change rate of the second display brightness, it is determined whether a display failure occurs in the AR-HUD.

[0026] Optionally, the first light sensor includes a sensing portion, a light shielding portion and a signal conversion portion;

[0027] The sensing unit is used to collect light emitted by the display device, and generate a sensing signal according to the brightness of the collected light and send it to the signal conversion unit;

[0028] The signal conversion unit is used to convert the sensing signal into the second display brightness and send it to the processor;

[0029] The light shielding portion is used to shield at least part of the light except the light emitted by the display device.

[0030] Optionally, the display module includes a display device and a light guide device;

[0031] The first light sensor is installed inside the light guide device.

[0032] In a third aspect, based on the same inventive concept, an embodiment of the present invention further provides a computer-readable storage medium, wherein the computer-readable storage medium stores a computer program code, and the computer program code is used to implement the AR-HUD fault detection method as described in the first aspect.

[0033] In a fourth aspect, based on the same inventive concept, an embodiment of the present invention further provides a computer program product, which includes: a computer program code, when the computer program code runs on a computer, enables the computer to execute the AR-HUD fault detection method as described in the first aspect.

[0034] The beneficial effects of the present invention are as follows:

[0035] The AR-HUD and fault detection method, storage medium, and program product provided by the embodiments of the present invention, based on the existing open-loop controlled AR-HUD, are configured with a light sensor to collect the display brightness of the light emitted by the display module, and determine whether the AR-HUD has a display fault related to brightness based on the numerical value between the display brightness of the light emitted by the display module and the display brightness determined by the processor, as well as the change rate of the two display brightnesses. BRIEF DESCRIPTION OF THE DRAWINGS

[0036] Figure 1 One of the structural schematic diagrams of the AR-HUD provided in an embodiment of the present invention;

[0037] Figure 2The second structural diagram of the AR-HUD provided in the embodiment of the present invention;

[0038] Figure 3 A flowchart of steps implemented by a processor of an AR-HUD provided in an embodiment of the present invention;

[0039] Figure 4 One of the schematic diagrams of the working principles of the display module and the first light sensor of the AR-HUD provided in an embodiment of the present invention;

[0040] Figure 5 The second schematic diagram of the working principle of the display module and the first light sensor of the AR-HUD provided in an embodiment of the present invention;

[0041] Figure 6 A schematic diagram of the structure of a first light sensor for an AR-HUD provided in an embodiment of the present invention;

[0042] Figure 7 One of the flowcharts of some steps implemented by the processor of the AR-HUD provided in an embodiment of the present invention;

[0043] Figure 8 The second flowchart of some steps implemented by the processor of the AR-HUD provided in the embodiment of the present invention;

[0044] Fig. 9 A schematic diagram of the structure of a fault detection device provided by an embodiment of the present invention;

[0045] Fig.10 A schematic diagram of a computer-readable storage medium provided for an embodiment of the present invention. DETAILED DESCRIPTION

[0046] In order to make the above-mentioned objects, features and advantages of the present invention more obvious and understandable, the present invention will be further described below with reference to the accompanying drawings and examples. However, the exemplary embodiments can be implemented in various forms and should not be understood as being limited to the embodiments described herein; on the contrary, these embodiments are provided to make the present invention more comprehensive and complete, and to fully convey the concepts of the exemplary embodiments to those skilled in the art. The same figure marks in the figures represent the same or similar structures, and thus their repeated descriptions will be omitted. The words expressing position and direction described in the present invention are all explained using the accompanying drawings as examples, but changes can be made as needed, and the changes made are all included in the scope of protection of the present invention. The drawings of the present invention are only used to illustrate the relative position relationship and do not represent the true proportions.

[0047] It should be noted that specific details are described in the following description to facilitate a full understanding of the present invention. However, the present invention can be implemented in a variety of other ways different from those described herein, and those skilled in the art can make similar generalizations without violating the connotation of the present invention. Therefore, the present invention is not limited to the specific embodiments disclosed below. The subsequent description of the specification is a preferred embodiment of the present application, but the description is for the purpose of illustrating the general principles of the present application and is not intended to limit the scope of the present application. The scope of protection of the present application shall be determined by the definition of the attached claims.

[0048] The AR-HUD and corresponding fault detection method, storage medium, and program product provided by the embodiments of the present invention are described in detail below with reference to the accompanying drawings.

[0049] In a first aspect, the present invention provides an AR-HUD. Figure 1 and Figure 2 As shown, it includes: a display module D and a processor U. Wherein, the processor U is electrically connected to the display module D. Specifically:

[0050] like Figure 3 As shown, the processor U is used to implement: S110, determining the current display image and sending it to the display module.

[0051] In a specific implementation process, the AR-HUD may further include a communication interface P. The communication interface P may be a serial peripheral interface (Serial Peripheral Interface, SPI) interface, a two-wire serial bus (Inter-Integrated Circuit, I 2 C) interface, RS232 interface, RS422 interface, RS485 interface, controller area network bus (Controller Area Network, CAN) interface and other wired communication interfaces, and / or wireless local area network (Wireless Local Area Network, WLAN) interface, Bluetooth interface and other wireless communication interfaces, which are not limited here. The AR-HUD is electrically connected to the vehicle's electronic control unit (Electronic Control Unit, ECU), body control module (Body Control Module, BCM), vehicle navigation system and other vehicle-mounted components through the communication interface P. The processor U can use the image sent by other vehicle-mounted components received through the communication interface P as the current display image; or, the AR-HUD generates the current display image based on the data sent by other vehicle-mounted components received through the communication interface P. The present invention is not overly limited here.

[0052] The display module D is used to display the display image sent by the processor U in a luminous manner.

[0053] As an optional implementation, the processor U is further configured to: determine a first display brightness according to the current display image and send it to the display module. The display module D is specifically configured to perform luminous display with the first display brightness as the target, and actually emit light with a second display brightness.

[0054] As another optional implementation, when the display module D displays the display image in a luminous manner, it can determine the corresponding target display brightness based on the display image, and display the display image in a luminous manner with the target display brightness as the target, and actually emit light of the second display brightness.

[0055] In the process of the processor U determining the first display brightness according to the current display image, or the display module D determining the corresponding target display brightness according to the display image, gamut mapping can be performed according to the grayscale of the current display image to determine the corresponding first display brightness / target display brightness.

[0056] For example, the corresponding first display brightness / target display brightness may be determined according to the average grayscale of the currently displayed image through a preset grayscale-brightness linear mapping relationship or a color correspondence table (Look Up Table, LUT).

[0057] Alternatively, gamma (γ) correction may be performed to determine the corresponding first display brightness / target display brightness according to the average grayscale of the currently displayed image. Specifically, the gamma correction is converted in the following manner:

[0058] L(I)=A·G(I) γ

[0059] Where I is the current display image, G(I) is the average grayscale of the current display image, and L(I) is the first display brightness L1(I) / target display brightness L corresponding to the current display image. 目标 (I), A and γ are preset parameters. The values ​​of preset parameters A and γ of different models of AR-HUD may be different due to factors such as structural performance.

[0060] The average grayscale G(I) of the current displayed image I can be determined as follows:

[0061]

[0062] Among them, (x n ,y k ) is the pixel coordinate in the current display image I, G(x n ,y k ) is the pixel point (xn ,y k ) in grayscale.

[0063] In order to detect the display failure of the AR-HUD, Figure 1 , Figure 2 The AR-HUD further includes a first light sensor S1. The first light sensor S1 is installed inside the display module D ( Figure 1 , Figure 2 The installation position of the first light sensor S1 is not shown in the figure. The first light sensor S1 is used to collect the second display brightness emitted by the display module D and send it to the processor U.

[0064] On this basis, if Figure 3 As shown, the processor U is also used to implement the following steps:

[0065] S120. During image display, real-time acquisition of a first display brightness and a second display brightness determined according to a currently displayed image.

[0066] S130. Determine whether a display failure occurs in the AR-HUD based on the first display brightness and the second display brightness, and the change rate of the first display brightness and the change rate of the second display brightness.

[0067] In this way, based on the existing open-loop controlled AR-HUD, the embodiment of the present invention sets a light sensor to collect the display brightness of the light emitted by the display module, and judges whether the AR-HUD has a display failure related to brightness according to the numerical value between the display brightness of the light emitted by the display module and the display brightness determined by the processor, as well as the change rate of the two display brightnesses.

[0068] Furthermore, if Figure 2 , Figure 4 , Figure 5 As shown, the display module D includes a display device L and a light guide device G. The display device L is used to display the display image sent by the processor U by emitting light, and the light guide device G is used to guide the light emitted by the display device L to the windshield of the vehicle.

[0069] In the specific implementation process, the display device L can be a thin film transistor liquid crystal display (TFT-LCD), a digital light processing (DLP) device, a liquid crystal on silicon (LCOS) device, etc., and this embodiment does not make too many restrictions here. The light guide device G can be a reflector module FM (such as Figure 4 , Figure 5 As shown), optical waveguide module and other structures, this embodiment does not make too many restrictions here. Further, the light guide device G can also have an imaging magnification function to reduce the size of the display device L. For example, by setting a certain curvature for the reflector F in the reflector module FM, the reflector module FM can obtain a refractive index that can change the direction of light, so that the image formed by the display image on the vehicle windshield can be enlarged or reduced.

[0070] Accordingly, the first light sensor S1 may be installed inside the light guide device L. For example, Figure 4 , Figure 5 As shown, if the light guide device L is the reflector module FM, the first light sensor S1 can be installed inside the reflector module so that the sensing portion S1 - 1 of the first light sensor S1 is located on the optical path of the light emitted by the display device L.

[0071] Furthermore, the display light emitted by the AR-HUD through the display module D will be projected onto the vehicle windshield. Due to the reversible principle of the light path, the external ambient light irradiated on the vehicle windshield will also enter the interior of the display module D through a light path opposite to the display light, thereby affecting the accuracy of the measurement value of the second display brightness emitted by the display module D by the first light sensor S1.

[0072] To address this problem, further optionally, when installing the first light sensor S1, the first light sensor S1 needs to be disposed at a position where it is difficult for the light from the external environment light to enter the display module D to be illuminated.

[0073] Or further optionally, as Figure 5 , Figure 6 As shown, the first light sensor S1 includes a sensing part S1-1, a signal conversion part S1-2 ( Figure 5The sensing unit S1-1 is used to collect the light emitted by the display device D, and generate a sensing signal according to the brightness of the collected light and send it to the signal conversion unit S1-2. The signal conversion unit S1-2 is used to convert the sensing signal into the second display brightness and send it to the processor U. The shielding unit S1-3 is used to shield at least part of the light except the light emitted by the display device.

[0074] For example, Figure 5 As shown, when the first light sensor S1 is installed on the light guide device G ( Figure 5 Taking the reflector module FM as an example, the shading portion S1-3 can be arranged in the propagation direction of the ambient light when the sensing portion S1-1 faces the ambient light and enters the light guide device G. The shading portion can be arranged to be a structure made of an opaque material, such as a metal material, a black plastic material, etc., and the embodiments of the present invention do not make too many restrictions on this.

[0075] Further, for the step S130, based on the first display brightness and the second display brightness, and the change rate of the first display brightness and the change rate of the second display brightness, it is determined whether the AR-HUD has a display failure, and it can be determined comprehensively whether the AR-HUD has a display failure according to the difference between the first display brightness and the second display brightness (hereinafter referred to as the first difference), the ratio and other brightness numerical indicators, and the difference between the change rate of the first display brightness and the change rate of the second display brightness (hereinafter referred to as the second difference), the ratio and other brightness change rate numerical indicators. Different implementation methods can be set according to actual needs.

[0076] For example, as an optional implementation, in step S130, based on the first display brightness and the second display brightness, and the rate of change of the first display brightness and the rate of change of the second display brightness, it is determined whether the AR-HUD has a display failure, such as Figure 7 As shown, specifically including:

[0077] S131A. Determine whether the first difference is greater than a preset first threshold.

[0078] The first difference is calculated as follows:

[0079] Δ L(I)=|L1(I)-L2(I)|

[0080] Among them, for the display image I, Δ L(I) is the first difference, L1(I) is the first display brightness, and L2(I) is the second display brightness.

[0081] If the first difference is greater than the preset first threshold, execute step S132A; if the first difference is less than or equal to the preset first threshold, execute step S135A.

[0082] S132A: Determine whether the second difference is greater than a preset second threshold.

[0083] The second difference is calculated as follows:

[0084]

[0085] For display image I, Δ L'(I) is the second difference, L1(I) is the first display brightness, and L2(I) is the second display brightness. J is the interval of the display image I Δ For display image J, L1(J) is the first display brightness, and L2(J) is the second display brightness. Δ T should be set shorter, such as 0.5 seconds.

[0086] If the second difference is greater than the preset second threshold, execute step S133A; if the second difference is less than or equal to the preset second threshold, execute step S134A.

[0087] S133A: Determine that a first type of display failure occurs in the AR-HUD.

[0088] S134A: Determine that the AR-HUD has a second type of display failure.

[0089] S135A: Determine that the AR-HUD has no display failure. Return to step S110.

[0090] For another example, as another optional implementation, in step S130, based on the first display brightness and the second display brightness, and the rate of change of the first display brightness and the rate of change of the second display brightness, it is determined whether the AR-HUD has a display failure, such as Figure 8 As shown, specifically including:

[0091] S131B. Determine whether the first difference is greater than a preset first threshold.

[0092] If the first difference is greater than the preset first threshold, execute step S132B; if the first difference is less than or equal to the preset first threshold, execute step S134B.

[0093] S132B, determine whether the ratio of the change rate of the first display brightness to the change rate of the second display brightness is greater than a preset third threshold.

[0094] If the ratio is greater than the preset third threshold, execute step S133B; if the ratio is less than or equal to the preset third threshold, execute step S134B.

[0095] S133B. Determine that a first type of display failure occurs in the AR-HUD.

[0096] S134B: Determine that the AR-HUD has no display failure. Return to step S110.

[0097] It should be noted that the embodiment of the present invention may also adopt other feasible implementations to determine whether the AR-HUD has a display failure, and is not limited to the implementations given above.

[0098] Furthermore, after determining that the AR-HUD has a display failure, the processor U is further configured to implement at least one of the following steps S141-S144:

[0099] S141. Send an alarm prompt message to the user.

[0100] In the specific implementation process, the processor U of the AR-HUD can be directly electrically connected to the vehicle's dashboard, alarm buzzer and other human-machine interaction components through the communication interface P, and send instructions through the human-machine interaction component to control the corresponding human-machine interaction component to send an alarm prompt message to the user. For example, the AR-HUD controls the corresponding fault light of the vehicle's dashboard to light up, or controls the vehicle's dashboard to pop up an alarm prompt page for the AR-HUD display fault to the user, and / or controls the alarm buzzer to send an alarm prompt sound, etc. The processor U of the AR-HUD can also be indirectly electrically connected to the human-machine interaction component through the communication interface P. For example, the AR-HUD is electrically connected to the vehicle control components such as ECU and BCM through the communication interface P, and the vehicle control components are electrically connected to the vehicle's dashboard, alarm buzzer and other components. The AR-HUD sends alarm status information to the vehicle control component, and the vehicle control component sends instructions to the connected human-machine interaction component according to the alarm status information to control the corresponding human-machine interaction component to send an alarm prompt message to the user. The embodiments of the present invention are not too limited here.

[0101] S142. Generate fault record information and send it to a fault recording device connected to the AR-HUD.

[0102] In the specific implementation process, the fault recording device can be a vehicle control component such as ECU, BCM, etc.

[0103] S143: Restart the AR-HUD.

[0104] As an optional implementation, if it is determined that the AR-HUD has the first type of display failure, the AR-HUD may be restarted.

[0105] In a specific implementation process, the AR-HUD further includes a power module (not shown in the figure), which is electrically connected to at least some components of the AR-HUD including the processor U, and is used to provide power to the electrically connected components. The processor U sends a restart instruction to the power module to control the power module to power on again, thereby restarting the AR-HUD.

[0106] S144: Determine a third display brightness according to the first display brightness and the second display brightness, and display the current display image with the third display brightness as a target.

[0107] As an optional implementation, if it is determined that the AR-HUD has the second type of display failure, a third display brightness can be determined based on the first display brightness and the second display brightness, and the current display image can be displayed with the third display brightness as the target.

[0108] In the specific implementation process, the third display brightness can be determined in different ways according to actual conditions. For example, the third display brightness is obtained by increasing the first difference value by the first display brightness; or the third display brightness is obtained by increasing the first display brightness by the target adjustment ratio according to a preset first increase relationship, wherein the preset first increase relationship is a correspondence between the first display brightness value, the second display brightness value and the adjustment ratio, and different combinations of the first display brightness value and the second display brightness value correspond to different adjustment ratios; other adjustment methods can also be used, and the embodiments of the present invention do not make too many restrictions here.

[0109] In a specific implementation process, the processor U may send the third display brightness to the display module D, and the display module D performs luminous display with the received third display brightness as the target.

[0110] In this way, the embodiment of the present invention can ensure the driving safety of the vehicle by performing corresponding processing after detecting the AR-HUD sending and displaying a fault. If the AR-HUD also classifies the severity of the fault during the detection process, different treatment measures can be taken for faults of different severities, thereby improving the driver's user experience and further ensuring the driver's driving safety.

[0111] Furthermore, when the AR-HUD projects the display image onto the vehicle windshield, the display effect is affected not only by the brightness of the display light emitted by the AR-HUD, but also by the brightness of the external ambient light. For example, when the brightness of the external ambient light is too high but the brightness of the image projected by the AR-HUD onto the vehicle windshield is relatively low, the driver will not be able to see the display image projected by the AR-HUD clearly.

[0112] Based on this, optionally, Figure 1 , Figure 2 As shown, the AR-HUD further includes a second light sensor S1. The second light sensor S2 is electrically connected to the processor U and is used to collect the current ambient light brightness.

[0113] On this basis, optionally, the first display brightness is determined according to the current display image and the current ambient light brightness. For example, in the process of determining the first display brightness according to the current display image in a gamma correction manner, the first display brightness is determined in the following manner:

[0114] L1(I)=A·G(I) γ +B·L 环境光 +C

[0115] Among them, L 环境光 is the current ambient light brightness, and B and C are also preset parameters. The values ​​of preset parameters B and C of different models of AR-HUD may be different due to factors such as structural performance.

[0116] Optionally, in step S144, determining a third display brightness according to the first display brightness and the second display brightness specifically includes:

[0117] The third target display brightness is determined according to the current ambient light brightness, the first display brightness, and the second display brightness.

[0118] In the specific implementation process, the third display brightness can be determined in different ways according to actual conditions. For example, the first display brightness is increased by the target adjustment ratio according to the preset second increase relationship to obtain the third display brightness, wherein the preset second increase relationship is the correspondence between the first display brightness value, the second display brightness value, the current ambient light brightness and the adjustment ratio, and different combinations of the first display brightness value, the second display brightness value and the current ambient light brightness value correspond to different adjustment ratios; or, the first display brightness is increased by the target adjustment value according to the preset third increase relationship to obtain the third display brightness, wherein the preset third increase relationship is the correspondence between the first display brightness value, the second display brightness value, the current ambient light brightness and the adjustment value, and different combinations of the first display brightness value, the second display brightness value and the current ambient light brightness value correspond to different adjustment values; other adjustment methods can also be used, and the embodiments of the present invention are not overly limited here.

[0119] Alternatively, if Figure 1 , Figure 2 As shown, the AR-HUD also includes a memory M, in which computer program instructions are stored. The computer program instructions are executed by the processor U to implement the steps implemented by the processor U as described above.

[0120] In a second aspect, based on the same inventive concept, an embodiment of the present invention further provides a fault detection method for an AR-HUD, comprising:

[0121] During the image display process, a first display brightness determined according to the current display image and a second display brightness acquired by a first light sensor are acquired in real time; wherein the first light sensor is located in the display module of the AR-HUD;

[0122] Based on the first display brightness and the second display brightness, and the change rate of the first display brightness and the change rate of the second display brightness, it is determined whether a display failure occurs in the AR-HUD.

[0123] As an optional implementation manner, the first display brightness is determined according to the average grayscale of the currently displayed image through a preset grayscale-brightness linear mapping relationship.

[0124] As another optional implementation, the first display brightness is determined according to the average grayscale of the currently displayed image through a preset LUT.

[0125] As another optional implementation, the first display brightness is determined according to the average grayscale of the current display image through gamma (γ) correction. The gamma correction is converted in the following manner:

[0126] L1(I)=A·G(I) γ

[0127] Wherein, I is the current display image, G(I) is the average grayscale of the current display image, L1(I) is the first display brightness, and A and γ are preset parameters.

[0128] The average grayscale G(I) of the current displayed image I is determined as follows:

[0129]

[0130] Among them, (x n ,y k ) are the pixel coordinates in the current display image I.

[0131] Furthermore, based on the first display brightness and the second display brightness, and the rate of change of the first display brightness and the rate of change of the second display brightness, it is determined whether the AR-HUD has a display failure. It is possible to comprehensively determine whether the AR-HUD has a display failure based on brightness numerical indicators such as the difference between the first display brightness and the second display brightness (i.e., the first difference), the ratio, and the difference between the rate of change of the first display brightness and the rate of change of the second display brightness (i.e., the second difference), the ratio, and other brightness change rate numerical indicators.

[0132] As an optional implementation manner, judging whether a display failure occurs in the AR-HUD based on the first display brightness and the second display brightness, and the change rate of the first display brightness and the change rate of the second display brightness, specifically includes:

[0133] Determine whether the first difference is greater than a preset first threshold;

[0134] If the first difference is greater than the preset first threshold, determining whether a second difference between the change rate of the first display brightness and the change rate of the second target display brightness is greater than a preset second threshold;

[0135] If the second difference is greater than the preset second threshold, it is determined that a first type of display failure occurs in the AR-HUD; if the second difference is less than or equal to the preset second threshold, it is determined that a second type of display failure occurs in the AR-HUD.

[0136] As another optional implementation, judging whether a display failure occurs in the AR-HUD based on the first display brightness and the second display brightness, and the change rate of the first display brightness and the change rate of the second display brightness, specifically includes:

[0137] Determine whether the first difference is greater than a preset first threshold;

[0138] If the first difference is greater than the preset first threshold, determining whether a ratio of the change rate of the first display brightness to the change rate of the second target display brightness is greater than a preset third threshold;

[0139] If the ratio is greater than the preset third threshold, it is determined that a first type of display failure occurs in the AR-HUD; if the first difference is less than or equal to the preset first threshold, if the ratio is less than or equal to the preset third threshold, it is determined that no display failure occurs in the AR-HUD.

[0140] Further, after determining that the AR-HUD has a display failure, the method further includes at least one of the following:

[0141] Send alarm information to users;

[0142] Generate fault record information and send it to a fault recording device connected to the AR-HUD;

[0143] Restarting the AR-HUD;

[0144] A third display brightness is determined according to the first display brightness and the second display brightness, and a current display image is displayed with the third display brightness as a target.

[0145] Optionally, if it is determined that the AR-HUD has a first type of display failure, the AR-HUD is restarted. If it is determined that the AR-HUD has a second type of display failure, a third display brightness is determined according to the first display brightness and the second display brightness, and the current display image is displayed with the third display brightness as the target.

[0146] In the specific implementation process, the third display brightness can be determined in different ways according to actual conditions. For example, the third display brightness is obtained by increasing the first difference value by the first display brightness; or the third display brightness is obtained by increasing the first display brightness by the target adjustment ratio according to a preset first increase relationship, wherein the preset first increase relationship is a correspondence between the first display brightness value, the second display brightness value and the adjustment ratio, and different combinations of the first display brightness value and the second display brightness value correspond to different adjustment ratios; other adjustment methods can also be used, and the embodiments of the present invention do not make too many restrictions here.

[0147] Furthermore, determining the third display brightness according to the first display brightness and the second display brightness specifically includes:

[0148] The third target display brightness is determined according to the current ambient light brightness, the first display brightness, and the second display brightness.

[0149] In the specific implementation process, the third display brightness can be determined in different ways according to actual conditions. For example, the first display brightness is increased by the target adjustment ratio according to the preset second increase relationship to obtain the third display brightness, wherein the preset second increase relationship is the correspondence between the first display brightness value, the second display brightness value, the current ambient light brightness and the adjustment ratio, and different combinations of the first display brightness value, the second display brightness value and the current ambient light brightness value correspond to different adjustment ratios; or, the first display brightness is increased by the target adjustment value according to the preset third increase relationship to obtain the third display brightness, wherein the preset third increase relationship is the correspondence between the first display brightness value, the second display brightness value, the current ambient light brightness and the adjustment value, and different combinations of the first display brightness value, the second display brightness value and the current ambient light brightness value correspond to different adjustment values; other adjustment methods can also be used, and the embodiments of the present invention are not overly limited here.

[0150] Optionally, the first display brightness is determined according to a current display image and current ambient light brightness.

[0151] As an optional implementation manner, in the process of determining the first display brightness according to the current display image in a gamma correction manner, the first display brightness is determined in the following manner:

[0152] L1(I)=A·G(I) γ +B·L 环境光 +C

[0153] Among them, L 环境光 is the current ambient light brightness, and B and C are also preset parameters. The values ​​of preset parameters B and C of different models of AR-HUD may be different due to factors such as structural performance.

[0154] Since the principle of the AR-HUD fault detection method described in the second aspect is similar to that of the AR-HUD described in the first aspect, it can be implemented with reference to the content of the first aspect, and the same implementation content will not be repeated here.

[0155] In a third aspect, based on the same inventive concept, an embodiment of the present invention further provides a fault detection device, such as Fig. 9 As shown, including:

[0156] A brightness acquisition module M1 is used to acquire, in real time during image display, a first display brightness determined according to a currently displayed image and a second display brightness acquired by a first light sensor; wherein the first light sensor is located in a display module of the AR-HUD;

[0157] The fault judgment module M2 is used to judge whether a display fault occurs in the AR-HUD based on the first display brightness and the second display brightness, and the change rate of the first display brightness and the change rate of the second display brightness.

[0158] Optionally, the fault detection device further includes: a fault processing module M3, configured to implement at least one of the following after determining that a display fault occurs in the AR-HUD:

[0159] Send alarm information to users;

[0160] Generate fault record information and send it to a fault recording device connected to the AR-HUD;

[0161] Restarting the AR-HUD;

[0162] A third display brightness is determined according to the first display brightness and the second display brightness, and a current display image is displayed with the third display brightness as a target.

[0163] It should be understood that the above-described embodiments of the fault detection device are merely illustrative. For example, the division of the modules is merely a logical functional division, and there may be other division methods in actual implementation. The modules described as separate components may or may not be physically separated, may be integrated in a processing unit, or each module may exist physically separately. The above-mentioned integrated modules may be implemented in the form of hardware or in the form of software functional modules. If the integrated modules are implemented in the form of software functional modules and sold or used as independent products, they may be stored in a computer-readable storage medium.

[0164] Since the specific manner in which each module of the fault detection device performs operations has been described in detail in the AR-HUD fault detection method described in the second aspect, the corresponding implementation can be referred to the content of the second aspect and will not be repeated here.

[0165] In a fourth aspect, based on the same inventive concept, an embodiment of the present invention further provides a computer-readable storage medium, wherein the computer-readable storage medium stores a computer program code, and the computer program code is used to implement the AR-HUD fault detection method described in the second aspect.

[0166] The computer storage medium may be any available medium or data storage device that can be accessed by a computer, including but not limited to magnetic storage (such as floppy disk, hard disk, magnetic tape, magneto-optical disk (MO)), optical storage 100 (such as Fig.10As shown) (such as CD, DVD, BD, HVD, etc.), and semiconductor memory (such as ROM, EPROM, EEPROM, non-volatile memory (NAND FLASH), solid state drive (SSD)), etc.

[0167] In the fifth aspect, based on the same inventive concept, an embodiment of the present invention further provides a computer program product 200, which includes: a computer program code, when the computer program code runs on a computer, enables the computer to execute the AR-HUD fault detection method as described in the second aspect.

[0168] The computer program product 200 may be implemented in one or more computer-readable storage media (eg, Fig.10 The program code for performing the operation of the present application can be written in any combination of one or more programming languages, including object-oriented programming languages ​​such as Java, C++, etc., and also conventional procedural programming languages ​​such as "C" language or similar programming languages.

[0169] The AR-HUD and fault detection method, storage medium, and program product provided by the embodiments of the present invention, based on the existing open-loop controlled AR-HUD, are configured with a light sensor to collect the display brightness of the light emitted by the display module, and determine whether a display fault related to brightness occurs in the AR-HUD based on the numerical value between the display brightness of the light emitted by the display module and the display brightness determined by the processor, as well as the change rate of the two display brightnesses.

[0170] Those skilled in the art will appreciate that the embodiments of the present application may be provided as methods, systems, or computer program products. Therefore, the present application may adopt the form of a complete hardware embodiment, a complete software embodiment, or an embodiment in combination with software and hardware. Moreover, the present application may adopt the form of a computer program product implemented in one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) that include computer-usable program code.

[0171] The present application is described with reference to the flowcharts and / or block diagrams of the methods, devices (systems), and computer program products according to the present application. It should be understood that each process and / or box in the flowchart and / or block diagram, as well as the combination of the processes and / or boxes in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to produce a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 A process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0172] These computer program instructions may also be stored in a computer-readable memory capable of directing a computer or other programmable data processing device to operate in a specific manner, so that the instructions stored in the computer-readable memory produce an article of manufacture comprising an instruction device, which implements the process Figure 1 A process or multiple processes and / or boxes Figure 1 A function specified in one or more boxes.

[0173] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operating steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing instructions for implementing the process. Figure 1 A process or multiple processes and / or boxes Figure 1 The steps for the functions specified in one or more boxes.

[0174] Obviously, those skilled in the art can make various changes and modifications to the present application without departing from the spirit and scope of the present application. Thus, if these modifications and variations of the present application fall within the scope of the claims of the present application and their equivalents, the present application is also intended to include these modifications and variations.

Claims

1. A fault detection method for an augmented reality head-up display device AR-HUD, characterized in that: include: During the image display process, a first display brightness determined according to the current display image and a second display brightness acquired by the first light sensor are acquired in real time; Wherein the first light sensor is located in the display module of the AR-HUD; Based on the first display brightness and the second display brightness, and the change rate of the first display brightness and the change rate of the second display brightness, it is determined whether a display failure occurs in the AR-HUD.

2. The method according to claim 1, characterized in that The determining whether a display failure occurs in the AR-HUD based on the first display brightness and the second display brightness, and the change rate of the first display brightness and the change rate of the second display brightness, specifically includes: Determine whether a first difference between the first display brightness and the second display brightness is greater than a preset first threshold; If the first difference is greater than the preset first threshold, determining whether a second difference between the change rate of the first display brightness and the change rate of the second target display brightness is greater than a preset second threshold; If the second difference is greater than the preset second threshold, it is determined that a first type of display failure occurs in the AR-HUD; if the second difference is less than or equal to the preset second threshold, it is determined that a second type of display failure occurs in the AR-HUD.

3. The method according to claim 2, characterized in that After determining that the AR-HUD has a display failure, the method further includes at least one of the following: Send alarm information to users; Generate fault record information and send it to a fault recording device connected to the AR-HUD; If it is determined that the AR-HUD has a first type of display failure, restarting the AR-HUD; If it is determined that the AR-HUD has a second type of display failure, a third display brightness is determined according to the first display brightness and the second display brightness, and the current display image is displayed with the third display brightness as the target.

4. The method according to claim 3, characterized in that The determining the third display brightness according to the first display brightness and the second display brightness specifically includes: The third target display brightness is determined according to the current ambient light brightness, the first display brightness, and the second display brightness.

5. The method according to claim 1, characterized in that The first display brightness is determined according to the current display image and the current ambient light brightness.

6. An AR-HUD, characterized in that: include: A display module, connected to the processor, for luminously displaying a display image sent by the processor; A first light sensor, installed inside the display module and connected to the processor, for collecting a second display brightness emitted by the display module and sending the second display brightness to the processor; The processor is used to implement the following steps: Determine the current display image in real time and send it to the display module; In the process of displaying an image, obtaining in real time a first display brightness and a second display brightness determined according to a currently displayed image; Based on the first display brightness and the second display brightness, and the change rate of the first display brightness and the change rate of the second display brightness, it is determined whether a display failure occurs in the AR-HUD.

7. The AR-HUD according to claim 6, wherein: The first light sensor includes a sensing part, a light shielding part and a signal conversion part; The sensing unit is used to collect light emitted by the display device, and generate a sensing signal according to the brightness of the collected light and send it to the signal conversion unit; The signal conversion unit is used to convert the sensing signal into the second display brightness and send it to the processor; The light shielding portion is used to shield at least part of the light except the light emitted by the display device.

8. The AR-HUD according to claim 6, wherein: The display module includes a display device and a light guide device; The first light sensor is installed inside the light guide device.

9. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a computer program code, and the computer program code is used to implement the AR-HUD fault detection method according to any one of claims 1 to 5.

10. A computer program product, characterized in that The computer program product includes: a computer program code, and when the computer program code is run on a computer, the computer executes the AR-HUD fault detection method according to any one of claims 1 to 5.

Images