Circuit device and head-up display device

Abstract

To provide a circuit arrangement and the like that can detect, for every division area, an occlusion error in HUD display.SOLUTION: A circuit arrangement 100 is used for a head-up display device 50. The circuit arrangement 100 includes a distortion correction circuit 110 and an error detection circuit 120. The distortion correction circuit 110 performs distortion correction of input image data IMA that is image data of an input image, and outputs output image data IMB that is image data of an output image after the distortion correction. First division areas in a first division area group obtained by dividing the input image and second division areas in a second division area group in the output image correspond to each other in the distortion correction. At this time, the error detection circuit 120 detects an occlusion error in a display image of the head-up display device 50 on the basis of, the output image data IMB of the second division areas.SELECTED DRAWING: Figure 2

Description

[Technical Field]

[0001] The present invention relates to a circuit device, a head-up display device, and the like. [Background technology]

[0002] Patent Document 1 discloses a circuit device used in a head-up display. This circuit device includes an error detection circuit and a processing circuit. The error detection circuit detects the occurrence of a first glare error when a glare index value calculated from image data for the head-up display exceeds a first threshold value. When the occurrence of the first glare error is detected, the processing circuit performs processing corresponding to the first glare error. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Japanese Patent Publication No. 2020-101784 Summary of the Invention [Problem to be solved by the invention]

[0004] In Patent Document 1, a glare index is calculated from image data of the entire screen, and the occurrence of a first glare error is detected based on the glare index. In other words, because glare detection is performed for the entire screen of the head-up display, there is a problem that only a common error response process can be performed for the entire screen. [Means for solving the problem]

[0005] One aspect of the present disclosure relates to a circuit device used in a head-up display device, the circuit device including: a distortion correction circuit that performs distortion correction on input image data, which is image data of an input image, and outputs output image data, which is image data of an output image after the distortion correction; and an error detection circuit that, when each first divided area of ​​a first divided area group into which the input image is divided corresponds to each second divided area of ​​a second divided area group in the output image in the distortion correction, detects an occlusion error in a display image of the head-up display device based on the output image data of each second divided area.

[0006] Another aspect of the present disclosure relates to a head-up display device including the circuit device described above, a processing device that controls the circuit device, and a display device that projects the display image based on the output image data from the circuit device. [Brief explanation of the drawings]

[0007] [Figure 1] Example of HUD display. [Figure 2] 1 shows a first configuration example of a head-up display device and a circuit device. [Figure 3] FIG. 4 is a diagram for explaining the operation of the error detection circuit. [Figure 4] 10A and 10B are diagrams illustrating the operation of the circuit device when an occlusion error is detected. [Figure 5] 1 shows a first detailed configuration example of a circuit device. [Figure 6] A detailed configuration example of an error detection circuit. [Figure 7] 10 shows a second detailed configuration example of a circuit device. [Figure 8] 10 shows a third detailed configuration example of a circuit device. [Figure 9] 4 shows a fourth detailed configuration example of a circuit device. [Figure 10] 2 shows a second configuration example of a head-up display device and a circuit device. [Figure 11] 10A and 10B are diagrams illustrating the operation of a head-up display device and a circuit device in a second configuration example. DETAILED DESCRIPTION OF THE INVENTION

[0008] Preferred embodiments of the present disclosure will be described in detail below. Note that the embodiments described below do not unduly limit the scope of the claims, and not all of the configurations described in the embodiments are necessarily essential components.

[0009] 1. First configuration example of display system and circuit device An example of a HUD display is shown in Figure 1. HUD is an abbreviation for head-up display, and in the following, head-up display may be abbreviated as HUD where appropriate.

[0010] The HUD includes a liquid crystal panel, a backlight device, and a reflector. The backlight device emits light, which passes through the liquid crystal panel and is reflected by the reflector toward a screen. The light reflected by the screen enters the user's eyes. This projects a virtual image object 6 corresponding to the object displayed on the liquid crystal panel into the user's field of vision. This virtual image object 6 is superimposed on the real space that forms the background of the HUD display. The areas within the HUD display area 5 where the virtual image object 6 is not displayed are transparent areas with no display because the liquid crystal panel is opaque, and the background is visible.

[0011] In this case, the HUD display object 6 may obscure the background, potentially reducing the visibility of the background that overlaps the display object 6. While Patent Document 1 described above detects whether the overall glare index for the display area 5 exceeds a threshold, it is desirable to be able to detect which part of the display area 5 has reduced background visibility. As an example, from the perspective of continuing to present information to the user, it is desirable to maintain the HUD display as long as possible. If it is possible to detect which part of the background visibility has reduced, it is possible to make only the part of the HUD display where background visibility has reduced transparent, rather than making the entire HUD display transparent, and maintain the rest of the HUD display.

[0012] 2 shows a first configuration example of a head-up display device 50 according to this embodiment and a circuit device 100 used in the head-up display device 50. The head-up display device 50 includes a display device 30, the circuit device 100, and a processing device 200.

[0013] The processing device 200 transmits input image data IMA, which is image data of an input image, to the circuit device 100. The processing device 200 is a so-called SoC, and is, for example, a processor such as a CPU or a microcomputer. SoC is an abbreviation for System on Chip. CPU is an abbreviation for Central Processing Unit.

[0014] The circuit device 100 includes an input circuit 105, a distortion correction circuit 110, an error detection circuit 120, and an output circuit 130. The circuit device 100 is, for example, an integrated circuit device in which a plurality of circuit elements are integrated on a semiconductor substrate.

[0015] The input circuit 105 receives input image data IMA from the processing device 200. The input circuit 105 may be a receiving circuit for various communication interfaces, examples of which include LVDS, DVI, DisplayPort, GMSL, and GVIF. LVDS stands for Low Voltage Differential Signaling, DVI stands for Digital Visual Interface, GMSL stands for Gigabit Multimedia Serial Link, and GVIF stands for Gigabit Video Interface.

[0016] The distortion correction circuit 110 performs distortion correction on the input image data IMA using coordinate conversion between pixel coordinates in the input image data IMA and pixel coordinates in the output image data IMB, and outputs the result as output image data IMB, which is image data for the output image. Distortion correction is image correction that applies image distortion to the image that is the opposite of the image distortion that occurs when the image displayed on the display panel is projected, thereby creating a HUD display with no or reduced distortion. Image distortion due to projection includes image distortion due to the curved surface of the screen, image distortion due to the HUD optical system, or both.

[0017] The distortion correction circuit 110 corresponds to a reverse warp engine or a forward warp engine. Reverse warp is a warp process that transforms pixel coordinates on the output image data IMB into corresponding reference coordinates and obtains pixel data on the output image data IMB from pixel data on the input image data IMA at those reference coordinates. Forward warp is a warp process that transforms pixel coordinates on the input image data IMA into corresponding destination coordinates and obtains pixel data on the output image data IMB at those destination coordinates from pixel data on the input image data IMA at those pixel coordinates. The coordinate transformations in reverse warp and forward warp are defined by warp parameters. The warp parameters are a table that associates coordinates on the input image data IMA with coordinates on the output image data IMB, a table that indicates the amount of movement between coordinates on the input image data IMA and coordinates on the output image data IMB, or polynomial coefficients that associate coordinates on the input image data IMA with coordinates on the output image data IMB.

[0018] The distortion correction circuit 110 and the error detection circuit 120 are logic circuits. The distortion correction circuit 110 and the error detection circuit 120 may each be configured as separate circuits, or the distortion correction circuit 110 and the error detection circuit 120 may be configured as an integrated circuit using automatic placement and routing or the like. Furthermore, some or all of these logic circuits may be realized by a processor such as a DSP. DSP stands for Digital Signal Processor. In this case, a program or instruction set describing the function of each circuit is stored in memory, and the function of each circuit is realized by the processor executing the program or instruction set.

[0019] The output circuit 130 transmits the output image data IMB to the display device 30. The output circuit 130 may be a transmission circuit of various communication interfaces, examples of which include LVDS, DVI, DisplayPort, GMSL, and GVIF.

[0020] The display device 30 displays a virtual image in the user's field of view based on the output image data IMB received from the circuit device 100. The display device 30 includes a display controller, a display driver, an image display device, and an optical system. However, the configuration of the display device 30 is not limited to this, and for example, the circuit device 100 may have a built-in display controller function.

[0021] The display controller transmits image data to the display driver and controls the display timing based on the received output image data IMB. The display driver drives the image display device based on the image data and display timing control from the display controller, and the image display device displays an image corresponding to the output image data IMB. The optical system includes a reflector and projects the image displayed by the image display device onto a screen. The screen may be a transparent projection target having a projection surface that reflects the projected light. For example, the screen may be a windscreen of a mobile vehicle equipped with a HUD. The image display device may be, for example, a liquid crystal display panel and a backlight device. Alternatively, the image display device may include a laser light source, a mirror that reflects the laser, and an actuator that drives the mirror to scan the laser. Alternatively, the image display device may be a digital mirror device including a laser light source, an array of micromirrors, and an actuator that drives each micromirror. Alternatively, the image display device may be a self-luminous display panel such as an OLED display panel. OLED stands for Organic Light Emitting Diode.

[0022] The error detection circuit 120 of the circuit device 100 detects occlusion errors in the display image of the display device 30 based on the output image data IMB. From the perspective of detecting occlusion errors in the HUD display, it is desirable to perform error determination based on image data as close as possible to the HUD display. For this reason, the error detection circuit 120 detects occlusion errors based on the output image data IMB. An occlusion error is an error in which a virtual object displayed by the HUD reduces the visibility of the background that overlaps the object. However, the occlusion error is not limited to a reduction in the visibility of a specific object, such as a pedestrian, but also refers to the possibility that a HUD object may reduce the visibility of any background.

[0023] 3 is a diagram illustrating the operation of error detection circuit 120. Error detection circuit 120 includes area determination unit 121 and error determination unit 122. Note that dotted lines are added to indicate divided areas and are not actually displayed.

[0024] As shown in FIG. 3, the region determination unit 121 sets a first group of divided regions by dividing the input image. Each first divided region in the first group of divided regions is indicated by ARA. Specifically, the divided regions are set by multiple straight lines in the horizontal scanning direction and multiple straight lines in the vertical direction. The circuit device 100 includes a region setting register (not shown). The processing device 200 writes setting information for the first group of divided regions to the region setting register, and the region determination unit 121 sets the first group of divided regions based on the setting information. Note that FIG. 3 shows an example in which the image is divided into 6x4 regions, but the number of divisions is not limited to this. Also, while FIG. 3 shows an example in which the image is divided evenly in the horizontal and vertical directions, it may be divided unevenly. For example, the upper part of the screen may be divided into finer vertical divisions, or the central part of the screen may be divided into finer horizontal divisions.

[0025] The input image is corrected for distortion in the opposite direction to that caused by HUD projection, and then projected onto the HUD, resulting in a display without distortion similar to the input image. In other words, setting a first group of divided areas in the input image is equivalent to setting a group of divided areas in the HUD display. This group of divided areas in the HUD display is referred to as a third group of divided areas, and each third group of divided areas is indicated by ARH. The shape of the third group of divided areas ARH is the same as the first group of divided areas ARA corresponding to that third group of divided areas ARH. Furthermore, the coordinates (u, v) on the input image and the coordinates (x, y) on the output image are associated by coordinate transformation for distortion correction. This association of coordinates divides the output image into a second group of divided areas corresponding to the first group of divided areas. Each second group of divided areas in the second group of divided areas is indicated by ARB. The second group of divided areas ARB has a distorted shape due to distortion correction.

[0026] As described above, each third divided area ARH in the HUD display can be associated with each second divided area ARB in the output image. As a result, the error detection circuit 120 can detect an occlusion error in the third divided area ARH corresponding to the second divided area ARB in the HUD display by detecting an error in the output image data IMB of the second divided area ARB in the output image.

[0027] Specifically, the region determination unit 121 determines which first division region ARA of the first division region group a coordinate (u, v) in the input image corresponding to a coordinate (x, y) in the output image belongs to. When multiple coordinates (u, v) corresponding to multiple coordinates (x, y) belong to the same first division region ARA, the region determination unit 121 determines that the multiple coordinates (x, y) belong to the same second division region ARB. In this way, the second division region ARB of the output image corresponding to the first division region ARA set in the input image is determined. The pixel data of the output image data IMB at the coordinate (x, y) is designated as PB(x, y). The error determination unit 122 determines an occlusion error in the third division region ARH corresponding to the second division region ARB from all pixel data PB(x, y) belonging to the second division region ARB. The error determination unit 122 performs this error determination for each second division region ARB.

[0028] 4 is a diagram illustrating the operation of the circuit device 100 when an occlusion error is detected. In FIG. 4, the divided area in which the occlusion error is detected is indicated by hatching, but this hatching is not actually displayed.

[0029] The error determination unit 122 outputs the error determination result to the output circuit 130. The error determination result is information about the second divided area ARB in which an occlusion error was detected. Specifically, the error determination result is the coordinates (x, y) of the pixel belonging to the second divided area ARB in which the occlusion error was detected. The output circuit 130 sets the output image data IMB for the second divided area ARB in which the occlusion error was detected to a transparent color. A transparent color is a color in which, when the color displayed on the display panel is projected by the HUD, nothing is displayed in the HUD display and the background is visible as is. Specifically, since the HUD display should become transparent when the pixels of the display panel block light, a color that appears black when displayed on the display panel corresponds to a transparent color. For example, black data in the image data becomes a transparent color in the HUD display.

[0030] The error determination unit 122 may output the error determination result to the input circuit 105 or the distortion correction circuit 110. The error determination result is information on the first division area ARA corresponding to the second division area ARB in which the occlusion error was detected. The input circuit 105 or the distortion correction circuit 110 sets the input image data IMA in the first division area ARA corresponding to the second division area ARB in which the occlusion error was detected to a transparent color.

[0031] In the above-described embodiment, the circuit device 100 is used in the head-up display device 50. The circuit device 100 includes a distortion correction circuit 110 and an error detection circuit 120. The distortion correction circuit 110 performs distortion correction on input image data IMA, which is image data of an input image, and outputs output image data IMB, which is image data of an output image after the distortion correction. Each first divided area ARA of the first divided area group obtained by dividing the input image corresponds to each second divided area ARB of the second divided area group in the output image in the distortion correction. At this time, the error detection circuit 120 detects an occlusion error in the display image of the head-up display device 50 based on the output image data IMB of each second divided area ARB.

[0032] According to this embodiment, when background visibility is partially reduced in the HUD display, the region where background visibility is reduced can be detected. Specifically, an occlusion error is detected based on the output image data IMB of each second divided region ARB, and the region on the HUD display onto which the second divided region where the occlusion error was detected is projected is detected as a region where background visibility is reduced. As a result, for example, the region determined to have an occlusion error can be displayed transparently to ensure background visibility, while maintaining the HUD display of the other regions, thereby allowing information to be continuously presented to the user.

[0033] Furthermore, according to this embodiment, occlusion errors are detected from the output image data IMB after distortion correction, so even if a processing error occurs during distortion correction, the occlusion error is determined taking into account the effect of that error. At this time, since the output image is an image distorted by distortion correction, a first division area group is set in the undistorted input image. According to this embodiment, each first division area ARA of the first division area group obtained by dividing the input image corresponds to each second division area ARB of the second division area group in the output image during distortion correction, so the error detection circuit 120 can detect an occlusion error in each second division area ARB using this correspondence.

[0034] In addition, in this embodiment, the error detection circuit 120 determines the correspondence between each first divided area ARA and each second divided area ARB based on the correspondence information between the coordinates GZA=(u,v) on the input image and the coordinates GZB=(x,y) on the output image, which are matched by distortion correction.

[0035] According to this embodiment, the coordinate transformation for distortion correction involves a coordinate transformation between the coordinate GZA=(u,v) on the input image and the coordinate GZB=(x,y) on the output image. Using this correspondence, the error detection circuit 120 can determine to which first division area ARA the coordinate GZA=(u,v) on the input image corresponding to the coordinate GZB=(x,y) on the output image belongs. This allows the first division area ARA on the input image to correspond to the second division area ARB on the output image.

[0036] In the present embodiment, the error detection circuit 120 converts the output image data IMB of the second divided area ARB determined to be an occlusion error among the second divided area group into transparent color data that becomes a transparent color in the display image of the head-up display device 50. The transparent color data is, for example, black data.

[0037] According to this embodiment, the second divided area ARB determined to be an occlusion error is set to transparent color data, so when the second divided area ARB is projected by the HUD, the area becomes transparent. As a result, the area that may have reduced visibility in the HUD display becomes transparent, ensuring visibility.

[0038] 2. First to third detailed configuration examples of the circuit device Fig. 5 shows a first detailed configuration example of the circuit device 100. In Fig. 5, the distortion correction circuit 110 includes a coordinate counter 112, a coordinate conversion circuit 113, an interpolation circuit 114, and a memory circuit 115. Here, an example will be described in which the distortion correction circuit 110 is a reverse warp engine.

[0039] The coordinate counter 112 outputs pixel coordinates GZB=(x,y) on the output image data IMB. The coordinate conversion circuit 113 converts the pixel coordinates (x,y) into reference coordinates GZA=(u,v) on the input image data IMA. The memory circuit 115 temporarily stores the input image data IMA and outputs pixel data PXD of the reference coordinates (u,v). Specifically, the coordinate conversion circuit 113 converts the reference coordinates (u,v) into a read address, and the memory circuit 115 reads the pixel data PXD of the reference coordinates (u,v) from the read address. More specifically, the coordinate conversion circuit 113 outputs read addresses of multiple pixels around the reference coordinates (u,v), and the memory circuit 115 reads the pixel data of the multiple pixels. The interpolation circuit 114 interpolates the multiple pixel data read corresponding to the reference coordinates (u,v) to obtain pixel data of the pixel coordinates (x,y) in the output image data IMB.

[0040] The area determination unit 121 of the error detection circuit 120 determines the first divided area ARA to which the reference coordinate GZA=(u,v) on the input image data IMA belongs, and thereby determines the second divided area ARB to which the pixel coordinate (x,y) on the output image data IMB belongs.

[0041] The error determination unit 122 determines whether or not each second divided area ARB corresponds to an occlusion error by calculating an occlusion determination index for each second divided area ARB and comparing the occlusion determination index with a threshold value. The occlusion determination index is an index that indicates the degree to which the HUD display of the third divided area ARH corresponding to the second divided area ARB occludes the real object when the second divided area ARB is projected by the display device 30.

[0042] Fig. 6 shows a detailed configuration example of the error detection circuit 120. In Fig. 6, the error determination unit 122 includes a brightness calculation unit SSA and a black pixel ratio calculation unit SSB, and a comparison unit CPA and a comparison unit CPB. Here, an example will be described in which the occlusion determination index is the average brightness value and the black pixel ratio, but the occlusion determination index is not limited to this. For example, the occlusion determination index may be only one of the average brightness value or the black pixel ratio.

[0043] The brightness calculation unit SSA calculates the average brightness value of each second divided area ARB as an occlusion determination index based on the area determination result and the output image data IMB. Specifically, the average brightness value Yave is calculated using the following equations (1) to (3). Yi=Cr×R+Cb×B+Cg×G (1) if(Yi≦255) Y=Yi, if(Yi>255) Y=255 ···(2) Yave = ΣY / Ntotal (3)

[0044] In the above formula (1), Cr, Cb, and Cg are predetermined coefficients. R is the R pixel value of the pixel at (x, y), B is the B pixel value of the pixel at (x, y), and G is the G pixel value of the pixel at (x, y). In the above formula (3), Σ indicates that Y of all pixels belonging to the second divided area that is the target of the Yave calculation is added. Ntotal indicates the number of all pixels belonging to the second divided area that is the target of the Yave calculation.

[0045] The comparison unit CPA compares the average brightness value Yave of each second division area ARB with a threshold value for the average brightness value, and determines that the second division area ARB for which Yave is greater than the threshold value is an occlusion error. A common threshold may be set for all second division areas ARB, or a threshold may be set independently for each second division area ARB.

[0046] The black pixel ratio calculation unit SSB calculates the black pixel ratio of each second divided area ARB as an occlusion determination index based on the area determination result and the output image data IMB. The black pixel ratio is the ratio of the number of black pixels to the total number of pixels belonging to the second divided area. Black pixels are pixels of black data, but are not limited to completely black data, as long as they are approximately black data. Specifically, the black pixel ratio BPratio is calculated using the following equations (4) and (5). BPX=(R <Rth)&&(B<Bth)&&(G<Gth) ···(4) BPratio=ΣBPX / Nto t al (5)

[0047] In the above formula (4), Rth is the threshold value for the R pixel value, Bth is the threshold value for the B pixel value, and Gth is the threshold value for the G pixel value. (Condition a) & & (Condition b) is 1 if both condition a and condition b are met, and 0 otherwise.

[0048] The comparison unit CPB compares the black pixel ratio BPratio of each second division area ARB with a black pixel ratio threshold, and determines that the second division area ARB having a BPratio smaller than the threshold is an occlusion error. A common threshold may be set for all second division areas ARB, or a separate threshold may be set for each second division area ARB.

[0049] The error determination unit 122 may output both the error determination result by the comparison unit CPA and the error determination result by the comparison unit CPB. Alternatively, the error determination unit 122 may determine that the second divided region ARB determined to be an error by at least one of the comparison unit CPA and the comparison unit CPB is an occlusion error, and output the determination result. Alternatively, the error determination unit 122 may determine that the second divided region ARB determined to be an error by both the comparison unit CPA and the comparison unit CPB is an occlusion error, and output the determination result.

[0050] 7 shows a second detailed configuration example of the circuit device 100. In FIG. 7, the error detection circuit 120 includes an area determination unit 121, an error determination unit 122, and an error detection coordinate conversion unit 123.

[0051] The error detection coordinate conversion unit 123 converts the coordinate GZB=(x,y) on the output image data IMB into the coordinate GZA=(u,v) on the input image data IMA. The area determination unit 121 determines the first divided area ARA to which the coordinate GZA on the input image data IMA belongs, based on the coordinates GZA and GZB from the error detection coordinate conversion unit 123, and thereby determines the second divided area ARB to which the coordinate GZB on the output image data IMB belongs. The error determination unit 122 calculates an occlusion determination index for each second divided area ARB based on the area determination result and the output image data IMB, and compares the occlusion determination index with a threshold to determine whether or not each second divided area ARB corresponds to an occlusion error.

[0052] 8 shows a third detailed configuration example of the circuit device 100. In FIG. 8, the error detection circuit 120 includes an area determination unit 121, an error determination unit 122, and an inverse distortion correction unit .

[0053] The inverse distortion correction unit 124 generates second input image data IMA2 by performing inverse distortion correction, which is the inverse of the distortion correction performed by the distortion correction circuit 110, on the output image data IMB. If the inverse distortion correction is a perfect inverse correction, the second input image data IMA2 will match the input image data IMA. However, since the inverse distortion correction may not be a perfect inverse correction due to interpolation processing, etc., it is sufficient that the second input image data IMA2 is approximately identical to the input image data IMA. The area determination unit 121 determines the first divided area ARA to which the coordinate GZA2 on the second input image data IMA2 belongs based on the area determination result and the second input image data IMA2. The error determination unit 122 calculates an occlusion determination index for each first divided area ARA based on the area determination result and the second input image data IMA2, and compares the occlusion determination index with a threshold to determine whether each first divided area ARA corresponds to an occlusion error. By determining whether or not each first divided area ARA corresponds to an occlusion error, it is determined whether or not each second divided area ARB corresponding to each first divided area ARA corresponds to an occlusion error.

[0054] In the above-described embodiment, the error detection circuit 120 obtains, for each second divided area ARB, an occlusion determination index that indicates occlusion due to the display image of the head-up display device 50, based on the output image data IMB of each second divided area ARB. The error detection circuit 120 detects an occlusion error based on the occlusion determination index of each second divided area ARB.

[0055] According to this embodiment, the error detection circuit 120 calculates the occlusion determination index for each second divided area ARB based on the output image data IMB of each second divided area ARB, thereby making it possible to evaluate the background visibility when each second divided area ARB is projected by the HUD. Then, the error detection circuit 120 detects an occlusion error based on the occlusion determination index for each second divided area ARB, making it possible to detect an area in the HUD display where background visibility may be reduced.

[0056] In this embodiment, the error detection circuit 120 obtains brightness information of each second divided area ARB as an occlusion determination index. In the example of Fig. 6, the brightness information is an average luminance value, but is not limited to this and may be information indicating the brightness of the image in the second divided area ARB.

[0057] According to this embodiment, the error detection circuit 120 can determine whether each second divided area ARB corresponds to an occlusion error based on the brightness information of each second divided area ARB. Specifically, the error detection circuit 120 can determine that a bright second divided area among the second divided area group is an occlusion error.

[0058] In this embodiment, the error detection circuit 120 also obtains black pixel ratio information for each second divided area ARB as an occlusion determination index.

[0059] According to this embodiment, the error detection circuit 120 can determine whether each second divided area ARB corresponds to an occlusion error based on the black pixel ratio information of each second divided area ARB. Specifically, the black pixel ratio indicates the ratio of transparent areas when the second divided area ARB is projected by the HUD. The error detection circuit 120 can determine that a second divided area with a high ratio of transparent areas when projected by the HUD is an occlusion error among the group of second divided areas.

[0060] In this embodiment, the error detection circuit 120 detects an occlusion error by comparing the occlusion determination index with a threshold value.

[0061] According to this embodiment, the error detection circuit 120 compares an occlusion determination index indicating the degree of background visibility when each second divided area ARB is projected by the HUD with a threshold value, and can determine that a second divided area ARB whose background visibility is lower than a predetermined value is an occlusion error.

[0062] In this embodiment, the circuit device 100 also includes a threshold register 150 that stores a threshold value for each first divided area ARA. The error detection circuit 120 detects an occlusion error by comparing the occlusion determination index for each second divided area ARB with the threshold value for each first divided area ARA corresponding to each second divided area ARB.

[0063] Since the output image is distorted by distortion correction, the first division area group is set to the undistorted input image. The error detection circuit 120 can determine whether each second division area ARB corresponds to an occlusion error by comparing the occlusion determination index of each second division area ARB with the threshold value of each first division area ARA corresponding to each second division area ARB.

[0064] 3. Fourth detailed configuration example of circuit device 9 shows a fourth detailed configuration example of the circuit device 100. The fourth detailed configuration example can be a combination of any of the first to third detailed configuration examples described above. In FIG. 9, the circuit device 100 includes an input circuit 105, a distortion correction circuit 110, an error detection circuit 120, an output circuit 130, an interface circuit 140, a threshold register 150, a status register 160, and an interrupt signal generation circuit 170.

[0065] The interface circuit 140 is a communication interface that allows the processing device 200 to control the circuit device 100. The interface circuit 140 is a serial interface that complies with the SPI standard or the I2C standard, for example. SPI stands for Serial Peripheral Interface, and I2C stands for Inter-Integrated Circuit.

[0066] The processing device 200 writes a threshold value to be compared with the occlusion determination index to the threshold value register 150 via the interface circuit 140. The processing device 200 sets a threshold value independently for each first division area ARA. The processing device 200 also sets a threshold value for the average luminance value and a threshold value for the black pixel ratio for each first division area ARA. The error detection circuit 120 detects an occlusion error by comparing the occlusion determination index for the second division area ARB with the threshold value set for the first division area ARA corresponding to that second division area ARB.

[0067] The error detection circuit 120 writes information indicating the second divided area ARB in which the occlusion error was detected, among the second divided area group, to the status register 160. The information indicating the second divided area ARB in which the occlusion error was detected may be information indicating the first divided area ARA corresponding to the second divided area ARB in which the occlusion error was detected.

[0068] When the error detection circuit 120 detects an occlusion error, the interrupt signal generation circuit 170 outputs an interrupt signal IRQ to the processing device 200. Specifically, when an occlusion error is detected in one or more second divided areas of the second divided area group, the interrupt signal generation circuit 170 outputs the interrupt signal IRQ to the processing device 200.

[0069] When the processing device 200 receives the interrupt signal IRQ, it performs processing to address the occlusion error. For example, when the processing device 200 receives the interrupt signal IRQ, it may turn off the backlight device of the display device 30 or stop transmitting the input image data IMA. Alternatively, when the processing device 200 receives the interrupt signal IRQ, it may obtain, from the status register 160 via the interface circuit 140, information indicating the second division region ARB in which the occlusion error was detected. The processing device 200 performs processing to address the occlusion error based on the obtained information. For example, the processing device 200 may transmit to the circuit device 100 input image data IMA in which the first division region ARA corresponding to the second division region ARB in which the occlusion error was detected is set to black data. In this case, the error detection circuit 120 does not need to output the occlusion error detection result to the output circuit 130.

[0070] In the present embodiment described above, the circuit device 100 includes the interrupt signal generation circuit 170 that generates an interrupt signal IRQ when the error detection circuit 120 detects an occlusion error.

[0071] According to this embodiment, when an occlusion error is detected by the error detection circuit 120, the interrupt signal generation circuit 170 generates an interrupt signal IRQ, thereby notifying the external processing device 200 that an occlusion error has occurred.

[0072] In this embodiment, the circuit device 100 also includes a status register 160 that stores information indicating the second divided region ARB in which an occlusion error has been detected, among the second divided region group.

[0073] According to this embodiment, when an occlusion error is detected by the error detection circuit 120, the status register 160 can store information indicating the second divided area ARB in which the occlusion error was detected among the second divided areas. For example, by accessing the status register 160, the external processing device 200 can determine in which second divided area ARB the occlusion error occurred, i.e., in which first divided area ARA the occlusion error occurred.

[0074] 4. Second Configuration Example of Display System and Circuit Device 10 shows a second configuration example of a head-up display device 50 and a circuit device 100. The second configuration example can be combined with any of the first to fourth detailed configuration examples described above. In FIG. 10, the head-up display device 50 includes a display device 30, a circuit device 100, a processing device 200, and a camera 400. The circuit device 100 includes an input circuit 105, a distortion correction circuit 110, an error detection circuit 120, an output circuit 130, an interface circuit 140, and a threshold register 150.

[0075] 11 is a diagram illustrating the operation of the head-up display device 50 and the circuit device 100 in the second configuration example. The camera 400 captures an image of the real space that overlaps the HUD display. The processing device 200 receives the camera image of the real space from the camera 400 and sets thresholds for each of the first divided regions based on the camera image.

[0076] Specifically, as shown in the left diagram of FIG. 11, the processing device 200 sets a fourth divided area group corresponding to the first divided area group for the area of ​​the camera image onto which the HUD display is projected. Each fourth divided area of ​​the fourth divided area group is indicated by ARC. The processing device 200 detects a specific object, such as a pedestrian, bicycle, or automobile, from the camera image and identifies a fourth divided area ARC that includes the detected object. In the example of FIG. 11, the processing device 200 identifies the fourth divided areas ARC in the first row, fourth column and the second row, fourth column.

[0077] As shown in the right diagram of FIG. 11, the processing device 200 changes the thresholds of each first divided region ARA based on the region identification results. In the right diagram of FIG. 11, the first divided region ARA corresponding to the fourth divided region ARC determined to include a specific object is indicated by hatching. Where i represents a row and j represents a column, Tij represents the threshold of the first divided region ARA in the i-th row and j-th column. The processing device 200 changes the thresholds T14 and T24 of the first divided region ARA corresponding to the fourth divided region ARC determined to include a specific object in a direction that makes it more likely to be determined as an occlusion error compared to the thresholds of the other first divided regions. Specifically, when Tij is a threshold for the average luminance value, the processing device 200 reduces the thresholds T14 and T24 compared to the other thresholds. When Tij is a threshold for the black pixel ratio, the processing device 200 increases the thresholds T14 and T24 compared to the other thresholds.

[0078] The processing device 200 writes the threshold value Tij into the threshold value register 150 via the interface circuit 140. The error detection circuit 120 detects an occlusion error by comparing the occlusion determination index obtained from the output image data IMB of the second divided area ARB in the i-th row and j-th column in the second divided area group with the threshold value Tij.

[0079] In the above-described embodiment, the head-up display device 50 includes a processing device 200 that controls the circuit device 100, and a display device 30 that projects a display image based on output image data IMB from the circuit device 100. The processing device 200 writes a threshold value for each first divided area ARA into a threshold register 150.

[0080] According to this embodiment, the processing device 200 writes the threshold value of each first divided area ARA into the threshold value register 150, so that the threshold value can be set independently for each first divided area ARA.

[0081] Furthermore, in this embodiment, the processing device 200 changes the threshold value of each first divided area ARA based on a camera image from the camera 400 that captures the real space that overlaps the projected display image.

[0082] According to this embodiment, the processing device 200 can change the threshold value of the first divided area ARA in accordance with the content of the camera image of the real space that overlaps with the area on the HUD display corresponding to that first divided area ARA. This allows the occlusion error determination threshold value to be controlled for each area in accordance with the situation in the real space.

[0083] The circuit device of the present embodiment described above is used in a head-up display device. The circuit device includes a distortion correction circuit and an error detection circuit. The distortion correction circuit performs distortion correction on input image data, which is image data of an input image, and outputs output image data, which is image data of an output image after the distortion correction. When each first divided area of ​​a first divided area group obtained by dividing the input image corresponds to each second divided area of ​​a second divided area group in the output image in terms of distortion correction, the error detection circuit detects an occlusion error in the display image of the head-up display device based on the output image data of each second divided area.

[0084] According to this embodiment, when background visibility is partially reduced in a HUD display, the area where background visibility is reduced can be detected. Specifically, an occlusion error is detected based on the output image data of each second division area, and the area on the HUD display onto which the second division area where the occlusion error was detected is projected is detected as an area where background visibility is reduced. Furthermore, according to this embodiment, since the output image is an image distorted by distortion correction, a first division area group is set in the undistorted input image. According to this embodiment, each first division area of ​​the first division area group obtained by dividing the input image corresponds to each second division area of ​​the second division area group in the output image in terms of distortion correction, and therefore the error detection circuit can detect an occlusion error in each second division area using this correspondence.

[0085] In addition, in this embodiment, the error detection circuit may calculate an occlusion determination index for each second divided area, which indicates occlusion due to the display image of the head-up display device, based on the output image data of each second divided area, and detect an occlusion error based on the occlusion determination index of each second divided area.

[0086] According to this embodiment, the error detection circuit calculates the occlusion determination index for each second divided region based on the output image data of the second divided region, thereby making it possible to evaluate the background visibility when each second divided region is projected by the HUD.The error detection circuit then detects an occlusion error based on the occlusion determination index for each second divided region, making it possible to detect regions in the HUD display where background visibility may be reduced.

[0087] In this embodiment, the error detection circuit may obtain brightness information of each second divided region as an occlusion determination index.

[0088] According to this embodiment, the error detection circuit can determine whether each second divided area corresponds to an occlusion error based on the brightness information of each second divided area. Specifically, the error detection circuit can determine that a bright second divided area among the second divided area group is an occlusion error.

[0089] In this embodiment, the error detection circuit may obtain black pixel ratio information for each second divided region as an occlusion determination index.

[0090] According to this embodiment, the error detection circuit can determine whether each second divided area corresponds to an occlusion error based on the black pixel ratio information of each second divided area. Specifically, the error detection circuit can determine that a second divided area that has a high ratio of transparent areas when projected by the HUD among the group of second divided areas has an occlusion error.

[0091] In this embodiment, the error detection circuit may detect an occlusion error by comparing the occlusion determination index with a threshold value.

[0092] According to this embodiment, the error detection circuit compares an occlusion determination index indicating the degree of background visibility when each second divided area is projected by the HUD with a threshold value, and can determine that a second divided area with background visibility lower than a predetermined level is an occlusion error.

[0093] In this embodiment, the circuit device may include a threshold register that stores a threshold value for each of the first divided regions. The error detection circuit may detect an occlusion error by comparing the occlusion determination index for each of the second divided regions with the threshold value for each of the first divided regions corresponding to each of the second divided regions.

[0094] Since the output image is distorted due to distortion correction, the first divided area group is set in the undistorted input image. The error detection circuit can determine whether each second divided area corresponds to an occlusion error by comparing the occlusion determination index of each second divided area with the threshold value of each first divided area corresponding to each second divided area.

[0095] In this embodiment, the error detection circuit may determine the correspondence between each of the first divided regions and each of the second divided regions based on correspondence information between coordinates on the input image and coordinates on the output image that are associated by distortion correction.

[0096] According to this embodiment, a coordinate transformation between coordinates on the input image and coordinates on the output image is performed in the coordinate transformation for distortion correction. The error detection circuit uses this correspondence to determine to which first divided area the coordinates on the input image corresponding to the coordinates on the output image belong. This allows the first divided area on the input image to correspond to the second divided area on the output image.

[0097] In this embodiment, the circuit device may also include an interrupt signal generation circuit that generates an interrupt signal when an occlusion error is detected by the error detection circuit.

[0098] According to this embodiment, when an occlusion error is detected by the error detection circuit, the interrupt signal generation circuit generates an interrupt signal, thereby notifying an external processing device that an occlusion error has occurred.

[0099] In this embodiment, the circuit device may also include a status register that stores information indicating a second divided region in the second divided region group in which an occlusion error has been detected.

[0100] According to this embodiment, when an occlusion error is detected by the error detection circuit, the status register can store information indicating which second divided area of ​​the second divided area group the occlusion error was detected in. For example, an external processing device can access the status register to determine in which second divided area the occlusion error occurred, i.e., in which first divided area the occlusion error occurred.

[0101] In addition, in this embodiment, the error detection circuit may convert the output image data of the second divided area of ​​the second divided area group that is determined to be an occlusion error into transparent color data that becomes a transparent color in the display image of the head-up display device.

[0102] According to this embodiment, the second divided area determined to be an occlusion error is set to transparent color data, so when the second divided area is projected by the HUD, the area becomes transparent. As a result, the area that may have reduced visibility in the HUD display becomes transparent, ensuring visibility.

[0103] The head-up display device of this embodiment also includes a circuit device described in any one of the above items, a processing device that controls the circuit device, and a display device that projects a display image based on output image data from the circuit device.

[0104] The head-up display device of this embodiment includes the circuit device described above, a processing device that controls the circuit device, and a display device that projects a display image based on image data output from the circuit device. The processing device writes the threshold value of each first division region to a threshold register.

[0105] According to this embodiment, the processing device writes the threshold value for each first division area into the threshold value register, so that the threshold value can be set independently for each first division area.

[0106] In this embodiment, the processing device may change the threshold value of each first divided region based on a camera image taken by a camera that captures the real space that overlaps the projected display image.

[0107] According to this embodiment, the processing device can change the threshold for the first divided region depending on the content of the camera image of the real space that overlaps with the region on the HUD display that corresponds to the first divided region, thereby controlling the threshold for determining occlusion errors for each region depending on the situation in the real space.

[0108] Although the present embodiment has been described in detail above, it will be readily apparent to those skilled in the art that many modifications are possible without substantially departing from the novel features and advantages of the present disclosure. Therefore, all such modifications are intended to be included within the scope of the present disclosure. For example, a term described at least once in the specification or drawings together with a different term having a broader or equivalent meaning may be replaced with that different term anywhere in the specification or drawings. Furthermore, all combinations of the present embodiment and modifications are also included within the scope of the present disclosure. Furthermore, the configurations and operations of the circuit device, head-up display device, processing device, display system, and the like are not limited to those described in the present embodiment, and various modifications are possible. [Explanation of symbols]

[0109] 5...display area, 6...display object, 30...display device, 50...head-up display device, 100...circuit device, 105...input circuit, 110...distortion correction circuit, 112...coordinate counter, 113...coordinate conversion circuit, 114...interpolation circuit, 115...memory circuit, 120...error detection circuit, 121...area determination unit, 122...error determination unit, 123...coordinate conversion unit for error detection, 124...inverse distortion correction unit, 130...output circuit, 140...interface circuit, 150...threshold register, 160...status register, 170...interrupt signal generation circuit, 200...processing device, 400...camera, ARA...first divided area, ARB...second divided area, ARC...fourth divided area, ARH...third divided area, IMA...input image data, IMB...output image data, IRQ...interrupt signal

Claims

1. A circuit device used in a head-up display device, a distortion correction circuit that performs distortion correction on input image data that is image data of an input image, and outputs output image data that is image data of an output image after the distortion correction; an error detection circuit that detects an occlusion error indicating the occurrence of an occlusion in a display image of the head-up display device, based on the output image data of each second divided area, when each first divided area of ​​a first divided area group obtained by dividing the input image corresponds to each second divided area of ​​a second divided area group in the output image in the distortion correction; Including, The error detection circuit A circuit device characterized in that the output image data of the second divided area among the second divided area group that is determined to be an occlusion error is converted into transparent color data that becomes a transparent color in the display image of the head-up display device.

2. 2. The circuit device according to claim 1, The error detection circuit a circuit device that calculates an occlusion determination index for each second divided region based on the output image data of each second divided region, the occlusion determination index indicating the degree to which a real object is obstructed when the display image is projected by the head-up display device, and detects the occlusion error based on the occlusion determination index for each second divided region.

3. 3. The circuit device according to claim 2, The error detection circuit A circuit device characterized in that brightness information of each of the second divided regions is obtained as the occlusion determination index.

4. 4. The circuit device according to claim 2, The error detection circuit A circuit device characterized in that black pixel ratio information of each of the second divided regions is obtained as the occlusion determination index.

5. 5. The circuit device according to claim 2, wherein The error detection circuit The circuit device detects the occlusion error by comparing the occlusion determination index with a threshold value.

6. 6. The circuit device according to claim 5, a threshold register that stores the threshold value set for each of the first divided regions; The error detection circuit A circuit device characterized in that the occlusion error is detected by comparing the occlusion judgment index of each of the second divided regions with the threshold value set for each of the first divided regions corresponding to each of the second divided regions.

7. 7. The circuit arrangement according to claim 1, The distortion correction circuit In the distortion correction, a coordinate conversion is performed between coordinates on the input image and coordinates on the output image; The error detection circuit A circuit device characterized in that it determines which first divided area of ​​the first divided area group each of the second divided areas corresponds to based on correspondence information indicating the correspondence between coordinates on the input image and coordinates on the output image in the coordinate transformation.

8. 8. The circuit arrangement according to claim 1, a circuit device comprising an interrupt signal generating circuit that generates an interrupt signal for a processing device that transmits the input image data to the circuit device to perform processing to deal with the occlusion error when the occlusion error is detected by the error detection circuit.

9. 9. The circuit device according to claim 8, A circuit device comprising: a status register that stores information indicating a second divided area in which an occlusion error has been detected, among the second divided area group.

10. A circuit device used in a head-up display device, a distortion correction circuit that performs distortion correction on input image data that is image data of an input image, and outputs output image data that is image data of an output image after the distortion correction; an error detection circuit that detects an occlusion error indicating the occurrence of an occlusion in a display image of the head-up display device, based on the output image data of each second divided area, when each first divided area of ​​a first divided area group obtained by dividing the input image corresponds to each second divided area of ​​a second divided area group in the output image in the distortion correction; Including, The distortion correction circuit In the distortion correction, a coordinate conversion is performed between coordinates on the input image and coordinates on the output image; The error detection circuit performing area determination to determine which of the first divided areas in the first divided area group each of the second divided areas corresponds to, based on correspondence information indicating correspondence between coordinates on the input image and coordinates on the output image in the coordinate transformation; A circuit device comprising: a circuit for detecting in which area of ​​the display image the occlusion has occurred, based on the result of the area determination and the output image data of each of the second divided areas.

11. A circuit arrangement according to any one of claims 1 to 10; a processing device that controls the circuit device; a display device that projects the display image based on the output image data from the circuit device; A head-up display device comprising:

12. a circuit device according to claim 6; a processing device that controls the circuit device; a display device that projects the display image based on the output image data from the circuit device; Including, The processing device includes: A head-up display device, characterized in that the threshold value set for each of the first divided areas is written to the threshold value register.

13. The head-up display device according to claim 12, The processing device includes: A head-up display device characterized in that the threshold value set for each of the first divided areas is changed based on a camera image from a camera that captures a real space that overlaps the projected display image.