Output image adjustment device and projection device

Abstract

To provide an output image adjustment device capable of allowing the whole of an image for calibration projected to a projection object to be included in the photographing range of an imaging device for imaging the image for calibration and to provide a projection device.SOLUTION: An output image adjustment device 22 includes: a projection part 32 for projecting an image for calibration 40 including a group of dot images representing a coordinate in a first direction and in a second direction crossing the first direction to a front wind shield 16; an imaging part 30 for imaging the image for calibration 40 projected on the front wind shield 16; and an image adjustment part 28 for performing adjustment for allowing the whole of the image for calibration 40 projected by the projection part 32 to be included in an imaging range 50 by the imaging part 30 on the basis of image data 44 obtained by imaging by the imaging part 30.SELECTED DRAWING: Figure 8

Description

[Technical Field]

[0001] The present disclosure relates to an output image adjustment device and a projection device. [Background technology]

[0002] The image projection system projects one image from two image projection means onto an image projection area, and includes: means for calculating a first projective transformation matrix from the checker intersection coordinates of a photographed checker sheet image; means for calculating a second projective transformation matrix using a spatial code image; means for projectively transforming the checker sheet image using the second projective transformation matrix, projecting the projected checker sheet image from the image projection means, photographing the checker sheet image projected onto the entire image projection area, detecting checker intersection coordinates in the photographed checker sheet image, calculating an average error between the detected checker intersection coordinates and the ideal values ​​of the checker intersection coordinates, and calculating a translation matrix using the calculated average error as a thrust; and means for correcting image data to be output to a predetermined image projection means using the second projective transformation matrix and the translation matrix. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Japanese Patent Application Laid-Open No. 2015-201760 Summary of the Invention [Problem to be solved by the invention]

[0004] The present disclosure aims to provide an output image adjustment device and a projection device that include the entire calibration image projected onto a projection target within the imaging range of an imaging device that captures the calibration image. [Means for solving the problem]

[0005] An output image adjustment device of one aspect of the present disclosure includes a projection unit that projects a calibration image including a group of points representing coordinates in a first direction and a second direction intersecting the first direction onto a projection target, an imaging unit that captures the calibration image projected onto the projection target, and an image adjustment unit that adjusts the calibration image projected by the projection unit to include the entire calibration image projected by the projection unit within the imaging range of the imaging unit based on the imaging data captured by the imaging unit.

[0006] In an output image adjustment device according to one aspect of the present disclosure, when a projection unit projects a calibration image onto a projection target, the image adjustment unit adjusts the image so that the entire calibration image is included in the imaging range. Therefore, with the output image adjustment device according to this aspect, the calibration image can be included in the imaging range. [Effects of the Invention]

[0007] According to the present disclosure, it is possible to provide an output image adjustment device and a projection device that include the entire calibration image projected onto a projection target within the imaging range of an imaging device that captures the calibration image. [Brief explanation of the drawings]

[0008] [Figure 1] 1A and 1B are diagrams illustrating how a projection device according to an embodiment of the present disclosure is used. [Figure 2] FIG. 1 is a block diagram illustrating a configuration of a projection device according to an embodiment of the present disclosure. [Figure 3] 10A and 10B are diagrams illustrating a calibration image created by an image creation unit included in the projection device according to the embodiment of the present disclosure. [Figure 4] 1A and 1B are diagrams illustrating a projected image projected onto a projection surface according to an embodiment of the present disclosure. [Figure 5] 10A and 10B are diagrams illustrating how the coordinates of the center points of dot images are calculated from imaging data captured by an imaging unit included in the projection device according to the embodiment of the present disclosure. [Figure 6] 10A and 10B are diagrams illustrating a state in which a dot image that straddles the boundary of an imaging area is projected onto a projection image projected onto a projection surface according to an embodiment of the present disclosure. [Figure 7]10A and 10B are diagrams illustrating imaging data obtained by an imaging device capturing an image projected onto a projection surface according to an embodiment of the present disclosure. [Figure 8] FIG. 3 is a flowchart illustrating an operation procedure of a CPU according to the first embodiment of the present disclosure. [Figure 9] 10A and 10B are diagrams illustrating a state in which a dot image is located inside a boundary in imaging data captured by an imaging unit included in a projection device according to an embodiment of the present disclosure. [Figure 10] 10A and 10B are diagrams illustrating a state in which dot images have moved inward in a calibration image formed by an image forming unit of a projection device according to an embodiment of the present disclosure. DETAILED DESCRIPTION OF THE INVENTION

[0009] An example of an embodiment of the present disclosure will be described below with reference to the drawings. In each drawing, the same or equivalent components and parts are designated by the same reference numerals. Furthermore, the dimensional proportions in the drawings are exaggerated for the sake of explanation and may differ from the actual proportions.

[0010] In addition, in each drawing, the arrow X direction is an example of a first direction of the present disclosure in image data to be described later, and the arrow Y direction is an example of a second direction of the present disclosure in image data to be described later. Note that in each drawing, the arrows X and Y are perpendicular to each other, but this is not limited thereto as long as the arrow X direction and the arrow Y direction are directions that intersect each other.

[0011] (composition) 1 is a diagram illustrating a state in which a projection device 20 according to the present disclosure is used. As an example, the projection device 20 according to the present disclosure is a so-called head-up display 10 that is placed on a dashboard 14 of an automobile and projects onto a front windshield 16.

[0012] 1, the projection device 20 is disposed in front of the steering wheel 18 as seen from the driver of the automobile, and projects an image toward a projection area 34 on the front windshield 16. The projection area 34 is set in front of the driver as shown in FIG. 1, and the driver can recognize necessary information as needed by reading the image projected on the front windshield 16.

[0013] As an example, the front windshield 16 is curved in a direction widening forward as seen from the driver. However, the curved shape of the front windshield 16 is not necessarily symmetrical in the longitudinal direction (vertical direction) and lateral direction (horizontal direction and depth direction) as seen from the driver, and may be distorted in either direction.

[0014] 2 is a block diagram illustrating the configuration of a projection device 20 according to the present disclosure. The projection device 20 according to the present disclosure includes an output image adjustment device 22 and an image creation unit 26. The output image adjustment device 22 also includes a control unit 24, a projection unit 32, an imaging unit 30, and an image adjustment unit 28.

[0015] The control unit 24 includes a CPU 24A (Central Processing Unit), which is an example of a processor, and a RAM 24B (Random Access Memory) used as a temporary work area for the CPU 24A. The control unit 24 also includes a non-volatile memory 24C that stores a control program that causes the control unit 24 to function, and an input / output interface (I / O) 24E. The CPU 24A, RAM 24B, non-volatile memory 24C, and input / output interface 24E are connected to each other via a bus 24D.

[0016] Furthermore, the CPU 24A of the control unit 24 reads the control program from the nonvolatile memory 24C and executes the overall control of the projection device 20 that the control unit 24 is responsible for.

[0017] The nonvolatile memory 24C is an example of a storage device that maintains stored information even if the power supplied to it is cut off, and is, for example, a semiconductor memory, but a hard disk may also be used.

[0018] Furthermore, the projection unit 32, the imaging unit 30, the image adjustment unit 28, and the image creation unit 26 are connected to the input / output interface 24E.

[0019] The projection unit 32 is a device that projects image data created by the image creation unit 26 onto a projection area 34 on the front windshield 16 based on the operating procedure of a program executed by the CPU 24A. The projection unit 32 is, for example, a dot-matrix liquid crystal projector that transmits white light emitted from a light source lamp through a liquid crystal panel, with each pixel having sub-pixels that transmit red, green, and blue colors. Each sub-pixel of the liquid crystal panel adjusts the brightness (luminance) of the transmitted light to project a color image onto the projection area 34. Image data will be described later.

[0020] The imaging unit 30 is a device that captures image data projected onto the projection area 34 by the projection unit 32 based on the operating procedure of a program executed by the CPU 24A. The imaging unit 30 is also an example of an imaging device in this embodiment.

[0021] Image adjustment unit 28 adjusts the image data created by image creation unit 26 based on the operating procedure of the program executed by CPU 24A. More specifically, image adjustment unit 28 is capable of adjusting the coordinates of pixels included in the image data, and adjusts the image projected by projection unit 32 by adjusting the coordinate values ​​of the image included in the image data based on the operating procedure executed by CPU 24A.

[0022] The image creating unit 26 creates image data based on the operation procedure of the program executed by the CPU 24 A. The image data created by the image creating unit 26 includes a calibration image, which will be described later.

[0023] 2, the image creation unit 26 and the image adjustment unit 28 may be executed by the same device or by different devices as long as they are capable of creating image data and adjusting the image data, respectively. In other words, the image creation unit 26 and the image adjustment unit 28 may both be functions of a program executed by the CPU 24A. In other words, the output image adjustment device 22 may be a function of another part of the projection device 20 excluding the image adjustment unit 28.

[0024] Next, with reference to FIG. 3, a calibration image created by the output image adjustment device 22 according to the first embodiment of the present disclosure and the output image adjustment device 22 will be described.

[0025] [Embodiment] (composition) The image creation unit 26 according to this embodiment creates image data for the calibration image shown in FIG. 3. The calibration image includes a group of dot images, which are points representing coordinates in the horizontal direction and the vertical direction intersecting the horizontal direction. In the following description, a dot image refers to an image of a point composed of several pixels (for example, two vertically and two horizontally, for a total of four pixels). As shown in FIG. 3, each dot image is arranged in a grid pattern with a predetermined interval (for example, 20 pixels) between adjacent dot images in both the vertical and horizontal directions. In other words, the grid-like dot images D are an example of a "group of points" in this embodiment. The projection area 34 of the front windshield 16 is an example of a projection target in this disclosure. The horizontal and vertical directions are examples of a first direction and a second direction in this disclosure.

[0026] The projection area 34 of the front windshield 16, which is the projection target, or the optical system of the projection device 20 may be distorted when viewed from the projection device 20. In this case, the image projected from the projection device 20 may change from a rectangularly arranged shape as shown in Fig. 3 to a distorted shape as shown in Fig. 4. In other words, the dot image projected from the projection device 20 may be projected at a position in the projection area 34 that is different from the expected coordinates.

[0027] As shown in FIG. 4, dot image D' projected at a position on the projection surface different from the expected coordinates corresponds to dot image D indicating each coordinate shown in FIG. 3. Here, as an example, single quotation marks "'" are added to the symbols of each dot image shown in FIG. 4 that correspond to the symbols of each dot image shown in FIG. 3. For example, the fact that dot image D1 included in calibration image 40 in FIG. 3 is projected at a position different from the expected position when projected onto projection area 34 is shown as D1' in FIG. 4. Similarly, D2 and D3 are shown as D2' and D3', and the same applies to other dot images not shown with symbols in FIGS. 3 and 4. Furthermore, as shown in FIGS. 6, 7, etc., when dot images are not to be distinguished, the symbol of the dot image is shown as "D" in the calibration image and as "D'" in the projected image.

[0028] In such a case, since it is known that the coordinates of each dot image D and the dot image D' correspond to each other as shown in Figures 3 and 4, the CPU 24A corrects the output image, thereby correcting the image displayed in the projection area 34. More specifically, the CPU 24A captures the distorted dot image D' as shown in Figure 4 by the imaging unit 30 when projected onto the projection area 34, and converts the image of the output image based on the coordinates of each dot image D so that the image of Figure 3 is output.

[0029] In this embodiment, the CPU recognizes the center of a dot image included in the imaging data as the center of the dot image. More specifically, as shown in Fig. 5, the CPU calculates the minimum circumscribing circle MC (the smallest circle that contacts the area of ​​the dot image from the outside) of each dot image from the imaging data, and determines the coordinates of the center point CP of the minimum circumscribing circle MC as the coordinates of the captured dot image.

[0030] Furthermore, the symbols of the dot images in the image data are enclosed in double quotation marks "".

[0031] (Measurement coordinate deviation) However, the inventors of the present disclosure have found through their studies that even if the calibration image 40 projected onto the projection area 34 by the imaging unit 30 is captured, there is a possibility that the coordinates of the dot image D may not be obtained accurately.

[0032] Specifically, as shown in FIG. 6, when a dot image D is projected across a boundary line 52 between an imaging range 50 of the imaging unit 30 and the outside of the imaging range 50, the portion of the dot image D' that extends beyond the boundary line 52 is not captured. In other words, the portion of the dot image D' that extends beyond the boundary line is processed as if it "did not exist" in the captured image data 44. Therefore, when a dot image D' is projected across the boundary line 52 as shown in FIG. 6, the coordinates of the center point CP of the dot image D" are recognized as being shifted from the coordinate position of the original center point CP indicated by the dashed-dot line to the coordinate position of the shifted center point MCP indicated by the dashed-dot line, as shown in FIG.

[0033] Here, in the projection device 20 according to this embodiment, the coordinates of the image data to be projected are adjusted based on the imaging data 44 captured by the imaging unit 30. The operation of the output image adjustment device 22 according to the embodiment of the present disclosure and the procedure for adjusting the calibration image 40 will be described with specific reference to Fig. 8 .

[0034] (Projected image adjustment procedure) The CPU 24A in this embodiment reads the program stored in the nonvolatile memory 24C and executes the procedure shown in FIG.

[0035] First, in step S102, the CPU 24A projects the calibration image 40 onto the projection area 34. More specifically, as shown in Fig. 3, the CPU 24A projects the calibration image 40 including dot images D arranged in a grid pattern onto the projection area 34. Then, the CPU 24A proceeds to step S104.

[0036] Next, in step S104, the CPU 24A uses the imaging unit 30 to capture the calibration image 40 projected onto the projection area 34. More specifically, as shown in FIG. 4, the CPU 24A captures a dot image D arranged in a grid pattern, which is estimated to have been projected distorted onto the projection area 34, and creates captured data 44. Then, the CPU 24A proceeds to step S106.

[0037] Next, in step S106, the CPU 24A calculates the number of dot images D' included in the captured data 44 and compares it with the number of dot images D included in the calibration image 40 projected in step S102. If the CPU 24A makes a positive determination in step S106, the process proceeds to step S110. On the other hand, if the CPU 24A makes a negative determination in step S106, the process proceeds to step S108.

[0038] Next, in step S108, the CPU 24A reduces the overall size of the calibration image 40. More specifically, the CPU 24A reduces the image area of ​​the calibration image 40 projected in step S102 while maintaining the center position. Then, the CPU 24A proceeds to step S110.

[0039] Furthermore, in step S110, the CPU 24A determines whether or not any dot image D" is in contact with the contour 54 of the imaging data 44, among the dot images D' arranged in a grid pattern in the imaging data 44. More specifically, the CPU 24A determines whether or not any dot image D" is in contact with the contour 54 (outer periphery) of the imaging data 44, as shown in FIG. 6. Then, if the CPU 24A makes a positive determination in step S110, that is, if it is determined that there is a dot image D" in contact with the contour 54 of the imaging data 44, as shown in FIG. 6, the CPU 24A proceeds to step S112. On the other hand, if the CPU 24A makes a negative determination in step S110, that is, if it is determined that all of the dot images D" included in the imaging data 44 are located with a gap G from the contour 54, as shown in FIG. 9, the CPU 24A proceeds to step S114.

[0040] Next, in step S112, the CPU 24A moves the dot images D of the calibration image 40 that correspond to the dot images D″ that are in contact with the contour 54 of the imaging data 44, detected in step S110, by a predetermined amount toward the center of the calibration image 40. More specifically, as shown in FIG. 10, the CPU 24A moves the outer dot images D of the calibration image 40 toward the inside of the calibration image 40 (toward the lower right side in FIG. 10). Then, the CPU 24A proceeds to step S110.

[0041] Then, in step S114, the CPU 24A calculates the amount of correction for the image to be projected, as shown in FIGS. 3 and 4, from the coordinates of the dot image D'' included in the image data 44 captured in step S104.

[0042] In the output image adjustment device 22 of this embodiment, when the projection unit 32 projects the calibration image 40 onto the projection area 34, the image adjustment unit 28 adjusts the image so that the entire calibration image 40 is included in the imaging range 50. Therefore, according to the output image adjustment device 22 of this embodiment, the calibration image 40 can be included in the imaging range 50.

[0043] Furthermore, when a dot image D included in the calibration image 40 falls outside the imaging range 50, the output image adjustment device 22 of this embodiment moves the dot image D inward from a boundary line 52 between the imaging range 50 and the outside of the imaging range 50. Therefore, according to the output image adjustment device 22 of this embodiment, when any of the dot images D included in the calibration image 40 falls outside the imaging range 50, it is easier to determine the imaging range 50 based on the state of the projection area 34 compared to when the imaging range 50 is expanded.

[0044] Furthermore, when the output image adjustment device 22 of this embodiment determines that the number of dot images D" included in the imaging data 44 is smaller than the number of points included in the calibration image 40, it moves the dot images D inward from the boundary line 52 between the imaging range 50 and the outside of the imaging range 50. Therefore, according to the output image adjustment device 22 of this embodiment, it is easier to enlarge the imaging range 50 compared to when adjustment is made to always include the calibration image 40 in the imaging range 50.

[0045] Furthermore, if the dot image D is in contact with the boundary line 52, even if a part of the outermost point of the calibration image 40 is located outside the boundary line 52, it will not be included in the imaging range 50, making it easy to misidentify the shape of the image of the outermost point of the calibration image 40.

[0046] On the other hand, the output image adjustment device 22 of this embodiment moves the dot image D included in the calibration image 40 to the inside of the boundary line 52 between the imaging range 50 and the outside of the imaging range 50, making it easier to recognize the shape of the outermost dot image D' in the calibration image 40. Therefore, with the output image adjustment device 22 of this embodiment, it is less likely to misidentify the coordinates of the dot image D' than when the dot image D in the calibration image 40 remains in contact with the boundary line 52.

[0047] Furthermore, when an outer dot image D included in the calibration image 40 abuts a boundary line 52 between the imaging range 50 and the outside of the imaging range 50, the output image adjustment device 22 of this embodiment moves inward only the dot image D that abuts the boundary line 52. Therefore, according to the output image adjustment device 22 of this embodiment, it is easier to enlarge the imaging range 50 compared to when all dot images D included in the calibration image 40 are moved inward.

[0048] Furthermore, according to the projection device 20 of this embodiment, the entire calibration image 40 used to calibrate the projected image is included in the imaging range 50 .

[0049] (Variation) In the above description, the CPU 24A narrows the intervals between the dot images D in step S108 to include the dot image D' in the imaging range 50, but the operation of the CPU 24A in this embodiment is not limited to this. For example, in step S108, the CPU 24A may operate to expand the imaging range 50 outward instead of narrowing the intervals between the dot images D.

[0050] Furthermore, in the above description, when the CPU 24A determines in step S106 that the number of dot images D included in the imaging data 44 is smaller than the number of dot images D included in the calibration image 40, the CPU 24A reduces the calibration image 40 inward. However, the operation of the CPU 24A according to this embodiment is not limited to this. For example, the CPU 24A may reduce the image to be projected to a size that assumes that the calibration image 40 is included in the imaging range 50, without counting the number of dot images D included in the imaging data 44.

[0051] Furthermore, in the above description, the CPU 24A moves the dot images D included in the calibration image 40 inward in step S112 until they are separated from the boundary line 52 between the imaging range 50 and the outside of the imaging range 50. However, the operation of the CPU 24A according to this embodiment is not limited to this. For example, the CPU 24A may move the dot images D inward until the outermost dot image D among the dot images D included in the imaging data 44 comes into contact with the boundary line 52.

[0052] In the above description, the CPU 24A moves only the outermost dot images D inward among the dot images D included in the calibration image 40 in step S112, but the operation of the CPU 24A according to this embodiment is not limited to this. For example, the CPU 24A may reduce the calibration image 40 to move all of the dot images D included in the imaging data 44 inward.

[0053] In these modified examples, the same functions and effects as those of this embodiment can be obtained.

[0054] In the above embodiments, the processing performed by the CPU 24A after reading the software (program) may be performed by various processors other than the CPU 24A. Examples of such processors include programmable logic devices (PLDs) whose circuit configuration can be changed after fabrication, such as field-programmable gate arrays (FPGAs), and dedicated electrical circuits, such as application-specific integrated circuits (ASICs), which are processors with circuit configurations specifically designed to perform specific processing. The processing may be performed by one of these processors, or by a combination of two or more processors of the same or different types (e.g., multiple FPGAs, or a combination of a CPU and an FPGA). The hardware structure of these processors is, more specifically, an electrical circuit that combines circuit elements such as semiconductor devices.

[0055] In addition, in each of the above embodiments, the processing program is described as being pre-stored (installed) in the non-volatile memory 24C, but this is not limiting. The program may be provided in a form stored in a non-transitory storage medium such as a CD-ROM (Compact Disk Read Only Memory), a DVD-ROM (Digital Versatile Disk Read Only Memory), or a USB (Universal Serial Bus) memory. The program may also be downloaded from an external device via a network.

[0056] The above describes an embodiment of the present disclosure with reference to the accompanying drawings. However, it is clear that a person with ordinary knowledge in the field of technology to which the present disclosure pertains can conceive of various modifications or application examples within the scope of the technical ideas set forth in the claims, and it is understood that these also naturally fall within the technical scope of the present disclosure.

[0057] Further preferred aspects of the present disclosure will be described below.

[0058] (Appendix 1) a projection unit that projects a calibration image including a group of points representing coordinates in a first direction and a second direction intersecting the first direction onto a projection target; an imaging unit that captures the calibration image projected onto a projection target; an image adjustment unit that adjusts the calibration image projected by the projection unit to include the entire calibration image in an imaging range of the imaging unit, based on imaging data captured by the imaging unit; An output image adjustment device comprising:

[0059] (Appendix 2) the image adjustment unit moves a group of points included in the calibration image from a boundary line between the imaging range and an outside of the imaging range to an inside; 2. The output image adjustment device of claim 1.

[0060] (Appendix 3) when the image adjustment unit determines that the number of points included in the captured data is smaller than the number of points included in the calibration image projected by the projection unit, the image adjustment unit moves the group of points inward from the boundary line; 3. The output image adjustment device of claim 2.

[0061] (Appendix 4) the image adjustment unit moves a group of points included in the calibration image inward, away from a boundary line between the imaging range and an outside of the imaging range; 4. An output image adjustment device according to claim 3.

[0062] (Appendix 5) the image adjustment unit moves only groups of points located at both ends in the first direction and the second direction, out of groups of points included in the calibration image, inward and away from the boundary line; 5. An output image adjustment device according to claim 4.

[0063] (Appendix 6) An output image adjustment device according to any one of Supplementary Note 1 to Supplementary Note 5; an image creation unit that creates the calibration image; A projection device comprising: [Explanation of symbols]

[0064] 10 Head-up display 14 Dashboard 16 Front windshield 18 Steering Wheel 20 Projection device 22 Output image adjustment device 24 Control Unit 24A CPU 24B RAM 24C non-volatile memory 24D Bus 24E Input / Output Interface 26 Image Creation Department 28 Image adjustment section 30 Imaging unit 32 Projection section 34 Projection area 40 proof images 42 Projection image 44 Imaging data 50 imaging range 52 Borderline 54 Contour

Claims

1. a projection unit that projects a calibration image including a group of points representing coordinates in a first direction and a second direction intersecting the first direction onto a projection target; an imaging unit that captures the calibration image projected onto a projection target; an image adjustment unit that adjusts the calibration image projected by the projection unit to include the entire calibration image in an imaging range of the imaging unit, based on imaging data captured by the imaging unit; An output image adjustment device comprising:

2. the image adjustment unit moves a group of points included in the calibration image from a boundary line between the imaging range and an outside of the imaging range to an inside; 2. The output image adjustment device according to claim 1.

3. when the image adjustment unit determines that the number of points included in the captured data is smaller than the number of points included in the calibration image projected by the projection unit, the image adjustment unit moves the group of points inward from the boundary line; 3. The output image adjustment device according to claim 2.

4. the image adjustment unit moves a group of points included in the calibration image inward, away from a boundary line between the imaging range and an outside of the imaging range; 4. The output image adjustment device according to claim 3.

5. the image adjustment unit moves only groups of points located at both ends in the first direction and the second direction, out of groups of points included in the calibration image, inward and away from the boundary line; 5. The output image adjustment device according to claim 4.

6. An output image adjustment device according to any one of claims 1 to 5; an image creation unit that creates the calibration image; A projection device comprising:

Images