Circuitry and head-up displays

Abstract

To provide a circuit device or the like that can reduce the buffer size of the image buffer used in distortion correction. [Solution] The circuit device 100 includes an image selection circuit 120 that selects selected image data IMS, which is image data of a narrower selection area than the input image, from input image data IMA, which is image data of the input image; a buffer memory 130 that stores the selected image data IMS; and a distortion correction circuit 140 that corrects the distortion of the input image and outputs output image data IMB, which is image data of the output image. The selection area is a plurality of areas or a polygonal area. When the input coordinates (X,Y) of the input image corresponding to the output coordinates (U,V) of the output image belong to the selection area, the distortion correction circuit 140 generates pixel data of the output image data IMB at the output coordinates (U,V) from the selected image data IMS stored in the buffer memory 130.

Description

Technical Field

[0001] The present invention relates to a circuit device, a head-up display, and the like.

Background Art

[0002] Patent Document 1 discloses an image processing apparatus including an image segmentation unit, an image editing buffer, and an image editing unit. When sequentially cutting out a plurality of divided images from an input image, the image segmentation unit provides an overlap region for the divided images adjacent to each other. The image editing buffer sequentially stores the plurality of divided images, and the image editing unit sequentially performs image editing processing on the plurality of divided images. Thereby, the capacity of the image editing buffer is reduced.

Prior Art Documents

Patent Documents

[0003]

Patent Document 1

Summary of the Invention

Problems to be Solved by the Invention

[0004] When there is an area in an image where a display object is not displayed, the buffer also stores that area, resulting in a problem that the memory capacity of the buffer increases. For example, in a head-up display, a display object is only displayed in a part of the display area, and most of it is a transparent area (black area as an image). By buffering such a transparent area, the memory capacity becomes large. In Patent Document 1, a line buffer of multiple lines is used, but since the input image is directly divided and stored in the line buffer, areas that do not need to be stored are also buffered.

Means for Solving the Problems

[0005] One aspect of the present disclosure relates to a circuit device that includes an image selection circuit that selects selected image data, which is image data of a narrower selected region than the input image, from input image data, which is image data of an input image; a buffer memory for storing the selected image data; and a distortion correction circuit that corrects the distortion of the input image and outputs output image data, which is image data of an output image, wherein the selected region is a plurality of regions or a polygonal region, and the distortion correction circuit generates pixel data of the output image at the output coordinates from the selected image data stored in the buffer memory when the input coordinates of the input image corresponding to the output coordinates of the output image belong to the selected region.

[0006] Another aspect of this disclosure relates to a head-up display including the above-described circuit device and a display device that projects the output image using a projection optical system. [Brief explanation of the drawing]

[0007] [Figure 1] An example of the input image before distortion correction and the output image after distortion correction. [Figure 2] An example of a technique that encloses the displayed object in a rectangle and stores only the image data within that rectangle in buffer memory. [Figure 3] An example configuration of a head-up display including the circuit device of this embodiment. [Figure 4] An example where the selected area consists of multiple regions. [Figure 5] An example where the selected area consists of multiple regions. [Figure 6] An example where the selected area is a polygonal region. [Figure 7] A first detailed configuration example of a distortion correction circuit. [Figure 8] Examples of selection regions and examples of selected images stored in buffer memory. [Figure 9] An example of the processing flow of a distortion correction circuit. [Figure 10] A diagram illustrating pixel interpolation at the boundary of a selected region. [Figure 11]A second detailed configuration example of the distortion correction circuit. [Modes for carrying out the invention]

[0008] Preferred embodiments of this disclosure will be described in detail below. The embodiments described below are not intended to unduly limit the scope of the claims, and not all configurations described in these embodiments are necessarily essential. For example, the following example shows the application of the memory-saving technique of this disclosure to a head-up display. However, the memory-saving technique of this disclosure is applicable to any image processing involving distortion correction, such as head-mounted displays, projectors, or curved displays. In the following, a head-up display may be abbreviated as a HUD.

[0009] Figure 1 shows an example of an input image before distortion correction and an output image after distortion correction. Distortion correction is the process of applying an image distortion that is the opposite of the image distortion that occurs when the image is projected onto the HUD screen. As a result, the distortion caused by distortion correction and the distortion caused by projection cancel each other out, resulting in an image display with no distortion or reduced distortion. Image distortion caused by projection includes image distortion due to the curved surface of the projection surface such as a screen, image distortion due to the projection optical system, or image distortion including both.

[0010] As shown in Figure 1, in distortion correction, the pixel coordinates of the output image are counted, for example, in a raster scan. The arrows in the output image in Figure 1 show the counting of the pixel coordinates of a single horizontal line. When the pixel coordinates of the output image are (U,V), (U,V) are converted to the coordinates (X,Y) of the input image based on the distortion correction table. Then, the pixel data of coordinates (U,V) in the output image is constructed based on the pixel data of coordinates (X,Y) in the input image. As (U,V) is scanned along one line of the output image, (X,Y) moves curvilinearly in the input image as shown by the arrows. This process is repeated until the last horizontal line of one frame of the output image, resulting in the distortion-corrected output image.

[0011] Thus, distortion correction constructs the pixel data of the output image while referencing the input image, requiring the input image to be temporarily stored in buffer memory. However, in a HUD, most of the screen is a transparent display where the background is visible through the screen, and the displayed object exists only in a part of the screen. Therefore, if a frame memory that stores the same number of pixels as the input image, or a line buffer that stores the same number of horizontal pixels as the input image, is used, a large portion of the memory will be filled with black data, resulting in wasted memory capacity. Note that while the transparent display is black data as image data, in Figure 1 the displayed object is shown in black. The same applies below.

[0012] As shown by the dotted rectangle in Figure 2, one possible method is to surround the display object with a single rectangle and store only the image data within that rectangle in buffer memory. However, the rectangle may not efficiently surround the display object, and a large amount of black data area may be included within the rectangle. This can occur when the display objects are far apart, as in Example 1, or when the width of the display objects is uneven, as in Example 2.

[0013] Figure 3 shows an example configuration of a head-up display 500 including the circuit device 100 of this embodiment. The head-up display 500 includes the circuit device 100, a display device 200, and a processing device 300.

[0014] The processing unit 300 transmits the input image data (IMA), which is the image data of the input image, to the circuit device 100. The processing unit 300 is a so-called SoC, which is a processor such as a CPU or microcomputer. SoC stands for System on Chip. CPU stands for Central Processing Unit.

[0015] The circuit device 100 includes an input interface circuit 110, an image selection circuit 120, a buffer memory 130, a distortion correction circuit 140, an output interface circuit 150, and a memory circuit 160. The circuit device 100 is, for example, an integrated circuit device in which a plurality of circuit elements are integrated on a semiconductor substrate.

[0016] The input interface circuit 110 receives input image data IMA from the processing device 300. The input interface circuit 110 may be an interface circuit for various image communication standards. As an example, it is a receiving circuit such as LVDS, DVI, DisplayPort, GMSL, or GVIF. LVDS is an abbreviation for Low voltage differential signaling, DVI is an abbreviation for Digital Visual Interface, GMSL is Gigabit Multimedia Serial Link, and GVIF is an abbreviation for Gigabit Video InterFace.

[0017] The memory circuit 160 stores region setting information 161 and a distortion correction table 162. The memory circuit 160 may include a non-volatile memory such as an EEPROM or OTP memory, a volatile memory such as an SRAM or DRAM, or a register using a flip-flop circuit or the like. The region setting information 161 and the distortion correction table 162 may be stored in different types of memory circuits. For example, the region setting information 161 may be stored in a register, and the distortion correction table 162 may be stored in a volatile memory or a non-volatile memory. The region setting information 161 may be written into the memory circuit 160 from the processing device 300 via an interface not shown. Alternatively, when the memory circuit 160 is a non-volatile memory, the region setting information 161 may be written into the memory circuit 160 in advance. The same applies to the distortion correction table 162.

[0018] The image selection circuit 120 selects selection image data IMS, which is the image data of the selection area, from the input image data IMA based on the area setting information 161. The selection area is an area narrower than the entire input image. Specifically, at least one of the following conditions may be satisfied: the number of horizontal pixels in the selection area is less than the number of horizontal pixels in the input image, and the number of vertical pixels in the selection area is less than the number of vertical pixels in the input image. The image selection circuit 120 writes the selection image data IMS into the buffer memory 130. The selection area is a plurality of areas or a polygonal area surrounding the display object in the input image. The display object is, for example, a character, symbol, figure, icon, picture, or photograph. Taking an in-vehicle HUD as an example, the display object is, for example, a vehicle speed display, a display indicating the state of the vehicle, a direction indication display, a warning light, an AR display, a traffic sign, or a navigation display. AR is the abbreviation of Augmented Reality. The display indicating the state of the vehicle is, for example, a display indicating the position of the shift lever, the fuel level, the coolant temperature, the interior temperature, the exterior temperature, the air conditioner setting, or the audio setting. The polygon is specifically a polygon having three or more vertices, excluding a rectangle in which all four vertices are right angles. All vertices of the polygon may be convex angles, or some vertices may be concave angles. A convex angle is an angle with an interior angle less than 180 degrees, and a concave angle is an angle with an interior angle greater than 180 degrees.

[0019] Figures 4 and 5 show examples where the selection area is a plurality of areas. In Example 1 of Figure 4, the input image includes display objects DBA and DBB arranged horizontally. The rectangular area ARA surrounding the display object DBA and the rectangular area ARB surrounding the display object DBB are the selection areas. The rectangular areas ARA and ARB do not overlap each other. In Example 2 of Figure 5, the input image includes display objects DBC and DBA arranged vertically. The rectangular area ARC surrounding the display object DBC and the rectangular area ARA surrounding the display object DBA are the selection areas. The rectangular areas ARC and ARA do not overlap each other. Note that the selection area may include three or more areas. Also, the shape of each area is not limited to a rectangle.

[0020] Figure 6 shows an example where the selection area is a polygonal region. In Example 3 of Figure 6, the input image includes vertically aligned display objects DBC and DBA. The selection area is the polygonal region ARD that surrounds display objects DBC and DBA. In this example, the polygonal region ARD is an octagon with six convex angles and two concave angles. Note that the interior angles of the convex angles of the polygon are not limited to 90 degrees, and the interior angles of the concave angles are not limited to 270 degrees.

[0021] The region setting information 161 is information that sets the position and shape of the selected region in the input image. The region setting information 161 may be the coordinates of all vertices of the region. Alternatively, in the case of a rectangular region, the region setting information 161 may be the coordinates of the reference position, the horizontal width, and the horizontal width. The reference position may be any vertex of the rectangle or the center point.

[0022] The buffer memory 130 temporarily stores the selected image data IMS. The buffer memory 130 may be a line buffer that stores image data for multiple scan lines, or a frame memory that stores image data for one frame. The buffer memory 130 may be a volatile memory such as SRAM or DRAM.

[0023] The distortion correction circuit 140 performs distortion correction on the input image based on the distortion correction table 162 and outputs output image data IMB, which is the image data of the output image. At this time, based on the region setting information 161, the distortion correction circuit 140 generates output image data IMB for the region corresponding to the selected image in the output image from the selected image data IMS, and sets the output image data IMB for the remaining regions to predetermined color data. The predetermined color data is color data that becomes transparent in the HUD display, for example, black data. The distortion correction table 162 is a table that associates the input coordinates (X,Y) of the input image with the output coordinates (U,V) of the output image. The distortion correction table 162 is also called the warp parameter.

[0024] The output interface circuit 150 transmits the output image data IMB to the display device 200. The output interface circuit 150 may be an interface circuit for various image communication standards, but examples include transmission circuits such as LVDS, DVI, DisplayPort, GMSL, or GVIF.

[0025] The image selection circuit 120 and the distortion correction circuit 140 are logic circuits. The image selection circuit 120 and the distortion correction circuit 140 may each be configured as separate circuits, or they may be configured as an integrated circuit using automatic placement and routing or the like. Furthermore, some or all of these logic circuits may be implemented by a processor such as a DSP. DSP stands for Digital Signal Processor. In this case, a program or instruction set describing the function of each circuit is stored in memory, and the processor executes this program or instruction set to realize the function of each circuit.

[0026] The display device 200 displays a virtual image in the user's field of view based on the output image data IMB received from the circuit device 100. The display device 200 includes a display controller, a display driver, an image display device, and an optical system. However, the configuration of the display device 200 is not limited thereto; for example, the circuit device 100 may incorporate the function of a display controller.

[0027] The display controller transmits image data to the display driver and controls the display timing based on the received output image data (IMB). The display driver drives the image display device based on the image data and display timing control from the display controller, and the image display device displays the image corresponding to the output image data (IMB). The optical system includes a reflector and the like, and projects the image displayed by the image display device onto a screen. The screen can be any transparent projection surface having a projection surface that reflects the projected light. For example, the screen is the windscreen of a mobile device on which a HUD is mounted. The image display device is, for example, a liquid crystal display panel and a backlight device. Alternatively, the image display device may be a laser light source, a mirror that reflects the laser, and an actuator that drives the mirror to scan the laser. Alternatively, the image display device may be a digital mirror device including a laser light source, an array of micro-mirrors, and an actuator that drives each micro-mirror.

[0028] Figure 7 shows a first detailed configuration example of the distortion correction circuit 140. The distortion correction circuit 140 includes a pixel interpolation unit 141, a fill unit 142, an address conversion unit 143, a coordinate correction unit 144, a warp processing unit 145, and a coordinate counter 146. The operation of each unit will be explained below using Figures 8 and 9. Note that in Figures 8 and 9, the coordinate (X,Y)=(100,100) is written as (X:100 Y:100).

[0029] Figure 8 shows an example of a selected region and an example of a selected image stored in buffer memory. Assume that the input image has 800 horizontal pixels and 400 vertical pixels, and that two regions, ARE and ARF, are set as selected regions in the input image. The reference point of each region is the top-left vertex. The reference point of region ARE in the input image is coordinate (X:100 Y:100), with 100 horizontal pixels and 100 vertical pixels. The reference point of region ARF in the input image is coordinate (X:500 Y:100), with 100 horizontal pixels and 100 vertical pixels. The selected image stored in buffer memory 130 has 100 + 100 = 200 horizontal pixels and 100 vertical pixels. The coordinates of the selected image in buffer memory 130 are denoted as (Xs, Ys). Within buffer memory 130, the reference point for region ARE is coordinate (Xs:0 Ys:0), and the reference point for region ARF is coordinate (Xs:100 Ys:0).

[0030] Figure 9 shows an example of the processing flow of the distortion correction circuit 140. In step S1, the distortion correction circuit 140 performs coordinate counting and warp processing. The coordinate counter 146 outputs the output coordinates (U,V) based on the count. Specifically, the coordinate counter 146 outputs (U,V)=(0,0) at the beginning of the frame, and increments the coordinates with each loop from S8, resetting the horizontal coordinate U and vertical coordinate V when (U,V) reaches the last pixel of the frame. In each loop increment, the coordinate counter 146 first increments the horizontal coordinate U, resets U when it reaches the number of horizontal pixels, and increments the vertical coordinate V, repeating this until the last pixel of the frame. The warp processing unit 145 converts the output coordinates (U,V) to input coordinates (X,Y) by referring to the distortion correction table 162.

[0031] In steps S2 to S6, the coordinate correction unit 144 corrects the input coordinates (X,Y) to the coordinates on the selected image in the buffer memory 130. The coordinates on the selected image in the buffer memory 130 will be called the post-selection coordinates (Xs,Ys). The following explanation will use the example in Figure 8.

[0032] In step S2, the coordinate correction unit 144 determines whether (X,Y) is within the range of (X:100 Y:100) to (X:200 Y:200), that is, whether it belongs to the region ARE. If the coordinate correction unit 144 determines that (X,Y) belongs to the region ARE, in step S3 the selected coordinates are set to (Xs,Ys)=(X-100,Y-100) and the process proceeds to step S7. That is, the coordinate correction unit 144 shifts the reference point of the region ARE (X:100 Y:100) to (Xs:0 Ys:0). If the coordinate correction unit 144 determines in step S2 that (X,Y) does not belong to the region ARE, the process proceeds to step S4.

[0033] In step S4, the coordinate correction unit 144 determines whether (X,Y) is within the range of (X:500 Y:100) to (X:600 Y:200), that is, whether it belongs to the ARF region. If the coordinate correction unit 144 determines that (X,Y) belongs to the ARF region, in step S5 it finds the coordinates (Xs',Ys')=(X-500,Y-100). After that, the coordinate correction unit 144 sets the selected coordinates to (Xs,Ys)=(Xs'+100,Ys'+0) and proceeds to step S7. That is, the coordinate correction unit 144 shifts the reference point of the ARF region from (X:500 Y:100) to (Xs':0 Ys':0), and then further shifts it to (Xs:100 Ys:0). If the coordinate correction unit 144 determines in step S4 that (X,Y) does not belong to the ARF region, it determines in step S6 that (X,Y) is outside the selected region and proceeds to step S7.

[0034] In step S7, the distortion correction circuit 140 generates pixel data for the output coordinates (U,V). If the process proceeds from step S3 or S5 to step S7, the address translation unit 143 translates (Xs,Ys) into an address in the buffer memory 130. Specifically, the address translation unit 143 outputs an address that specifies multiple pixels surrounding (Xs,Ys) in the input image. The pixel interpolation unit 141 reads the pixel data of multiple pixels from the corresponding address in the buffer memory 130 and generates the pixel data for (U,V) of the output image by performing pixel interpolation using that pixel data. If the process proceeds from step S6 to step S7, the fill unit 142 converts the pixel data for (U,V) of the output image into predetermined color data. The predetermined color data is, for example, black data. The output image data IMB is composed of the pixel data output by the pixel interpolation unit 141 and the fill unit 142. In step S8, the distortion correction circuit 140 determines whether all coordinates of the output coordinates (U,V) have been counted. The distortion correction circuit 140 terminates processing if all coordinates have been counted, and returns to step S1 if not all coordinates have been counted.

[0035] It should be noted that the method of combining the selected regions and storing them in the buffer memory 130 is not limited to the example in Figure 8, and various other methods are possible. For example, in Figure 8, the vertical coordinates of the reference points of the two regions in the input image are aligned at Y=100, Y=100, but the vertical coordinates of the reference points may differ, such as Y=100, Y=200. In that case, for example, the vertical coordinates of the reference points of the two regions may be shifted to Ys=0, Ys=100 while maintaining the difference, and the selected regions may be stored in the buffer memory 130. Alternatively, when the two regions are arranged vertically in the input image, the two regions may be aligned vertically and combined before being stored in the buffer memory 130. In this case, the horizontal coordinates of the reference points of the two regions may be aligned to Xs=0, and the selected regions may be stored in the buffer memory 130. Furthermore, if the selected region is a polygon, the polygonal region may be divided into multiple regions, which may then be recombined and stored in the buffer memory 130. For example, the region ARD in Figure 6 may be divided into two rectangular regions, one above the other. Assuming that the upper rectangular region has 100 x 100 pixels and the lower rectangular region has 200 x 100 pixels, for example, the reference point of the upper rectangular region within the buffer memory 130 may be (Xs:0 Ys:0) and the reference point of the lower rectangular region may be (Xs:0 Ys:100).

[0036] Figure 10 illustrates pixel interpolation at the boundary of the selected region. The region outside the selected region in the input image is referred to as the invalid region. Here, we use the ARE region in the input image of Figure 8 as an example, but the same applies to the ARF region.

[0037] The coordinate correction unit 144 selects multiple pixels around the input coordinate (X,Y) if it determines that the input coordinate (X,Y) belongs to the region ARE. Figure 10 shows an example in which pixels P1 to P16 in a 4x4 area are selected, but pixels in an nxm area may also be selected. Each of n and m is an integer of 2 or more. Based on the region setting information 161, the coordinate correction unit 144 excludes pixels P1, P2, P5, P6, P9, P10, P13, and P14 from pixels P1 to P16 that belong to the invalid region, and leaves pixels P3, P4, P7, P8, P11, P12, P15, and P16 that belong to the region ARE. The coordinate correction unit 144 converts the input coordinate (X,Y) of the remaining pixels to the selected coordinate (Xs,Ys). The address conversion unit 143 converts the selected coordinate (Xs,Ys) to an address. The pixel interpolation unit 141 reads the pixel data of pixels P3, P4, P7, P8, P11, P12, P15, and P16 belonging to region ARE based on the address, performs pixel interpolation, and generates pixel data for output coordinates (U,V).

[0038] Figure 11 shows a second detailed configuration example of the distortion correction circuit 140. The following will mainly describe the differences from the first detailed configuration example in Figure 7. The distortion correction circuit 140 further includes a table selection unit 147.

[0039] The distortion correction table 162 includes multiple tables corresponding to multiple correction regions. Here, we show an example in which the distortion correction table 162 includes a first table TB1 corresponding to a first correction region and a second table TB2 corresponding to a second correction region. The first and second correction regions are regions set on the output image. These correction regions may be defined as regions separate from the selected region, or the correction region may be the selected region.

[0040] The table selection unit 147 selects the first table TB1 if the output coordinates (U,V) output by the coordinate counter 146 belong to the first correction region, and selects the second table TB2 if the output coordinates (U,V) belong to the second correction region. The warp processing unit 145 uses the table selected by the table selection unit 147 to convert the output coordinates (U,V) to input coordinates (X,Y).

[0041] For example, if the HUD has a first projection optical system and a second projection optical system, the area projected by the first projection optical system may be the first correction area, and the area projected by the second projection optical system may be the second correction area.

[0042] Alternatively, the area where AR objects are displayed may be the first correction area, and the area where non-AR objects are displayed may be the second correction area. In this case, vibration correction may be performed on the first table TB1 to make the AR objects track real-world objects. Vibration of a moving body equipped with a HUD causes a discrepancy between the position of the AR objects projected by the HUD and the position of real-world objects seen by the user. Vibration correction corrects this positional discrepancy. In vibration correction, the distortion correction circuit 140 calculates the amount of the above-mentioned positional discrepancy based on information from an acceleration sensor, gyro sensor, LiDAR, or camera. The distortion correction circuit 140 corrects the first table TB1 so that the AR objects are shifted by the calculated discrepancy amount, and then corrects the distortion of the first correction area using the corrected first table TB1. In this example, the first and second correction areas may be selected areas.

[0043] In this embodiment, the circuit device 100 includes an image selection circuit 120, a buffer memory 130, and a distortion correction circuit 140. The image selection circuit 120 selects selected image data IMS, which is image data of a selection region narrower than the input image, from the input image data IMA, which is image data of the input image. The buffer memory 130 stores the selected image data IMS. The distortion correction circuit 140 corrects the distortion of the input image and outputs output image data IMB, which is image data of the output image. The selection region is a plurality of regions or a polygonal region. If the input coordinates (X,Y) of the input image corresponding to the output coordinates (U,V) of the output image belong to the selection region, the distortion correction circuit 140 generates pixel data of the output image data IMB at the output coordinates (U,V) from the selected image data IMS stored in the buffer memory 130.

[0044] According to this embodiment, since the selected image data IMS, which is image data of a selected area narrower than the input image, is stored in the buffer memory 130, the buffer size for distortion correction is reduced compared to when the input image is buffered directly. As a result, the chip size of the circuit device 100 is reduced, or the cost of the circuit device 100 is reduced.

[0045] In the example in Figure 4, the selected region corresponds to multiple regions ARA and ARB; in the example in Figure 5, it corresponds to multiple regions ARA and ARC; and in the example in Figure 6, it corresponds to the polygonal region ARD. However, as mentioned above, the selected region is not limited to these.

[0046] In this embodiment, if the input coordinates (X,Y) do not belong to the selected region, the distortion correction circuit 140 converts the pixel data at the output coordinates (U,V) into predetermined color data. The predetermined color data may be, for example, color data that becomes transparent when displayed by the display device 200.

[0047] According to this embodiment, pixel data for the output image data IMB is generated even outside the selected area, so it is sufficient to buffer the image data of the selected area in the buffer memory 130. Furthermore, since the area outside the selected area is a transparent area, it is sufficient to convert the pixel data of that area to predetermined color data.

[0048] In this embodiment, the selection region may include a first region and a second region as multiple regions. Hereinafter, using Figure 8 as an example, we assume that the first region is region ARE and the second region is region ARF. The image selection circuit 120 selects the first image data of the first region ARE and the second image data of the second region ARF as the selected image data IMS and stores them in the buffer memory 130. When the input coordinates (X,Y) belong to the first region ARE, the distortion correction circuit 140 generates pixel data at the output coordinates (U,V) from the first image data stored in the buffer memory 130. When the input coordinates (X,Y) belong to the second region ARF, the distortion correction circuit 140 generates pixel data at the output coordinates (U,V) from the second image data stored in the buffer memory 130.

[0049] According to this embodiment, when the selected region consists of multiple regions, the output image data IMB is generated from the image data of the region to which the input coordinates (X,Y) corresponding to the output coordinates (U,V) belong. This ensures that distortion correction is appropriately performed on the image data of each region of the selected region stored in the buffer memory 130.

[0050] As explained in S2-S6 of Figures 8 and 9, the image selection circuit 120 converts the input coordinates (X,Y) to the post-selection coordinates (U,V) in the selection region formed by combining the first region ARE and the second region ARF, and stores the selected image data IMS in the buffer memory 130. If the input coordinates (X,Y) belong to the first region ARE, the distortion correction circuit 140 converts the input coordinates (X,Y) to the post-selection coordinates (Xs,Ys) of the first region ARE, and references the first image data stored in the buffer memory 130 based on the post-selection coordinates (Xs,Ys). If the input coordinates (X,Y) belong to the second region ARF, the distortion correction circuit 140 converts the input coordinates (X,Y) to the post-selection coordinates (Xs,Ys) of the second region ARF, and references the second image data stored in the buffer memory 130 based on the post-selection coordinates (Xs,Ys).

[0051] According to this embodiment, the input coordinates (X,Y) are converted to the selected coordinates (Xs,Ys) in the selected image stored in the buffer memory 130, so that pixel data corresponding to the input coordinates (X,Y) is appropriately obtained from the image data stored in the buffer memory 130.

[0052] In this embodiment, the circuit device 100 also includes a memory circuit 160 that stores region setting information 161 for setting the selection region. The distortion correction circuit 140 determines whether the input coordinates (X,Y) belong to the selection region based on the region setting information 161.

[0053] According to this embodiment, when the display area of ​​an object on the screen is known, the memory circuit 160 can store area setting information 161 that selects that display area. Then, the distortion correction circuit 140 performs distortion correction based on the area setting information 161, thereby obtaining output image data IMB in which the displayed object is appropriately distortion-corrected, and saving memory capacity of the buffer memory 130.

[0054] In this embodiment, the input image includes multiple display objects. The selection region is a group of regions, each containing one of the multiple display objects, or a polygonal region encompassing multiple display objects.

[0055] According to this embodiment, memory capacity is saved by selecting the display area for the displayed object and not storing the area without a displayed object in the buffer memory 130.

[0056] In the example in Figure 4, the input image contains multiple display objects DBA and DBB, and the selection area consists of multiple regions ARA and ARB, with each region containing a display object. In the example in Figure 5, the input image contains multiple display objects DBA and DBC, and the selection area consists of multiple regions ARA and ARC, with each region containing a display object. In the example in Figure 6, the input image contains multiple display objects DBA and DBC, and the selection area is a polygonal region ARD that contains multiple display objects DBA and DBC. However, as mentioned above, the display objects and selection areas are not limited to these.

[0057] As explained in Figure 10, the distortion correction circuit 140 reads the pixel data of multiple pixels P1 to P16 surrounding the input coordinates (X,Y) from the buffer memory 130, and interpolates the pixel data of the multiple pixels P1 to P16 to obtain the pixel data of the output image data IMB corresponding to the output coordinates (U,V). At this time, if the input coordinates (X,Y) belong to the selected region, and some of the multiple pixels P1 to P16 do not belong to the selected region, the distortion correction circuit 140 performs interpolation using the pixel data of pixels P3, P4, P7, P8, P11, P12, P15, and P16 that belong to the selected region among the multiple pixels P1 to P16.

[0058] According to this embodiment, even when the input coordinates (X,Y) are near the boundary of the selected region, pixel interpolation can be performed by using the pixel data of the selected region stored in the buffer memory 130 and excluding pixel data outside the selected region that is not stored in the buffer memory 130.

[0059] In this embodiment, the output interface circuit 150 outputs output image data IMB to the display device 200, which projects the output image using a projection optical system. The distortion correction circuit 140 corrects image distortion caused by distortion of the projection optical system or distortion of the projection surface.

[0060] As explained in Figure 1, etc., a buffer memory 130 is required to buffer the input image when correcting image distortion caused by distortion of the projection optical system or distortion of the projection surface. According to this embodiment, a selected image, which is a part of the input image, is buffered, so the memory capacity of the buffer memory 130 is saved.

[0061] In this embodiment, the head-up display 500 includes one of the above-mentioned circuit devices 100 and a display device 200 that projects an output image using a projection optical system.

[0062] Although this embodiment has been described in detail above, it will be readily apparent to those skilled in the art that many modifications are possible without substantially departing from the novelty and effects of this disclosure. Therefore, all such modifications are included within the scope of this disclosure. For example, any term that appears at least once in the specification or drawings together with a broader or synonymous term may be replaced with that different term anywhere in the specification or drawings. Furthermore, all combinations of this embodiment and its modifications are also included within the scope of this disclosure. In addition, the configuration and operation of the input interface circuit, image selection circuit, buffer memory, distortion correction circuit, output interface circuit, memory circuit, circuit device, processing device, display device, and head-up display are not limited to those described in this embodiment, and various modifications are possible. [Explanation of Symbols]

[0063] 100...Circuit device, 110...Input interface circuit, 120...Image selection circuit, 130...Buffer memory, 140...Distortion correction circuit, 141...Pixel interpolation unit, 142...Fill unit, 143...Address conversion unit, 144...Coordinate correction unit, 145...Warp processing unit, 146...Coordinate counter, 147...Table selection unit, 150...Output interface circuit, 160...Memory circuit, 161...Region setting information, 162...Distortion correction table, 200...Display device, 300...Processing device, 500...Head-up display, ARA~ARF...Region, DBA~DBC...Display object, IMA...Input image data, IMB...Output image data, IMS...Selected image data, (X,Y)...Input side coordinates, (Xs,Ys)...Post-selection coordinates, (U,V)...Output side coordinates

Claims

1. An image selection circuit that selects selected image data, which is image data of a narrower selection region than the input image, from input image data, which is image data of the input image. A buffer memory for storing the selected image data, A distortion correction circuit that corrects the distortion of the input image and outputs output image data, which is the image data of the output image, Includes, The aforementioned selection region is a combination of regions or a polygonal region. The aforementioned distortion correction circuit is A circuit device characterized in that, if the input coordinates of the input image corresponding to the output coordinates of the output image belong to the selected region, pixel data of the output image data at the output coordinates is generated from the selected image data stored in the buffer memory.

2. In the circuit device described in claim 1, The aforementioned distortion correction circuit is A circuit device characterized in that, if the input coordinates do not belong to the selected region, the pixel data in the output coordinates is converted to predetermined color data.

3. In the circuit device described in claim 1, The selected region includes a first region and a second region as the plurality of regions, The image selection circuit selects the first image data from the first region and the second image data from the second region as the selected image data and stores them in the buffer memory. The aforementioned distortion correction circuit is If the input coordinates belong to the first region, the pixel data at the output coordinates is generated from the first image data stored in the buffer memory. A circuit device characterized in that, when the input coordinates belong to the second region, the pixel data at the output coordinates is generated from the second image data stored in the buffer memory.

4. In the circuit device described in claim 3, The aforementioned image selection circuit is The input coordinates are converted into post-selection coordinates in the selected region formed by combining the first region and the second region, and the selected image data is stored in the buffer memory. The aforementioned distortion correction circuit is If the input coordinates belong to the first region, the input coordinates are converted to the selected coordinates of the first region, and the first image data stored in the buffer memory is referenced based on the selected coordinates. A circuit device characterized in that, if the input coordinates belong to the second region, the input coordinates are converted to the selected coordinates of the second region, and the second image data stored in the buffer memory is referenced based on the selected coordinates.

5. In the circuit device described in claim 1, Includes a storage circuit that stores region setting information for setting the aforementioned selected region, The aforementioned distortion correction circuit is A circuit device characterized by determining whether the input coordinates belong to the selected region based on the region setting information.

6. In the circuit device described in claim 1, The aforementioned input image includes multiple display objects, The aforementioned selection region is Each region includes each of the multiple displays, or A circuit device characterized by being a polygonal region that encloses the plurality of displays.

7. In the circuit device described in claim 1, The aforementioned distortion correction circuit is The pixel data of multiple pixels surrounding the input coordinates is read from the buffer memory, and the pixel data of the multiple pixels is interpolated to obtain the pixel data of the output image data corresponding to the output coordinates. A circuit device characterized in that, if the input coordinates belong to the selected region and some of the plurality of pixels do not belong to the selected region, the interpolation process is performed using the pixel data of the pixels that belong to the selected region among the plurality of pixels.

8. In the circuit device described in claim 1, The output interface circuit is, The output image data is output to a display device that projects the output image using a projection optical system. The aforementioned distortion correction circuit is A circuit device characterized by correcting image distortion caused by distortion of the projection optical system or distortion of the projection surface.

9. A circuit device according to any one of claims 1 to 8, A display device that projects the output image using a projection optical system, A head-up display characterized by including the following.

Images