Display image adjustment method and augmented reality display device

Abstract

The application provides a display image adjustment method and an augmented reality display device. The display image adjustment method comprises the following steps. Image data received is converted to a coordinate system of the augmented reality display device to obtain initial coordinate information. Based on the initial coordinate information, an initial image is provided to an effective display area of the augmented reality display device. When an adjustment instruction is received, the initial coordinate information is adjusted in a virtual adjustment coordinate area to obtain adjusted coordinate information. Based on the adjusted coordinate information, an adjusted image is provided to the effective display area of the augmented reality display device. The display image adjustment method and the augmented reality display device provided by the application can adjust the display content of the AR display device according to the needs of users.

Description

TECHNICAL FIELD

[0001] The present application relates to an image display adjustment mechanism, and particularly relates to an image display adjustment method and an augmented reality display device. BACKGROUND

[0002] With the progress of technology, an augmented reality (AR) technology combining a virtual world and a real world scene is developed. The AR technology is a highly visualized interactive way, which can present relevant digital information in a physical environment. In terms of hardware, a carrier combining a processor, a display, a sensor, and an input device can become an AR device. Currently, the AR technology can be realized in an optical projection system, a monitor, a mobile device, a head-mounted display, a head-up display, a computer, and the like.

[0003] In a carrier such as an AR glass, which cannot be provided with a large-size display screen, a user can improve the problem of too small picture content or too small subtitles by magnifying the screen picture, so as to clearly see the picture content. However, after the magnifying operation in the display area, the display area can only display the picture content of a partial area, and cannot further adjust the display content. Therefore, it is inconvenient to watch the content through the small-size display screen in the adjustment of the display picture.

[0004] The background section of this document is included in advance to assist with understanding the application. Therefore, the contents of the background section can include some concepts that are not prior art known to those of ordinary skill in the art. The contents of the background section does not mean that the contents or problems to be solved by one or more embodiments of the application were known at the time the application was filed. SUMMARY

[0005] The present application provides an image display adjustment method and an augmented reality (AR) display device, which can adjust the display content of the augmented reality display device according to the user's needs.

[0006] The image display adjustment method of the present application is suitable for an augmented reality display device, and includes: converting received image data to a coordinate system of the augmented reality display device to obtain initial coordinate information; providing an initial image to an effective display area of the augmented reality display device based on the initial coordinate information; adjusting the initial coordinate information in a virtual adjustment coordinate area to obtain adjusted coordinate information when an adjustment instruction is received, wherein the effective coordinate area corresponds to the effective display area, and the effective coordinate area is located in the virtual adjustment coordinate area; and providing an adjusted image to the effective display area of the augmented reality display device based on the adjusted coordinate information.

[0007] The augmented reality display device of the present application comprises: an image processor configured to receive image data from a signal source; an operation unit for inputting adjustment instructions; a controller coupled to the operation unit and the image processor, the controller being configured to determine whether the operation unit inputs adjustment instructions and transmit the adjustment instructions to the image processor, wherein the image processor is configured to: convert the received image data to a coordinate system of the augmented reality display device to obtain initial coordinate information; based on the initial coordinate information, provide an initial image to an effective display area of the augmented reality display device; when receiving the adjustment instructions, adjust the initial coordinate information in a virtual adjustment coordinate area to obtain adjusted coordinate information, wherein the effective coordinate area corresponds to the effective display area, and the area of the virtual adjustment coordinate area is greater than the area of the effective coordinate area; and based on the adjusted coordinate information, provide an adjusted image to the effective display area of the augmented reality display device.

[0008] Based on the above, the present disclosure can adjust the content of the display area of the augmented reality display device according to the user's needs, move the display image to any position, so that the user can further clearly see the content, or avoid the display image blocking the user's view. BRIEF DESCRIPTION OF DRAWINGS

[0009] Figure 1 is a block diagram of an augmented reality display device according to an embodiment of the present application.

[0010] Figure 2 is a flowchart of a display image adjustment method of an image processor according to an embodiment of the present application.

[0011] Figure 3 is a flowchart of a display image adjustment method of an image processor according to an embodiment of the present application.

[0012] Figure 4A is a schematic diagram of an augmented reality display device according to an embodiment of the present application. Figure 4B is a schematic diagram of an effective display area and a virtual adjustment coordinate area according to an embodiment of the present application.

[0013] Figures 5A-5C is a schematic diagram of an adjusted image according to an embodiment of the present application.

[0014] Figures 6A-6C is a schematic diagram of an adjusted image in an effective display area of an augmented reality display device according to an embodiment of the present application.

[0015] Figures 7A-7C is a schematic diagram of an adjusted image according to an embodiment of the present application.

[0016] Figures 8A-8C is a schematic diagram of an adjusted image in an effective display area of an augmented reality display device according to an embodiment of the present application. Detailed Implementation

[0017] The foregoing and other technical contents, features, and effects of the present invention will be clearly presented in the following detailed description of a preferred embodiment with reference to the accompanying drawings. The directional terms mentioned in the following embodiments, such as up, down, left, right, front, or back, are merely for reference to the accompanying drawings. Therefore, the directional terms used are for illustrative purposes and not for limiting the present invention.

[0018] Figure 1 This is a block diagram of an augmented reality (AR) display device according to an embodiment of the present invention. Please refer to... Figure 1 The augmented reality display device 100 (hereinafter referred to as the AR display device) includes an image processor 110, a controller 120, and an operation unit 130, and may optionally include a storage device 140 and a projection device 150. The image processor 110 is coupled to the controller 120, the storage device 140, and the projection device 150. The controller 120 is coupled to the operation unit 130.

[0019] The image processor 110 can be hardware with computing capabilities (such as chipsets, processors, etc.), software components (such as operating systems, applications, etc.), or a combination of hardware and software components. The image processor 110 may be, for example, a Central Processing Unit (CPU), a Graphics Processing Unit (GPU), or other programmable microprocessors, digital signal processors (DSPs), programmable controllers, application-specific integrated circuits (ASICs), programmable logic devices (PLDs), or other similar devices. The image processor 110 is adapted to receive image data from a signal source (not shown), and to perform coordinate mapping and transformation of the image data. For example, the image processor 110 can adjust the image data to fit the image size within a valid coordinate region (e.g., adjust the image size and aspect ratio, or maintain the image aspect ratio and adjust the image size), and then coordinate map it (e.g., mark the starting point coordinates and dimensions of the image data). The effective coordinate area is located in the virtual adjustment coordinate area, which corresponds to the effective display area of ​​the AR display device 100, that is, the area of ​​the AR display device 100 where the user can view the image.

[0020] The controller 120, for example, is a chip set, judges whether the operation unit 130 has inputted an adjustment instruction, and transmits the adjustment instruction to the image processor 110 after receiving the adjustment instruction from the operation unit 130.

[0021] The operation unit 130 includes at least one button for inputting the adjustment instruction. For example, the buttons can include a zoom adjustment button for zooming the image (zoom in, zoom out) and a moving adjustment button for adjusting the position of the image (up, down, left, right, and position return).

[0022] The storage device 140, for example, is any type of fixed or removable random access memory, read only memory, flash memory, secure digital card, hard disk, or other similar device or combination of these devices. The storage device 140 stores a plurality of program code segments, which are executed by the image processor 110 after being installed, thereby implementing the display image adjustment method.

[0023] The projection device 150 is coupled to the controller 120 and the image processor 110, and is used to project the initial image and the adjusted image.

[0024] The projection device 150 includes a projection element 151, a modulation element 153, and a light source 155. The modulation element 153 is coupled to the projection element 151 and the light source 155. The image processor 110 is coupled to the modulation element 153. The controller 120 is coupled to the light source 155. The projection element 151, for example, is a waveguide element, and the modulation element 153, for example, is a digital micromirror device (DMD). In an embodiment, the AR display device 100, for example, is a smart glass, displays by a projection mode, and projects light to the user's eyes through the projection device 150 to present the image.

[0025] The controller 120 transmits the adjustment instruction to the image processor 110 according to the operation (zoom adjustment, moving adjustment) of the operation unit 130. The image processor 110 adjusts the initial coordinate information according to the adjustment instruction, generates the adjusted image data, and transmits the adjusted image data to the modulation element 153 of the projection device 150. The modulation element 153 modulates the light emitted by the light source 155 based on the adjusted image data to generate the image light beam (i.e., the adjusted image), and then projects the image light beam to the user's eyes through the projection element 151.

[0026] Figure 2 is a flowchart of a display image adjustment method according to an embodiment of the present application. Please refer to Figure 1 and Figure 2The display image adjustment method of the present application is applicable to the augmented reality display device 100. In step S205, the image processor 110 converts the received image data to the coordinate system of the AR display device 100 to obtain initial coordinate information. Specifically, the initial coordinate information is used to indicate the coordinate position of the image data (for example, the starting point coordinate of the image data is (0, 0) and the size of the image data is 100*100). Then, in step S210, the image processor 110 provides an initial image to the effective display area of the AR display device 100 based on the initial coordinate information. Specifically, the image processor 110 generates initial image data based on the initial coordinate information and the received image data, and transmits the initial image data to the modulation element 153 of the projection device 150 to make the initial image appear in the effective display area of the AR display device 100. It is particularly pointed out that the effective display area is the area of the AR display device 100 that allows the user to view the image.

[0027] In step S215, the image processor 110 adjusts the initial coordinate information in the virtual adjustment coordinate area to obtain adjusted coordinate information when receiving the adjustment instruction. Here, the effective coordinate area corresponds to the effective display area of the AR display device 100, and the effective coordinate area is located in the virtual adjustment coordinate area. That is, the area of the effective coordinate area is smaller than that of the virtual adjustment coordinate area. For example, the area of the virtual adjustment coordinate area is N times the area of the effective coordinate area, where 1 < N < 9.

[0028] For example, in an embodiment, the adjustment instruction is a displacement instruction. When the image processor 110 receives the displacement instruction, it displaces the initial image in the virtual adjustment coordinate area to the displacement direction indicated by the displacement instruction to obtain the adjusted coordinate information (for example, the displacement instruction is to displace n units to the right, that is, to move the starting point coordinate from (0, 0) to the position of (n, 0)).

[0029] In an embodiment, the adjustment instruction is a scaling instruction. Here, the zoom-in and zoom-out function of the display image is mainly realized by adjusting the resolution. That is, when the image processor 110 receives the scaling instruction, it performs resolution conversion on the image data to convert the initial coordinate information of the first size (M1* N1) to the adjusted coordinate information of the second size (M2* N2). The center point of the initial coordinate information of the first size (M1* N1) and the center point of the adjusted coordinate information of the second size overlap. For example, when the image processor 110 receives the zoom-in instruction, it reduces the resolution of the image data to zoom in the presented image.

[0030] In an embodiment, the adjustment instruction can also include a displacement instruction and a scaling instruction.

[0031] Afterwards, in step S220, the image processor 110 provides the adjusted image to the active display area of the AR display device 100 based on the adjusted coordinate information. Specifically, the image processor 110 generates adjusted image data based on the adjusted coordinate information and the received image data, and transmits the adjusted image data to the modulating element 153 of the projection device 150 so that the adjusted image is presented in the active display area of the AR display device 100. It is particularly mentioned that if the size of the adjusted image is smaller than the size of the active display area, the blocks in the active display area that are not displayed with the adjusted image are set to be in the see-through state. If the coordinate positions of the adjusted partial image data exceed the active coordinate area, the adjusted image data is generated only with the image data located in the active coordinate area, but the present application is not limited thereto. In other embodiments, the adjusted image data can be generated with all the adjusted image data, and the modulating element 153 only receives the image data corresponding to the active display area or the modulating element 153 only modulates the light emitted by the light source 155 according to the image data corresponding to the active display area to generate the image light beam.

[0032] Figure 3 is a flowchart of a display image adjustment method of an image processor according to an embodiment of the present application. Figure 4A is a schematic diagram of an AR display device according to an embodiment of the present application. Figure 4B is a schematic diagram of an active coordinate area and a virtual adjustment coordinate area according to an embodiment of the present application. In Figure 4A and Figure 4B , the AR display device 100 is an AR glass, which includes left and right lenses 410, 420, and the lenses 410, 420 have active display areas 411, 421, respectively. As for the lens 410, it has an active coordinate area 412 corresponding to the active display area 411 and a virtual adjustment coordinate area VD for adjusting the display image, as shown in Figure 4B . In this embodiment, the virtual adjustment coordinate area VD is 9 times the active coordinate area 412. The lens 420 is also set as the lens 410. It is particularly mentioned that in Figure 4B , the white area is a non-image area, the gray area (such as “A” shown in Figure 4B ) is an image area, which is the same as the general image indication, and in Figure 4A , in the active display areas 411, 421 of the AR glass of the AR display device 100, the gray area is a non-image area (i.e., no image light beam is provided), and the white area is an image area (i.e., an image light beam is provided), so that the gray area of the active display areas 411, 421 of the AR glass of the AR display device 100 can be regarded as a see-through area.

[0033] Please refer to Figure 1 , Figure 3 ,Figure 4A and Figure 4B After the AR display device is activated, the screen is displayed in step S305. Specifically, when the image processor 110 receives the power-on command, it reads the display settings stored in the storage device 140 from the previous power-off of the AR display device 100, and converts the image data to the coordinate system of the AR display device 100 based on the display settings to obtain initial coordinate information, and displays the screen accordingly, that is, provides the screen to the effective display areas 411 and 421 of the AR display device 100.

[0034] Next, in step S310, the image processor 110 detects whether a power-off command has been received. If a power-off command is received, the AR display device 100 is turned off. If no power-off command is received, steps S315 to S325 are executed. The order of steps S315 and S352 is not restricted.

[0035] In step S315, the image processor 110 determines whether it has received a scaling command from the controller 120. Upon receiving the scaling command, in step S325, the image is adjusted. That is, the image processor 110 performs resolution conversion on the image data to convert the initial coordinate information of the first size (M1×N1) into adjusted coordinate information of the second size (M2×N2). Furthermore, based on the adjusted coordinate information, the adjusted image is provided to the effective display area of ​​the AR display device 100. Then, the process returns to step S310.

[0036] If no scaling command is received, in step S320, the image processor 110 determines whether a displacement command has been received from the controller 120. Upon receiving the displacement command, in step S325, the image is adjusted. That is, the image processor 110 displaces the initial coordinate information in the direction specified by the displacement instruction to obtain adjusted coordinate information. Furthermore, based on the adjusted coordinate information, the adjusted image is provided to the effective display area of ​​the AR display device 100. Then, the process returns to step S310. If no displacement command is received, the process returns to step S310 and continues to monitor for a power-off command.

[0037] Figures 5A-5C This is a schematic diagram of an adjusted image according to an embodiment of the present invention. Figures 6A-6C This is a schematic diagram illustrating the adjustment of an image within the effective display area of ​​an AR display device according to an embodiment of the present invention. In this embodiment, only... Figures 5A-5C Example Figures 6A-6C The effective display area 411, the corresponding effective coordinate area 412, and the virtual adjustment coordinate area VD, the effective display area 421 and the effective display area 411 have the same settings, which will not be described again here.

[0038] existFigures 5A-5C and Figures 6A-6C In the embodiment shown, after the AR display device 100 is powered on, as... Figure 6A As shown, the effective display areas 411 and 421 respectively display image data (letter “A”). In the effective display areas 411 and 421, apart from the letter “A”, all other parts of the black area are see-through areas.

[0039] Next, when the user presses the zoom adjustment button to zoom in on the image data, the letter "A" (as shown in the image) in the effective coordinate area 412... Figure 5A As shown, the area will be enlarged based on the center of the effective coordinate region 412 (e.g., Figure 5B (as shown), and provides the adjusted image to the effective display area 411 of the AR display device 100 (e.g. Figure 6B As shown). At this point, when the letter "A" is magnified and exceeds the effective coordinate area 412, the user can move the magnified image data upwards and to the right by pressing the adjustment knob (e.g., Figure 5C (as shown), and provides the adjusted image to the effective display area 411 of the AR display device 100 (e.g. Figure 6C (As shown). In this way, the user can move the displayed content at will and view the details of the displayed content further.

[0040] Figures 7A-7C This is a schematic diagram of an adjusted image according to an embodiment of the present invention. Figures 8A-8C This is a schematic diagram illustrating the adjustment of an image within the effective display area of ​​an AR display device according to an embodiment of the present invention. In this embodiment, only... Figures 7A-7C Example Figures 8A-8C The effective display area 411, the corresponding effective coordinate area 412, and the virtual adjustment coordinate area VD, the effective display area 421 and the effective display area 411 have the same settings, which will not be described again here.

[0041] exist Figures 7A-7C and Figures 8A-8C In the embodiment shown, after the AR display device 100 is powered on, as... Figure 8A As shown, the effective display area 411 displays image data (letter "A"). In the effective display area 411, except for the letter "A", all other parts of the black area are transparent areas.

[0042] If the image obstructs the user's view, the user can press the zoom-out button to zoom out the image data (e.g., Figure 7B As shown), the letter "A" in the effective coordinate region 412 will be scaled down with the center of the effective coordinate region 412 as the reference (e.g., Figure 7BThe adjusted image is provided to the active display area 411 of the AR display device 100 (as shown in FIG. 4B). Figure 8B The user can perform a moving operation on the reduced image data by pressing the moving knob (as shown in FIG. 4C), and the adjusted image is provided to the active display area 411 of the AR display device 100 (as shown in FIG. 4D). Figure 7C The user can perform a moving operation on the reduced image data by pressing the moving knob (as shown in FIG. 4C), and the adjusted image is provided to the active display area 411 of the AR display device 100 (as shown in FIG. 4D). Figure 8C The user can perform a moving operation on the reduced image data by pressing the moving knob (as shown in FIG. 4C), and the adjusted image is provided to the active display area 411 of the AR display device 100 (as shown in FIG. 4D).

[0043] In summary, the AR display device of the present disclosure can enlarge or reduce the display image and move the image position, and can adjust the content of the active display area of the AR display device according to the user's needs, and can move the image to any position to allow the user to see the content more clearly or to avoid the display image from blocking the user's view. Accordingly, the present disclosure can adjust the display image within the virtual adjustment coordinate area without being limited by mechanical components.

[0044] In addition, the AR display device can be further applied in a medical system. When a doctor wears the AR display device (e.g., AR glasses) to perform surgery, the central area of the lens of the AR glasses needs to be maintained in a see-through state because the doctor needs to focus on the surgical object. In this regard, the user can easily adjust the size and position of the image and determine the see-through state of the active display area by using the display image adjustment method of the AR display device 100. In addition, the image data can be replaced with information such as a heartbeat graph or blood pressure of a patient to assist the doctor in performing surgery.

[0045] The above description is only some preferred embodiments of the present disclosure, and does not limit the scope of the present disclosure. Any simple equivalent changes and modifications made according to the claims and the description of the present disclosure are still within the scope of the present disclosure. In addition, any embodiment or claim of the present disclosure does not necessarily achieve all the purposes or advantages or features disclosed in the present disclosure. Furthermore, the abstract and title are only used to assist patent document searching and do not limit the scope of the present disclosure. In addition, the terms "first", "second", etc. mentioned in the specification or claims are only used to name elements or distinguish different embodiments or ranges, and do not limit the upper or lower limit of the number of elements.

[0046] Legend of Reference Numerals

[0047] 100: AR display device

[0048] 110: image processor

[0049] 120: controller

[0050] 130: operation unit

[0051] 140: storage device

[0052] 150: projection device

[0053] 151: projection element

[0054] 153: modulation element

[0055] 155: light source

[0056] 410, 420: lens

[0057] 411, 421: effective display area

[0058] 412: effective coordinate area

[0059] VD: virtual adjustment coordinate area

[0060] S205-S220, S305-S325: step

Claims

1. A method for adjusting displayed images, applicable to augmented reality display devices, characterized in that, The method for adjusting the displayed image includes: Receive power-on command; Based on the display settings stored during the previous shutdown of the augmented reality display device, the received image data is converted to the coordinate system of the augmented reality display device to obtain initial coordinate information; Based on the initial coordinate information, an initial image is provided to the effective display area of ​​the augmented reality display device; Upon receiving an adjustment instruction, the initial coordinate information is adjusted within a virtual adjustment coordinate area to obtain adjusted coordinate information, wherein the effective coordinate area corresponds to the effective display area, and the effective coordinate area is located within the virtual adjustment coordinate area; Based on the adjusted coordinate information, the adjusted image is provided to the effective display area of ​​the augmented reality display device; and The adjusted coordinate information is stored as the display settings.

2. The display image adjustment method according to claim 1, characterized in that, The adjustment command is a scaling command, and the steps of adjusting the initial coordinate information to obtain the adjusted coordinate information include: A resolution transformation is performed on the image data to convert the initial coordinate information of the first size into the adjusted coordinate information of the second size.

3. The display image adjustment method according to claim 1, characterized in that, After the step of adjusting the initial coordinate information to obtain the adjusted coordinate information, the method further includes: If the size of the adjusted image is smaller than the size of the effective display area, the blocks in the effective display area that do not display the adjusted image are set to perspective mode.

4. The display image adjustment method according to claim 1, characterized in that, The area of ​​the virtual adjustment coordinate region is N times the area of ​​the effective coordinate region, 1 <N≦9。 5. The display image adjustment method according to claim 1, characterized in that, The adjustment command is a displacement command, and the step of adjusting the initial coordinate information to obtain the adjusted coordinate information further includes: The initial coordinate information is shifted in one direction to obtain the adjusted coordinate information.

6. The display image adjustment method according to claim 2, characterized in that, The center point of the first dimension overlaps with the center point of the second dimension.

7. An augmented reality display device, characterized in that, The augmented reality display device includes: An image processor configured to receive image data from a signal source; The operation unit is used to input adjustment commands; A controller, coupled to the operating unit and the image processor, is configured to determine whether the operating unit has input the adjustment command and to transmit the adjustment command to the image processor. The image processor is configured to: Receive power-on command; Based on the display settings stored during the previous shutdown of the augmented reality display device, the received image data is converted to the coordinate system of the augmented reality display device to obtain initial coordinate information; Based on the initial coordinate information, an initial image is provided to the effective display area of ​​the augmented reality display device; Upon receiving the adjustment instruction, the initial coordinate information is adjusted within a virtual adjustment coordinate area to obtain the adjusted coordinate information, wherein the area of ​​the virtual adjustment coordinate area is larger than the area of ​​the effective display area. Based on the adjusted coordinate information, the adjusted image is provided to the effective display area of ​​the augmented reality display device; and The adjusted coordinate information is stored as the display settings.

8. The augmented reality display device according to claim 7, characterized in that, The augmented reality display device further includes a projection device coupled to the controller for projecting the initial image and the adjusted image.

Images