Display correction method and device for augmented reality and augmented reality glasses

Abstract

The invention relates to a display correction method and device for augmented reality and augmented reality glasses, and is suitable for the augmented reality glasses, and the method comprises the steps: obtaining an original display image of the augmented reality glasses and corresponding first pixel-level information; obtaining a target environment image of the augmented reality glasses; obtaining corresponding second pixel-level information based on the target environment image; based on the first pixel-level information and the second pixel-level information, pixel-level adaptive adjustment is carried out on the original display image to obtain a target display image, the problem of display effect deviation caused by ambient light superposition in augmented reality glasses is solved, and the display effect of the augmented reality glasses can be improved through corresponding adjustment of pixels. A display image observed by a user through augmented reality is accurate and stable, and the visual experience of the user is improved.

Description

Technical Field

[0001] The present application relates to the field of augmented reality technology, and in particular to a display correction method, device and augmented reality glasses for augmented reality. Background Art

[0002] Augmented Reality (AR) is an innovative technology designed to seamlessly integrate virtual information into the real world, expanding the boundaries of our perception. This technology cleverly combines cutting-edge technologies such as multimedia processing, real-time tracking, advanced display technology, and precision sensors to overlay and complement virtual objects with the real environment, creating a new sensory experience that transcends traditional visuals. AR display is a key component of this technology. AR display systems typically use a semi-transparent display to overlay the displayed content with the real scene behind it.

[0003] However, existing augmented reality devices still have the following problems: after the ambient light is superimposed on the original display image information, the final displayed image will have contrast shift or color cast, resulting in a reduced viewing experience. Summary of the Invention

[0004] In this embodiment, a display correction method, device, and augmented reality glasses are provided to solve the problem of display effect deviation caused by the superposition of ambient light when the augmented reality glasses are used.

[0005] In a first aspect, this embodiment provides a display correction method for augmented reality, applicable to augmented reality glasses, the method comprising:

[0006] Acquire an original display image of the augmented reality glasses and corresponding first pixel-level information;

[0007] Acquire a target environment image based on the field of view of the augmented reality glasses; and obtain corresponding second pixel-level information based on the target environment image;

[0008] Based on the first pixel-level information and the second pixel-level information, pixel-level adaptive adjustment is performed on the original display image to obtain a target display image.

[0009] In some embodiments, acquiring a target environment image based on the field of view of the augmented reality glasses includes:

[0010] Acquire a target pixel of the original display image;

[0011] Acquiring an initial environment image captured by the augmented reality glasses;

[0012] Taking the field of view range of the augmented reality glasses and the target pixels as target values, the field of view and pixels of the initial environment image are cut to obtain a target environment image.

[0013] In some embodiments, performing pixel-level adaptive adjustment on the original display image based on the first pixel-level information and the second pixel-level information to obtain a target display image includes:

[0014] Inputting the first pixel-level information and the second pixel-level information into a preset display correction model in a pixel-pairing manner, and outputting pixel-level display adjustment information;

[0015] Based on the display adjustment information, each pixel in the original display image is adjusted to obtain a target display image.

[0016] In some embodiments, the first pixel level information includes first brightness information; the second pixel level information includes second brightness information; the display adjustment information includes brightness adjustment information;

[0017] The process of generating the brightness adjustment information by the display correction model includes:

[0018] Multiplying the second brightness information by the ambient light transmittance of the augmented reality glasses to obtain intermediate brightness information;

[0019] Superimposing the intermediate brightness information with the first brightness information to obtain a first superimposed result;

[0020] Calculating a first deviation value between the first superposition result and each pixel of the first brightness information;

[0021] The brightness adjustment information is obtained based on the first deviation value.

[0022] In some of these embodiments, the first pixel-level information includes first color information; the second pixel-level information includes second color information;

[0023] The step of inputting the first pixel-level information and the second pixel-level information into a preset display correction model in a pixel-pairing manner to output pixel-level display adjustment information includes: inputting the first color information and the second color information into a pre-trained deep learning neural network model in a pixel-pairing manner to obtain color adjustment information;

[0024] The training data of the deep learning neural network model includes input data and output data, the input data includes image color information and environmental color information, and the output data includes target color adjustment information.

[0025] The first pixel level information further includes first color information; the second pixel level information further includes second color information; the display adjustment information further includes color adjustment information;

[0026] The process of generating the color adjustment information by the display correction model includes:

[0027] superimposing the second color information with the first color information to obtain a second superimposed result;

[0028] The color adjustment information is obtained based on the second superposition result.

[0029] In some embodiments, adjusting each pixel in the original display image based on the display adjustment information to obtain a target display image includes:

[0030] Based on the color adjustment information and the brightness adjustment information, each pixel in the original display image is adjusted to obtain a target display image.

[0031] In some embodiments, the method further comprises:

[0032] Obtaining preset system parameters, wherein the system parameters include at least one of ambient light transmittance, color temperature requirement, and color gamut standard;

[0033] The corresponding display correction model is acquired based on the system parameters.

[0034] In a second aspect, this embodiment provides a display correction device for augmented reality, suitable for augmented reality glasses, the device comprising:

[0035] an original display data acquisition module, configured to acquire an original display image of the augmented reality glasses and corresponding first pixel-level information;

[0036] An environmental data acquisition module, configured to acquire a target environmental image based on the field of view of the augmented reality glasses; and obtain corresponding second pixel-level information based on the target environmental image;

[0037] The display correction module is configured to perform pixel-level adaptive adjustment on the original display image based on the first pixel-level information and the second pixel-level information to obtain a target display image.

[0038] In a third aspect, an augmented reality pair of glasses is provided in this embodiment. The augmented reality glasses include a pair of glasses, and an environment capture camera and a processor provided on the pair of glasses.

[0039] The environment capture camera is used to collect and send real-time environment images;

[0040] The processor is configured to execute the steps of any one of the methods described in the first aspect.

[0041] In some embodiments, the environment capture camera is arranged at the temple of the glasses body or the nose bridge connection, and the angle between the optical axis of the environment capture camera and the visual axis of the augmented reality glasses is less than 1°.

[0042] In a fourth aspect, the present application further provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the display correction method for augmented reality described in the first aspect.

[0043] Compared with the related art, the display correction method, device and augmented reality glasses for augmented reality provided in this embodiment obtain the original display image and the corresponding first pixel-level information of the augmented reality glasses; and obtain the target environment image of the augmented reality glasses; based on the target environment image, obtain the corresponding second pixel-level information; based on the first pixel-level information and the second pixel-level information, perform pixel-level adaptive adjustment on the original display image to obtain the target display image, thereby solving the display effect deviation problem caused by the superposition of ambient light in the augmented reality glasses, and can make the display image observed by the user through augmented reality accurate and stable through the corresponding adjustment of pixels, thereby improving the user's visual experience.

[0044] The details of one or more embodiments of the present application are set forth in the following drawings and description to make other features, objects, and advantages of the present application more readily apparent. BRIEF DESCRIPTION OF THE DRAWINGS

[0045] The drawings described herein are used to provide a further understanding of the present application and constitute a part of the present application. The illustrative embodiments of the present application and their descriptions are used to explain the present application and do not constitute an improper limitation on the present application. In the drawings:

[0046] Figure 1 This is a structural block diagram of the augmented reality glasses in an embodiment of the present application;

[0047] Figure 2 Schematic diagram of the flow of a display correction method for augmented reality in an embodiment of the present application;

[0048] Figure 3 This is a schematic structural diagram of the augmented reality glasses in a preferred embodiment of the present application;

[0049] Figure 4 Schematic diagram of the flow of a display correction method for augmented reality in a preferred embodiment of the present application;

[0050] Figure 5This is a structural block diagram of a display correction device for augmented reality in an embodiment of the present application.

[0051] Reference numerals: 11, glasses body; 12, environment capture camera; 13, processor; 51, original display data acquisition module; 52, environment data acquisition module; 53, display correction module. DETAILED DESCRIPTION

[0052] In order to more clearly understand the purpose, technical solutions and advantages of the present application, the present application is described and illustrated below in conjunction with the accompanying drawings and embodiments.

[0053] Unless otherwise defined, technical or scientific terms used in this application shall have the ordinary meanings as understood by persons of ordinary skill in the art to which this application belongs. The terms "a," "an," "the," "these," and similar expressions in this application do not denote limitations on quantity and may be singular or plural. The terms "comprise," "include," "have," and any variations thereof, as used in this application, are intended to cover non-exclusive inclusions. For example, a process, method, system, product, or device comprising a series of steps or modules (units) is not limited to the listed steps or modules (units) but may include unlisted steps or modules (units) or other steps or modules (units) inherent to the process, method, product, or device. The terms "connected," "connected," "coupled," and similar expressions used in this application are not limited to physical or mechanical connections but may include electrical connections, whether direct or indirect. As used in this application, "plurality" means two or more. "And / or" describes an association between associated objects, indicating that three possible relationships exist. For example, "A and / or B" can mean: A exists alone; A and B exist simultaneously; or B exists alone. Generally, the character " / " indicates that the objects in the preceding and following relationship are in an "or" relationship. The terms "first", "second", "third", etc. involved in this application are only used to distinguish similar objects and do not represent a specific ordering of the objects.

[0054] In this embodiment, a display correction method for augmented reality is provided, which is suitable for Figure 1The augmented reality glasses shown. Typically, augmented reality glasses use an optical system to superimpose the light of a virtual image with the light of the real environment, directing it to the human eye's retina, where it is ultimately integrated into a unified visual perception by the brain. For example, through augmented reality glasses, a virtual vase can be observed placed on a real table in front of the eye. The vase is a visual effect created by the virtual image. In some embodiments, this process specifically involves: the light signal of the digital content generated by the augmented reality glasses' microdisplay (such as Micro-LED, LCoS, or OLED) is guided to the lens through a series of optical elements (such as waveguides, prisms, lenses, or reflectors), forming an optical path for the virtual image. Ultimately, the virtual image is projected onto an optical virtual image plane in front of the eye (e.g., 2 meters away). The human eye lens must focus on this plane to be clearly visible. In the prior art, external ambient light often affects the display quality of the virtual image. Specifically, when the ambient light is superimposed on the virtual image information of the augmented reality glasses, the displayed image will have contrast shift or color cast, resulting in a reduced visual experience. To address this technical problem, this embodiment aims to process the virtual image before output to eliminate the deviation.

[0055] Figure 2 This is a flow chart of a display correction method for augmented reality in an embodiment of the present application, such as Figure 2 As shown, the method includes:

[0056] Step S210: Acquire the original display image of the augmented reality glasses and the corresponding first pixel-level information.

[0057] Specifically, a virtual image of the augmented reality glasses, i.e., an original display image, is obtained, and corresponding first pixel-level information is obtained. The first pixel-level information includes at least one of pixel brightness and color. The first pixel-level information can be obtained by extracting it from the original display image or directly obtaining it from data cached by the augmented reality glasses.

[0058] Step S220: acquiring a target environment image based on the field of view of the augmented reality glasses; and obtaining corresponding second pixel-level information based on the target environment image.

[0059] Specifically, the real-time environmental image is collected by the environment capture camera 12 configured on the augmented reality glasses, and the second pixel-level information describing the environment is extracted from it. Since the field of view of the human eye when observing the display screen is fixed after the augmented reality glasses are worn, obtaining the field of view range of the augmented reality glasses can improve the correspondence between the second pixel-level information and the first pixel-level information in the display field of view for accurate correction. In the specific implementation process, based on the acquired field of view range, the collected environmental image can be adjusted, and the configuration parameters of the environmental capture camera can also be adjusted to obtain the target environmental image. In addition, the type of the second pixel-level information corresponds to the first pixel-level information, that is, both include brightness information, or both include color information.

[0060] The above steps S210 and S220 can be performed simultaneously, which helps to improve the time accuracy of display correction.

[0061] Step S230 : performing pixel-level adaptive adjustment on the original display image based on the first pixel-level information and the second pixel-level information to obtain a target display image.

[0062] Specifically, each pixel point involved in the first pixel-level information and the second pixel-level information is analyzed and calculated to obtain adjustment information of each pixel point, and then pixel-level adaptive adjustment is performed on the original display image.

[0063] In this embodiment, by obtaining the original display image of the augmented reality glasses and the corresponding first pixel-level information; obtaining the target environment image of the augmented reality glasses; obtaining the corresponding second pixel-level information based on the target environment image; and performing pixel-level adaptive adjustment on the original display image based on the first pixel-level information and the second pixel-level information to obtain the target display image, the problem of display effect deviation caused by the superposition of ambient light in the augmented reality glasses is solved. Through the corresponding adjustment of pixels, the display image observed by the user through augmented reality can be accurate and stable, thereby improving the user's visual experience.

[0064] In some embodiments, obtaining a target environment image of the augmented reality glasses based on the field of view of the augmented reality glasses includes:

[0065] Step S211: Acquire the target pixel of the original display image. Specifically, the target pixel is the pixel size of the original display image projected into the field of view.

[0066] Step S212: Acquire the initial environment image captured by the augmented reality glasses.

[0067] Step S213: Using the field of view range and target pixels of the augmented reality glasses as target values, adjust the field of view and pixels of the initial environment image to obtain the target environment image.

[0068] For example, the field of view of the augmented reality glasses is 30° (H) × 20° (V), and the display image projected within the field of view (i.e., the original display image) has a pixel size of 1920 × 1080. The environment capture camera 12 captures the initial environment image, intercepts the portion of the image that overlaps with the field of view of the augmented reality glasses display, and converts the pixels of this portion of the environment image to 1920 × 1080 to obtain the target environment image. This allows for pixel-matching between the original display image and the environment image.

[0069] In this embodiment, by adjusting the initial environment image, information alignment between the environment image and the display image is achieved, thereby improving alignment efficiency and position precision.

[0070] In some embodiments, performing pixel-level adaptive adjustment on the original display image based on the first pixel-level information and the second pixel-level information to obtain the target display image includes:

[0071] In step S231 , the first pixel-level information and the second pixel-level information are input into a preset display correction model in a pixel-pairing manner, and pixel-level display adjustment information is output.

[0072] Step S232 : adjusting each pixel in the original display image based on the display adjustment information to obtain a target display image.

[0073] Specifically, the first pixel-level information includes at least one of the brightness and chromaticity information of each pixel of the original virtual image, and the second pixel-level information is information corresponding to the type of the first pixel-level information. The two are input into a preset display correction model in a pixel-pairing manner to output display adjustment information of the corresponding type. For example, under strong ambient light (such as outdoor sunlight), the original display image will become blurred due to insufficient brightness; under weak ambient light, it may be too bright and dazzling. By analyzing and calculating the brightness information in the first pixel-level information and the second pixel-level information, display adjustment information for brightness adjustment is output, and the brightness of the original display image is adaptively increased or decreased so that the target display image remains clear after entering the eye. Among them, the display correction model is a pre-trained deep learning model or other correction algorithm model, which can be selected according to actual application needs and is not limited in this embodiment.

[0074] In this embodiment, pixel-level analysis and adaptive adjustment are achieved through a display correction model, thereby further improving the consistency of virtual and real light and shadow and enhancing the accuracy of spatial correction of display deviations.

[0075] In some embodiments, the display correction model is a pre-trained deep learning neural network model; the training data of the deep learning neural network model includes input data and output data, the input data includes image brightness information and ambient brightness information, and the output data includes target brightness adjustment information; the deep learning neural network model includes a pooling layer and a convolution layer. The image brightness information is obtained based on the original display image, and the ambient brightness information is obtained based on the target environment image. It should be noted that the original display image used in the training model of this embodiment and the original display image used in the application model prediction can be obtained based on the same processing method, but they are not the same image. The same applies to the target environment image.

[0076] The first pixel-level information includes first brightness information; the second pixel-level information includes second brightness information. The above step S231 specifically includes: inputting the first brightness information and the second brightness information into a pre-trained deep learning neural network model in a pixel-paired manner to obtain brightness adjustment information.

[0077] In some embodiments, the first pixel level information includes first brightness information; the second pixel level information includes second brightness information; and the display adjustment information includes brightness adjustment information.

[0078] The process of generating brightness adjustment information by the display correction model includes:

[0079] Step S310: multiply the second brightness information by the ambient light transmittance of the augmented reality glasses to obtain intermediate brightness information.

[0080] In step S320, the intermediate brightness information is superimposed with the first brightness information to obtain a first superposition result. Specifically, step S320 is only a calculation process and does not directly adjust the original display image.

[0081] Step S330 : Calculate a first deviation value between the first superposition result and each pixel of the first brightness information.

[0082] Step S340: Obtain brightness adjustment information based on the first deviation value. Specifically, the deviation value is inverted to obtain a correction compensation value, namely, the brightness adjustment information.

[0083] In this embodiment, the brightness adjustment information is gradually acquired through superposition and comparison analysis in combination with the ambient light transmittance, thereby solving the brightness deviation problem with high quality.

[0084] In some embodiments, the display correction model is a pre-trained deep learning neural network model; the training data of the deep learning neural network model includes input data and output data, the input data includes image color information and environmental color information, and the output data includes target color adjustment information. The deep learning neural network model includes a pooling layer and a convolution layer. The image color information is obtained based on the original display image, and the environmental color information is obtained based on the target environmental image. It should be noted that the original display image used when training the model in this embodiment and the original display image used when applying the model for prediction can be obtained based on the same processing means, but they are not the same image; the same applies to the target environmental image.

[0085] The first pixel-level information includes first color information; the second pixel-level information includes second color information. The above step S231 also includes: inputting the first color information and the second color information into a pre-trained deep learning neural network model in a pixel-pairing manner to obtain color adjustment information.

[0086] In this embodiment, the color adjustment information is predicted by a pre-trained deep learning neural network model, thereby solving the problem of color deviation with high quality.

[0087] In some embodiments, adjusting each pixel in the original display image based on the display adjustment information to obtain the target display image includes adjusting each pixel in the original display image based on the color adjustment information and the brightness adjustment information to obtain the target display image. The color adjustment information and the brightness adjustment information can be calculated simultaneously to improve processing efficiency.

[0088] In this embodiment, the brightness adjustment information and the color adjustment information are used to adjust the brightness and color deviation respectively, thereby improving the display effect of the target display image in multiple dimensions.

[0089] In some embodiments, the method further comprises:

[0090] Step S410 , obtaining preset system parameters, where the system parameters include at least one of ambient light transmittance, color temperature requirement, and color gamut standard.

[0091] Step S420: Acquire a corresponding display correction model based on the system parameters.

[0092] Specifically, the ambient light transmittance of augmented reality glasses, the color temperature requirements input by the user, and the color gamut standards (such as sRGB, DCI-P3, AdobeRGB, etc.) can be set in the system in advance. By referring to the above system parameters to select the display correction model, the targeted display adjustment information can be improved, bringing a better experience to users.

[0093] In this embodiment, augmented reality glasses are provided. Figure 1 It is a structural diagram of augmented reality glasses, such as Figure 1 As shown, the augmented reality glasses include a glasses body 11, an environment capture camera 12 and a processor 13 provided on the glasses body 11. The environment capture camera 12 is used to collect real-time environment images; the processor 13 is used to execute the steps of any one of the methods in the first aspect above.

[0094] In this embodiment, pixel-level image correction is used to solve the display effect deviation problem caused by the superposition of ambient light in the augmented reality device, so that the display image observed by the user through augmented reality is accurate and stable, thereby improving the user's visual experience.

[0095] In some embodiments, the environment capture camera 12 is disposed on the glasses body 11 of the augmented reality glasses. Preferably, the environment capture camera 12 is located at the temples or the nose bridge connection. Figure 3 The bridge of the nose is connected to the environment image capture camera pointed by the arrow, which is the environment capture camera 12. When the augmented reality glasses are normally worn, the angle between the optical axis of the environment capture camera 12 and the visual axis of the augmented reality glasses is less than 1°.

[0096] In this embodiment, the setting of the position of the environment capture camera 12 helps to reduce the misalignment and distortion between the environment captured image and the environment image observed by human eyes.

[0097] The present embodiment is described and illustrated below through preferred embodiments. Figure 4 This is a flow chart of the display correction method for augmented reality in this preferred embodiment, as shown in FIG. Figure 4 As shown, the steps involved in the method include:

[0098] S1. Obtaining an original display image of the augmented reality glasses and corresponding original display image information. The original display image information includes the brightness and color (the latter refers to the color coordinates) of each pixel in the original display image.

[0099] S2. An environment capture camera 12 located on the augmented reality glasses captures, in real time, environmental image information within the display field of view of the augmented reality glasses, including brightness and color information. The environment capture camera 12 is located at the temples or at the bridge of the nose, and the angle between the optical axis of the environment capture camera and the visual axis of the augmented reality glasses is less than 1°.

[0100] S3. Input the environmental image information and the original display image information into the display correction model in a pixel-pairing manner to generate display adjustment information for each pixel within the display field of view, where the display adjustment information includes brightness adjustment information and color correction information.

[0101] S4. Generate a display adjustment instruction according to the brightness adjustment information and the color correction information, perform adaptive adjustment on the brightness and color of the original display image information, and obtain a target display image after real-time correction.

[0102] In this preferred embodiment, pixel-level image correction is performed using a one-to-one correspondence between field of view and pixel to improve the fineness and accuracy of display correction.

[0103] It should be noted that the steps shown in the above process or the flowchart in the accompanying drawings can be executed in a computer system such as a set of computer-executable instructions, and although a logical order is shown in the flowchart, in some cases, the steps shown or described can be executed in an order different from that shown here.

[0104] In this embodiment, a display correction device for augmented reality is also provided. The device is used to implement the above-mentioned embodiments and preferred embodiments, and the details that have been described will not be repeated. The terms "module," "unit," "subunit," etc. used below may refer to a combination of software and / or hardware that implements a predetermined function. Although the devices described in the following embodiments are preferably implemented in software, implementation using hardware, or a combination of software and hardware, is also possible and contemplated.

[0105] Figure 5 This is a structural block diagram of the display correction device for augmented reality in this embodiment, as shown in FIG. Figure 5 As shown, the device is suitable for augmented reality glasses and includes: an original display data acquisition module 51, an environmental data acquisition module 52 and a display correction module 53.

[0106] The original display data acquisition module 51 is used to obtain the original display image of the augmented reality glasses and the corresponding first pixel-level information.

[0107] The environment data acquisition module 52 is used to acquire a target environment image based on the field of view of the augmented reality glasses; and obtain corresponding second pixel-level information based on the target environment image.

[0108] The display correction module 53 is configured to perform pixel-level adaptive adjustment on the original display image based on the first pixel-level information and the second pixel-level information to obtain a target display image.

[0109] In some of the embodiments, the environmental data acquisition module 52 is further used to obtain the target pixels of the original display image; obtain the initial environmental image captured by the augmented reality glasses; and adjust the field of view and pixels of the initial environmental image with the field of view range and target pixels of the augmented reality glasses as target values ​​to obtain the target environmental image.

[0110] In some of the embodiments, the display correction module 53 is further used to input the first pixel-level information and the second pixel-level information into a preset display correction model in a pixel-pairing manner, and output pixel-level display adjustment information; based on the display adjustment information, each pixel in the original display image is adjusted to obtain a target display image.

[0111] In some embodiments, the first pixel-level information includes first brightness information; the second pixel-level information includes second brightness information; and the display adjustment information includes brightness adjustment information. The display correction module 53 is further configured to multiply the second brightness information by the ambient light transmittance of the augmented reality glasses to obtain intermediate brightness information; superimpose the intermediate brightness information with the first brightness information to obtain a first superimposed result; calculate a first deviation value between the first superimposed result and each pixel of the first brightness information; and obtain the brightness adjustment information based on the first deviation value.

[0112] In some embodiments, the first pixel-level information also includes first color information; the second pixel-level information also includes second color information; the display correction module 53 is further used to: input the first color information and the second color information into a pre-trained deep learning neural network model in a pixel pairing manner to obtain color adjustment information; wherein, the training data of the deep learning neural network model includes input data and output data, the input data includes image color information and environmental color information, and the output data includes target color adjustment information.

[0113] In some embodiments, the display correction module 53 is further configured to adjust each pixel in the original display image based on the color adjustment information and the brightness adjustment information to obtain a target display image.

[0114] In some embodiments, the display correction module 53 is further configured to obtain preset system parameters, including at least one of ambient light transmittance, color temperature requirement, and color gamut standard; and obtain a corresponding display correction model based on the system parameters.

[0115] It should be noted that the above modules can be functional modules or program modules, and can be implemented through software or hardware. For modules implemented through hardware, the above modules can be located in the same processor; or the above modules can be located in different processors in any combination.

[0116] In addition, in conjunction with the display correction method for augmented reality provided in the above embodiments, a storage medium may also be provided in this embodiment to implement the display correction method. The storage medium stores a computer program; when the computer program is executed by a processor, it implements any of the display correction methods for augmented reality provided in the above embodiments.

[0117] It should be understood that the specific embodiments described herein are only used to explain this application and are not used to limit it. Based on the embodiments provided in this application, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of this application.

[0118] Obviously, the accompanying drawings are merely examples or embodiments of the present application. A person skilled in the art can also apply the present application to other similar situations based on these drawings without inventive effort. Furthermore, it is understandable that, although the work involved in this development process may be complex and lengthy, certain design, manufacturing, or production changes based on the technical content disclosed in this application are merely routine technical means for a person skilled in the art and should not be considered to constitute a deficiency in the disclosure of the present application.

[0119] The term "embodiment" as used in this application refers to specific features, structures, or characteristics described in conjunction with the embodiment that can be included in at least one embodiment of this application. The appearance of this phrase in various places in the specification does not necessarily mean that the embodiment is the same, nor does it mean that it is mutually exclusive with other embodiments and is independent or optional. It is understood, either explicitly or implicitly, by those skilled in the art that the embodiments described in this application can be combined with other embodiments when there is no conflict.

[0120] The above-described embodiments merely represent several implementation methods of the present application. While the descriptions are relatively specific and detailed, they should not be construed as limiting the scope of patent protection. It should be noted that a person of ordinary skill in the art may make various modifications and improvements without departing from the spirit of the present application, and these modifications and improvements fall within the scope of protection of the present application. Therefore, the scope of protection of the present application shall be determined by the appended claims.

Claims

1. A display correction method for augmented reality, characterized in that: Applicable to augmented reality glasses, the method includes: Acquire an original display image of the augmented reality glasses and corresponding first pixel-level information; Acquire a target environment image based on the field of view of the augmented reality glasses; and obtain corresponding second pixel-level information based on the target environment image; Based on the first pixel-level information and the second pixel-level information, pixel-level adaptive adjustment is performed on the original display image to obtain a target display image.

2. The display correction method for augmented reality according to claim 1, characterized in that: Acquiring a target environment image based on the field of view of the augmented reality glasses includes: Acquire a target pixel of the original display image; Acquiring an initial environment image captured by the augmented reality glasses; Taking the field of view range of the augmented reality glasses and the target pixels as target values, the field of view and pixels of the initial environment image are cut to obtain a target environment image.

3. The display correction method for augmented reality according to claim 1, characterized in that: Based on the first pixel-level information and the second pixel-level information, performing pixel-level adaptive adjustment on the original display image to obtain a target display image, comprising: Inputting the first pixel-level information and the second pixel-level information into a preset display correction model in a pixel-pairing manner, and outputting pixel-level display adjustment information; Based on the display adjustment information, each pixel in the original display image is adjusted to obtain a target display image.

4. The display correction method for augmented reality according to claim 3, characterized in that: The first pixel level information includes first brightness information; the second pixel level information includes second brightness information; the display adjustment information includes brightness adjustment information; The process of generating the brightness adjustment information by the display correction model includes: Multiplying the second brightness information by the ambient light transmittance of the augmented reality glasses to obtain intermediate brightness information; Superimposing the intermediate brightness information with the first brightness information to obtain a first superimposed result; Calculating a first deviation value between the first superposition result and each pixel of the first brightness information; The brightness adjustment information is obtained based on the first deviation value.

5. The display correction method for augmented reality according to claim 4, characterized in that: The first pixel-level information includes first color information; the second pixel-level information includes second color information; The step of inputting the first pixel-level information and the second pixel-level information into a preset display correction model in a pixel-pairing manner to output pixel-level display adjustment information includes: inputting the first color information and the second color information into a pre-trained deep learning neural network model in a pixel-pairing manner to obtain color adjustment information; The training data of the deep learning neural network model includes input data and output data, the input data includes image color information and environmental color information, and the output data includes target color adjustment information.

6. The display correction method for augmented reality according to claim 5, characterized in that: Adjusting each pixel in the original display image based on the display adjustment information to obtain a target display image includes: Based on the color adjustment information and the brightness adjustment information, each pixel in the original display image is adjusted to obtain a target display image.

7. The display correction method for augmented reality according to claim 3, characterized in that: The method further comprises: Obtaining preset system parameters, wherein the system parameters include at least one of ambient light transmittance, color temperature requirement, and color gamut standard; The corresponding display correction model is acquired based on the system parameters.

8. A display correction device for augmented reality, characterized in that: Applicable to augmented reality glasses, the device comprises: an original display data acquisition module, configured to acquire an original display image of the augmented reality glasses and corresponding first pixel-level information; An environmental data acquisition module, configured to acquire a target environmental image based on the field of view of the augmented reality glasses; and obtain corresponding second pixel-level information based on the target environmental image; The display correction module is configured to perform pixel-level adaptive adjustment on the original display image based on the first pixel-level information and the second pixel-level information to obtain a target display image.

9. An augmented reality glasses, characterized in that: The augmented reality glasses include a glasses body, and an environment capture camera and a processor arranged on the glasses body; The environment capture camera is used to collect and send real-time environment images; The processor is configured to execute the steps of the method according to any one of claims 1 to 7.

10. The augmented reality glasses according to claim 9, characterized in that: The environment capture camera is arranged on the augmented reality glasses, and the angle between the optical axis of the environment capture camera and the visual axis of the augmented reality glasses is less than 1°.

Images