Image processing method and device, electronic equipment and storage medium

Abstract

The application provides an image processing method and device, electronic equipment and storage medium. The method is applied to the electronic equipment, and includes: acquiring a test image; determining brightness values of interest points in different regions of the test image; determining a gray scale value of a target point based on the brightness values of the interest points in the different regions; and adjusting the test image according to the gray scale value of the target point. The above method can improve the accuracy of the electronic equipment in capturing the position of the interest point in the test image.

Description

Technical Field

[0001] This application relates to the field of coordinate system calibration, and more particularly to an image processing method, apparatus, electronic device, and storage medium. Background Technology

[0002] Augmented Reality Glasses (AR glasses) are head-mounted displays that utilize augmented reality technology to overlay virtual information onto the user's real world. AR glasses typically include one or more cameras, sensors, and display technology to present a combination of digital content and the real world. Liquid Crystal on Silicon (LCOS) is a display technology commonly used in projectors and head-mounted displays. It works by using liquid crystals on a silicon substrate to control the reflection of light, thereby forming an image. While LCOS technology offers high resolution and good contrast, it also has limitations, particularly in optical uniformity. AR glasses using LCOS technology may encounter problems with poor image uniformity, making it difficult to capture positional points in the test image used to detect the AR glasses. Summary of the Invention

[0003] This application discloses an image processing method, apparatus, electronic device, and storage medium, which solves the technical problem of difficulty in capturing position points in test images of AR glasses.

[0004] In a first aspect, this application provides an image processing method applied to an electronic device, the method comprising: acquiring a test image; determining the brightness values ​​of interest points in different regions of the test image; determining the grayscale value of a target point based on the brightness values ​​of the interest points in the different regions; and adjusting the test image according to the grayscale value of the target point.

[0005] In one possible implementation, determining the grayscale value of the target point based on the brightness values ​​of the points of interest in the different regions includes: selecting candidate brightness values ​​from the brightness values ​​of the points of interest in the different regions according to a testing method; and determining the grayscale value of the target point based on the candidate brightness values.

[0006] In one possible implementation, determining the grayscale value of the target point based on the candidate brightness values ​​includes: calculating the average value Y of the candidate brightness values; and obtaining the grayscale value of the target point based on the Gamma function and the average value Y.

[0007] In one possible implementation, the Gamma function is: Where K is a constant, Gray is the grayscale value of the target point, and gamma is a parameter of the display screen of the electronic device.

[0008] In one possible implementation, the method further includes: marking the position of the point of interest corresponding to the candidate brightness value.

[0009] In one possible implementation, the test image is a solid color image.

[0010] In one possible implementation, the testing method includes a binocular fusion test.

[0011] Secondly, this application provides an image processing apparatus, the image processing apparatus comprising: an acquisition module for acquiring a test image; a determination module for determining the brightness values ​​of interest points in different regions of the test image; the determination module further for determining the grayscale value of a target point based on the brightness values ​​of the interest points in the different regions; and a processing module for adjusting the test image according to the grayscale value of the target point.

[0012] Thirdly, this application provides an electronic device including a processor and a memory, wherein the processor is configured to execute a computer program stored in the memory to implement the image processing method described above.

[0013] Fourthly, this application provides a computer-readable storage medium storing at least one instruction that, when executed by a processor, implements the image processing method described above.

[0014] The image processing method provided in this application determines the brightness values ​​of interest points in different regions of a test image, determines the grayscale value of a target point based on the brightness values ​​of interest points in different regions, and adjusts the test image according to the grayscale value of the target point so that the brightness difference of interest points in the test image is small, which makes it easier for electronic devices to identify the position of interest points in the test image and determine the display effect of AR glasses based on the position. Attached Figure Description

[0015] Figure 1 This is an energy distribution map of the test image provided in the embodiments of this application.

[0016] Figure 2 This is a schematic diagram illustrating an application scenario of the image processing method provided in the embodiments of this application.

[0017] Figure 3 This is a schematic diagram of the structure of the first device provided in the embodiments of this application.

[0018] Figure 4 This is a flowchart of the image processing method provided in the embodiments of this application.

[0019] Figure 5 This is a schematic diagram of the test image before processing provided in the embodiments of this application.

[0020] Figure 6 This is a schematic diagram of the processed test image provided in the embodiments of this application.

[0021] Figure 7 This is a schematic diagram of the structure of the image processing apparatus provided in the embodiments of this application. Detailed Implementation

[0022] For ease of understanding, some concepts related to the embodiments of this application are illustrated and explained by way of example for reference.

[0023] It should be noted that in this application, "at least one" means one or more, and "more than one" means two or more. "And / or" describes the relationship between related objects, indicating that three relationships can exist. For example, A and / or B can represent: A alone, A and B simultaneously, or B alone, where A and B can be singular or plural. The terms "first," "second," "third," "fourth," etc. (if present) in the specification, claims, and drawings of this application are used to distinguish similar objects, not to describe a specific order or sequence.

[0024] Compared to traditional Liquid Crystal Display (LCD) technology, LCOS technology offers advantages such as higher resolution, better color saturation, and wider viewing angles. LCOS technology can be applied to various display devices, including projectors, televisions, monitors, and mobile devices. In these applications, LCOS solutions can provide high-quality image and video output and support multiple formats and resolutions. Furthermore, LCOS technology can be combined with other technologies, such as Digital Light Processing (DLP), to improve display effects and performance. In short, LCOS technology is an advanced display technology that provides users with a high-quality visual experience and is suitable for a wide range of display devices and application scenarios.

[0025] However, AR glasses using LCOS technology display images with higher brightness in the center and lower brightness in the four corners (e.g., ...). Figure 1 As shown in the image, this is due to the optical characteristics of the LCOS panel itself and the design of the optical system. When attempting to improve corner brightness by adjusting the exposure, it may cause overexposure in the central area, resulting in distortion. This can make corner content unclear, difficult to recognize icons, or inaccurately positioned.

[0026] In practical applications, this issue can affect the performance of AR glasses, especially in situations requiring high precision. For example, in the Binocular Fusion Test, the system needs to accurately capture and process image data from two cameras to simulate the stereoscopic visual experience of the human eye observing objects. If display problems prevent the accurate recognition or positioning of icons, the test results will be inaccurate, impacting the final user experience.

[0027] To address the aforementioned issues, this application provides an image processing method that determines the brightness values ​​of interest points in different regions of a test image, determines the grayscale value of a target point based on the brightness values ​​of the interest points in different regions, and adjusts the test image according to the grayscale value of the target point so that the brightness differences of interest points in the test image are small, making it easier for electronic devices to identify the position of interest points in the test image and determine the display effect of AR glasses based on the position.

[0028] To better understand the image processing method, electronic device, and storage medium provided in the embodiments of this application, the application scenarios of the image processing method of this application are described below.

[0029] Figure 2 This is a schematic diagram illustrating an application scenario of the image processing method provided in this application embodiment. The image processing method provided in this application embodiment is applied to a first device 1, which is communicatively connected to a second device 2. For example, the first device 1 can be communicatively connected to the second device 2 through a wireless communication module 102 and / or other communication modules.

[0030] In this embodiment of the application, the first device 1 may be a smart screen, mobile phone, tablet computer, smart wearable device, laptop computer, netbook, etc., and the first device 1 is used to capture the image projected by the second device 2.

[0031] In this embodiment, the second device 2 is an augmented reality (AR) / virtual reality (VR) device. This embodiment uses AR glasses as an example for illustration.

[0032] To better understand the image processing method provided in this application, the specific structure of the first device is described below. (See attached document.) Figure 3 The diagram shown is a structural schematic of a first device 1 provided in an embodiment of this application. The first device 1 includes, but is not limited to, such as... Figure 3As shown, the first device 1 may include a camera 100, a display screen 101, a wireless communication module 102, a memory 103, a processor 104, an input / output (I / O) interface 105, and a bus 106. The processor 104 is coupled to the camera 100, the display screen 101, the wireless communication module 102, the memory 103, the processor 104, and the I / O interface 105 via the bus 106.

[0033] In some embodiments of this application, camera 100 is a test instrument camera, typically referring to a high-precision camera specifically designed for measuring and analyzing the performance of optical systems. Camera 100 features high resolution, high dynamic range, and accurate color reproduction capabilities to ensure accurate capture and analysis of subtle differences in images. In AR glasses testing, using a specialized test instrument camera ensures the acquisition of high-quality image data, thereby enabling accurate evaluation of the AR glasses' performance. For example, camera 100 could be a scientific-grade CCD or CMOS camera.

[0034] The display screen 101 can be a touch screen, specifically a touch-sensitive liquid crystal display device. Alternatively, the display screen 101 can be a non-touch screen. The display screen 101 is used to display images captured by the camera 100.

[0035] In some embodiments of this application, the wireless communication module 102 may provide one or more of the following wireless communication solutions: Wireless Fidelity (Wi-Fi), Bluetooth (BT), mobile communication network, Frequency Modulation (FM), Near Field Communication (NFC), Infrared (IR), etc.

[0036] The memory 103 may include one or more random access memory (RAM) and one or more non-volatile memory (NVM). The RAM can be directly read and written by the processor 104, and can be used to store executable programs (such as machine instructions) of the operating system or other running programs, as well as user and application data.

[0037] Random access memory can include static random-access memory (SRAM), dynamic random-access memory (DRAM), synchronous dynamic random-access memory (SDRAM), double data rate synchronous dynamic random-access memory (DDR SDRAM), etc.

[0038] Non-volatile memory can also store executable programs and user and application data, and can be pre-loaded into random access memory for direct reading and writing by the processor 104. Non-volatile memory can include disk storage devices and flash memory.

[0039] The memory 103 is used to store one or more computer programs. The one or more computer programs are configured to be executed by the processor 104. The one or more computer programs include a plurality of instructions, which, when executed by the processor 104, can implement a data processing method that is executed on the first device 1.

[0040] In other embodiments, the first device 1 further includes an external memory interface for connecting to an external memory to expand the storage capacity of the first device 1.

[0041] Processor 104 may include one or more processing units, such as an application processor (AP), a modem processor, a graphics processing unit (GPU), an image signal processor (ISP), a controller, a video codec, a digital signal processor (DSP), a baseband processor, and / or a neural network processing unit (NPU). These different processing units may be independent devices or integrated into one or more processors.

[0042] The processor 104 provides computing and control capabilities. For example, the processor 104 is used to execute computer programs stored in the memory 103 to implement the data transfer method described above.

[0043] I / O interface 105 is used to provide a channel for user input or output. For example, I / O interface 105 can be used to connect various input and output devices, such as mouse, keyboard, touch screen, etc., so that users can enter information or visualize information.

[0044] Bus 106 is used at least to provide a channel for communication between the camera 100, display screen 101, wireless communication module 102, memory 103, processor 104 and I / O interface 105 in the first device 1.

[0045] It is understood that the structures illustrated in the embodiments of this application do not constitute a specific limitation on the first device 1. In other embodiments of this application, the first device 1 may include more or fewer components than illustrated, or combine some components, or split some components, or have different component arrangements. The illustrated components may be implemented in hardware, software, or a combination of software and hardware.

[0046] To resolve the above issues, please refer to Figure 4 As shown, Figure 4 This is a flowchart of an image processing method provided in an embodiment of this application, applied to a first device (e.g., Figure 3 The first device in the process flow diagram. Depending on different requirements, the order of the steps in this flowchart can be changed, and some steps can be omitted.

[0047] Step S01: Obtain the test image.

[0048] In some embodiments of this application, when the second device is AR glasses, in order to test the display of the AR glasses, an image can be projected through the AR glasses, and the first device can be used to capture the image projected by the AR glasses to obtain a test image. The display of the AR glasses can then be tested based on the test image.

[0049] In this embodiment, the test image can be a solid color image. For example, the test image is a solid color image such as a pure white image or a pure green image. In this embodiment, the test image is a pure white image, and the grayscale of the pixels in the pure white image is 255.

[0050] Step S02: Determine the brightness values ​​of points of interest in different regions of the test image.

[0051] In this embodiment, the captured test image may have high brightness in the center and low brightness in the surrounding corners. This means that during the testing of the second device using the first device, the first device cannot accurately capture the icons corresponding to the pixels in the surrounding corners of the test image to obtain the coordinates of those icons, thus failing to accurately test the display of the second device. To solve this problem, the exposure of the first device when capturing the second device can be increased to make the surrounding corners clear. However, this will cause the center of the test image to become distorted due to overexposure, easily leading to situations where the icons corresponding to the center cannot be identified or the identified icons are inaccurately positioned, reducing the accuracy of the test results. To solve this problem, the brightness values ​​of points of interest in different areas of the test image can be obtained, and the test image can be adjusted based on these brightness values, thereby accurately capturing the icons corresponding to the pixels in the surrounding corners to obtain the coordinates of those icons.

[0052] In one embodiment, testing the test image using different testing methods can determine the brightness values ​​of points of interest in different regions of the test image. For example, when detecting a second device using a binocular fusion test, the brightness values ​​of the midpoint and corner points in the test image can be obtained. For example, as... Figure 1 As shown, the brightness values ​​of nine different points of interest in the test image were obtained, labeled as ①, ②, ③, ④, ⑤, ⑥, ⑦, ⑧, and ⑨, respectively. The brightness value of the first point is 999.88; the brightness value of the second point is 2562.42; the brightness value of the third point is 1390.84; the brightness value of the fourth point is 1697.79; the brightness value of the fifth point is 3689.42; the brightness value of the sixth point is 2731.53; the brightness value of the seventh point is 1466.19; the brightness value of the eighth point is 2411.53; and the brightness value of the ninth point is 1316.2.

[0053] Step S03: Determine the grayscale value of the target point based on the brightness values ​​of the points of interest in different regions.

[0054] In this embodiment, after determining the brightness values ​​of interest points in different regions of the test image, the brightness values ​​of interest points related to the test method can be selected to calculate the grayscale value of the target point. Specifically, determining the grayscale value of the target point based on the brightness values ​​of interest points in different regions includes: selecting candidate brightness values ​​from the brightness values ​​of interest points in different regions according to the test method; and determining the grayscale value of the target point based on the candidate brightness values. For example, as... Figure 1As shown, after obtaining the brightness values ​​of the locations corresponding to nine different points of interest in the test image, five candidate brightness values ​​are selected from the brightness values ​​of the points of interest in the nine different regions according to the test method. These are: the brightness value of the first location point (999.88), the brightness value of the third location point (1390.84), the brightness value of the fifth location point (3689.42), the brightness value of the seventh location point (1466.19), and the brightness value of the ninth location point (1316.2). Since the brightness value of the fifth location point differs significantly from the brightness values ​​of the first, third, seventh, and ninth location points, it needs to be adjusted according to the Gamma curve formula. The Gamma function is: Where Y is the brightness, K is a constant, gamma is the gamma parameter of the display screen of the first device, and Gray is the grayscale value (0~255). Since the brightness of each pixel in the test image cannot be completely consistent, in this embodiment of the application, the brightness value Y of the target point of the test image can be the average of other candidate brightness values. Then Y, K, and gamma are all known, and the value of grayscale Gray can be calculated.

[0055] In this embodiment of the application, determining the grayscale value of the target point based on the candidate brightness values ​​includes: calculating the average value Y of the candidate brightness values; and obtaining the grayscale value of the target point based on the average value Y and the Gamma function. For example, as... Figure 5 As shown, when the test image is a pure white image, the grayscale value of the pixels in the test image is equal to 255. Given the brightness values ​​of the first position (999.88), the third position (1390.84), the seventh position (1466.19), and the ninth position (1316.2), the average brightness value of these four positions, Y, can be obtained as: Y = (999.88 + 1390.84 + 1466.19 + 1316.2) / 4 = 1195.53. If the gamma constant of the display screen of the first device is 2.2 and K = 0.0187, then... We can calculate that Gray = 152.

[0056] In this embodiment of the application, the image processing method includes marking the positions of interest points corresponding to candidate brightness values. Specifically, interest points corresponding to candidate brightness values ​​can be marked using different shapes, such as squares, circles, triangles, etc.

[0057] Step S04: Adjust the test image according to the grayscale value of the target point.

[0058] In this embodiment of the application, after calculating the grayscale value of the target point in the test image, the grayscale of the target point is adjusted according to the grayscale value. For example, Figure 5The grayscale value of the fifth position point in the pure white test image is 255. After calculating the grayscale value of the fifth position point to be 152, the grayscale value of the fifth position point is adjusted from 255 to 152, as follows. Figure 6 As shown.

[0059] After adjusting the grayscale of the fifth position point in the test image, the first device sends the adjusted test image to the second device. This makes the coordinates of the position points corresponding to the five candidate brightness values ​​obtained by the second device based on the projection of the test image and the image captured by the first camera in the binocular fusion test smaller than the coordinates of the position points corresponding to the five candidate brightness values ​​obtained by the second camera, thereby improving the test results.

[0060] In this embodiment of the application, the brightness values ​​of interest points in different regions of the test image are determined, and the grayscale value of the target point is determined based on the brightness values ​​of interest points in different regions; and the test image is adjusted according to the grayscale value of the target point so that the brightness difference of interest points in the test image is small, which makes it easier for electronic devices to identify the position of interest points in the test image and determine the display effect of AR glasses based on the position.

[0061] Figure 7 This is a structural diagram of the image processing apparatus provided in an embodiment of this application. The image processing apparatus 700 may include multiple functional modules composed of computer program segments. The computer programs of each program segment in the image processing apparatus 700 may be stored in the memory of a computer device and executed by at least one processor to perform (see details). Figure 4 (Description) Image processing functions.

[0062] In this embodiment, the image processing device 700 can be divided into multiple functional modules according to the functions it performs. The functional modules may include: an acquisition module 701, a determination module 702, and a processing module 703. As used in this application, a module refers to a series of computer program segments that can be executed by at least one processor and perform a fixed function, and which are stored in memory. In this embodiment, the image processing device 700 can be used to implement, for example... Figure 4 The image processing method shown. Figure 7 As shown, the image processing device 700 is applied to electronic devices (such as...) Figure 3 In the first device shown, the image processing apparatus 700 includes: The acquisition module 701 is used to acquire a test image; the determination module 702 is used to determine the brightness values ​​of interest points in different regions of the test image; the determination module 702 is also used to determine the grayscale value of a target point based on the brightness values ​​of the interest points in different regions; and the processing module 703 adjusts the test image according to the grayscale value of the target point.

[0063] In one possible implementation, the determining module 702 is further configured to select candidate brightness values ​​from the brightness values ​​of points of interest in the different regions according to the testing method; and determine the grayscale value of the target point based on the candidate brightness values.

[0064] In one possible implementation, the determining module 702 is further configured to calculate the average value Y of the candidate brightness values; and obtain the grayscale value of the target point based on the Gamma function and the average value Y.

[0065] In one possible implementation, the Gamma function is: Where K is a constant, Gray is the grayscale value of the target point, and gamma is a parameter of the display screen of the electronic device.

[0066] In one possible implementation, the processing module is further configured to mark the location of the point of interest corresponding to the candidate brightness value.

[0067] In one possible implementation, the test image is a solid color image.

[0068] In one possible implementation, the testing method includes a binocular fusion test.

[0069] This application also provides a computer-readable storage medium storing a computer program, the computer program including program instructions, and the method implemented when the program instructions are executed can refer to the methods in the above embodiments of this application.

[0070] This application provides a computer program product that includes a computer program that, when run on a processor, causes the processor to execute the image processing method described in any of the possible implementations above.

[0071] The computer-readable storage medium can be the internal memory of the computer device described in the above embodiments, such as the hard disk or memory of the computer device. The computer-readable storage medium can also be an external storage device of the computer device, such as a plug-in hard disk, smart media card (SMC), secure digital (SD) card, flash card, etc., provided on the computer device.

[0072] In some embodiments, the computer-readable storage medium may include a stored program area and a stored data area, wherein the stored program area may store an operating system, an application program required for at least one function, etc.; and the stored data area may store data created based on the use of the computer device, etc.

[0073] In the above embodiments, the descriptions of each embodiment have different focuses. For parts that are not described in detail or recorded in a certain embodiment, please refer to the relevant descriptions of other embodiments.

[0074] Those skilled in the art will recognize that the units and algorithm steps of the various examples described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are implemented in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of this application.

[0075] In the embodiments provided in this application, it should be understood that the disclosed devices / terminal equipment and methods can be implemented in other ways. For example, the device / terminal equipment embodiments described above are merely illustrative. For instance, the division of modules or units is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the displayed or discussed mutual coupling or direct coupling or communication connection may be through some interfaces; the indirect coupling or communication connection between devices or units may be electrical, mechanical, or other forms.

[0076] The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs.

[0077] The above-described embodiments are only used to illustrate the technical solutions of this application, and are not intended to limit them. Although this application has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features. Such modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of this application, and should all be included within the protection scope of this application.

Claims

1. An image processing method applied to electronic devices, characterized in that, The method includes: Obtain the test image; Determine the brightness values ​​of points of interest in different regions of the test image; The grayscale value of the target point is determined based on the brightness values ​​of the points of interest in the different regions; and The test image is adjusted based on the grayscale value of the target point.

2. The image processing method according to claim 1, characterized in that, The process of determining the grayscale value of the target point based on the brightness values ​​of the interest points in the different regions includes: Candidate brightness values ​​are selected from the brightness values ​​of points of interest in the different regions according to the testing method; The grayscale value of the target point is determined based on the candidate brightness values.

3. The image processing method according to claim 2, characterized in that, Determining the grayscale value of the target point based on the candidate brightness value includes: Calculate the average value Y of the candidate brightness values; The grayscale value of the target point is obtained based on the average value Y and the Gamma function.

4. The image processing method according to claim 3, characterized in that, The Gamma function is: Where K is a constant, Gray is the grayscale value of the target point, and gamma is a parameter of the display screen of the electronic device.

5. The image processing method according to claim 2, characterized in that, The method further includes: Mark the location of the point of interest corresponding to the candidate brightness value.

6. The image processing method according to claim 1, characterized in that, The test image is a solid color image.

7. The image processing method according to claim 2, characterized in that, The testing method includes the binocular fusion test.

8. An image processing apparatus, characterized in that, The image processing device includes: The acquisition module is used to acquire test images; The determination module is used to determine the brightness values ​​of points of interest in different regions of the test image; The determining module is further configured to determine the grayscale value of the target point based on the brightness values ​​of the points of interest in the different regions; and The processing module is used to adjust the test image based on the grayscale value of the target point.

9. An electronic device, characterized in that, The electronic device includes a processor and a memory, the processor being configured to execute a computer program stored in the memory to implement the image processing method as described in any one of claims 1 to 7.

10. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores at least one instruction, which, when executed by a processor, implements the image processing method as described in any one of claims 1 to 7.

Images