Image processing method, head-mounted display device and storage medium

Abstract

The embodiments of the present application belong to the field of image processing. Provided are an image processing method, a head-mounted display device, a server and a storage medium. The method comprises: acquiring the current diopter of the eyeballs of a wearer of a head-mounted display device, and acquiring a disparity range corresponding to the current diopter; acquiring a first image separately corresponding to each micro-lens, and generating a disparity map between the first images respectively corresponding to the micro-lenses; on the basis of the disparity map and the disparity range, determining an image block to be processed in each first image; rearranging each pixel in said image block in each first image to obtain N*M second images, such that a disparity separately corresponding to each pixel in each second image is within the disparity range; and displaying the N*M second images, such that a micro-lens array synthesizes and displays the N*M second images. The method can improve the versatility of a head-mounted display device and reduce the usage cost of the head-mounted display device.

Description

Image processing method, head-mounted display device and storage medium

[0001] The present application claims priority from the Chinese patent application No. 2024107196581, filed on June 24, 2024, and entitled "Image processing method, head-mounted display device and storage medium", the content of which is incorporated herein by reference in its entirety. TECHNICAL FIELD

[0002] The present application relates to the technical field of image processing, and in particular to an image processing method, a head-mounted display device and a storage medium. BACKGROUND

[0003] The head-mounted display device guides the video or image light emitted by a miniature image display (such as a transmissive or reflective liquid crystal display screen, an organic electroluminescent device, or a DMD device) to the user's eyes through optical technology to realize a virtual, magnified image in the user's near vision range, thereby providing the user with intuitive, visual 2D / 3D images, videos and text information. For people with abnormal vision, the lens can be designed and fitted according to the user's diopter, or a rear-mounted magnetic lens can be provided for fitting, so that the head-mounted display device can meet the needs of people with abnormal vision. However, the diopter of the user's eyeball may change over time, which causes the previously designed lens to be unsuitable. In addition, if multiple users want to use the head-mounted display device at different times and the diopters of the eyeballs of the multiple users are different, different lenses need to be purchased, which is costly, the head-mounted display device has poor universality, and the user experience is not good. SUMMARY

[0004] Embodiments of the present application provide an image processing method, system, head-mounted display device and storage medium, which aims to improve the universality of the head-mounted display device and reduce the use cost of the head-mounted display device, thereby improving the user experience.

[0005] In a first aspect, an image processing method is provided, which is applied to a head-mounted display device, the head-mounted display device comprising a microlens array, the microlens array comprising N*M microlenses, N and M being positive integers, and at least one of N and M being greater than or equal to 2, the method comprising:

[0006] obtaining a current diopter of an eyeball of a wearer of the head-mounted display device, and obtaining a parallax range corresponding to the current diopter;

[0007] obtaining a first image corresponding to each of the microlenses, and generating a parallax map between the first images corresponding to each of the microlenses;

[0008] determining, according to the disparity map and the disparity range, to-be-processed image blocks in the first images, each pixel in the to-be-processed image blocks each corresponding to a disparity out of the disparity range;

[0009] performing rearrangement processing on each pixel in the to-be-processed image blocks in each of the first images, to obtain N*M second images, so that each pixel in each of the second images each corresponds to a disparity in the disparity range;

[0010] displaying the N*M second images, so that the microlens array synthesizes and displays the N*M second images.

[0011] In a second aspect, the embodiments of the present application further provide an image processing method applied to a server, the server being in communication connection with a head-mounted display device, the head-mounted display device including a microlens array, the microlens array including N*M microlenses, N and M being positive integers, and at least one of N and M being greater than or equal to 2, the method including:

[0012] obtaining an image processing request sent by the head-mounted display device, the image processing request being triggered by the head-mounted display device in a case where a current refractive power of a wearer's eyeball is detected to be different from a preset refractive power;

[0013] obtaining the current refractive power of the wearer's eyeball from the image processing request, and obtaining a disparity range corresponding to the current refractive power;

[0014] obtaining a first image corresponding to each of the microlenses, and generating a disparity map between the first images corresponding to the microlenses;

[0015] determining, according to the disparity map and the disparity range, to-be-processed image blocks in the first images, each pixel in the to-be-processed image blocks each corresponding to a disparity out of the disparity range;

[0016] performing rearrangement processing on each pixel in the to-be-processed image blocks in each of the first images, to obtain N*M second images, so that each pixel in each of the second images each corresponds to a disparity in the disparity range;

[0017] sending the N*M second images to the head-mounted display device, so that the head-mounted display device displays the N*M second images, so that the microlens array synthesizes and displays the N*M second images.

[0018] In a third aspect, an embodiment of the present application further provides a computer device, comprising a processor, a memory, a computer program stored in the memory and executable by the processor, and a data bus for realizing connection communication between the processor and the memory, wherein the computer program, when executed by the processor, realizes the image processing method according to the first aspect or the second aspect.

[0019] In a fourth aspect, an embodiment of the present application further provides a storage medium for computer readable storage, wherein the storage medium stores one or more programs, and the one or more programs are executable by one or more processors to realize the image processing method according to the first aspect or the second aspect.

[0020] The embodiment of the present application provides an image processing method, a computer device and a storage medium. The embodiment of the present application rearranges the pixels in the to-be-displayed image, so that the parallax corresponding to each pixel in the rearranged image is within the parallax range corresponding to the current refractive power of the wearer's eyeball. In this way, after the rearranged image is synthesized and displayed by the microlens array, the wearer can see a clear image without the need to customize the lens, and multiple users can use the head-mounted display device at different times, improving the versatility of the head-mounted display device, reducing the use cost of the head-mounted display device, and greatly improving the user experience. BRIEF DESCRIPTION OF DRAWINGS

[0021] In order to more clearly illustrate the technical solutions of the embodiments of the present application, the drawings needed in the embodiment description will be briefly introduced. Obviously, the drawings in the following description are some embodiments of the present application, and other drawings can also be obtained by those skilled in the art without creative labor.

[0022] FIG. 1 is a structural schematic block diagram of a computer device according to an embodiment of the present application;

[0023] FIG. 2 is a step flowchart of an image processing method according to an embodiment of the present application;

[0024] FIG. 3 is an example diagram of a microlens array in an embodiment of the present application;

[0025] FIG. 4 is a sub-step flowchart of the image processing method in FIG. 2;

[0026] FIG. 5 is an example diagram in which a plurality of images are synthesized by a microlens array in an embodiment of the present application;

[0027] FIG. 6 is a flowchart of another image processing method according to an embodiment of the present application. DETAILED DESCRIPTION

[0028] The technical solutions in the embodiments of the present application will be clearly and completely described below with reference to the drawings in the embodiments of the present application. Obviously, the described embodiments are some of the embodiments of the present application, rather than all the embodiments of the present application. Based on the embodiments in the present application, all other embodiments obtained by those of ordinary skill in the art without creative effort belong to the scope of protection of the present application.

[0029] The flowchart shown in the drawings is only an example and does not necessarily include all the contents and operations / steps, nor does it have to be executed in the order described. For example, some operations / steps can be further decomposed, combined or partially merged, so the actual execution order can be changed according to the actual situation.

[0030] It should be understood that the terms used in the specification of the present application are only for the purpose of describing specific embodiments and are not intended to limit the present application. As used in the specification and the appended claims of the present application, unless otherwise clear from the context, the singular forms "a", "an" and "the" are intended to include the plural forms.

[0031] The head-mounted display device directs the video or image light emitted by a miniature image display (such as a transmissive or reflective liquid crystal display, an organic electroluminescent device, a DMD device) to the user's eyes through optical technology to realize a virtual, magnified image in the user's near vision range, providing the user with intuitive, visual 2D / 3D images, videos and text information. For people with abnormal vision, the lens can be designed and fitted according to the user's diopter, or a rear-mounted magnetic lens can be provided for fitting, so that the head-mounted display device can meet the needs of people with abnormal vision. However, the diopter of the eyeball of the same user may change over time, resulting in that the previously designed lens cannot be applied, in addition, if multiple users want to use the head-mounted display device at different times and the diopters of the eyeballs of the multiple users are different, different lenses need to be purchased, which is high in cost, poor in universality of the head-mounted display device, and poor in user experience.

[0032] To solve the above problems, the present application provides an image processing method, a computer device and a storage medium. The method rearranges the pixels in the to-be-displayed image, so that the parallax of each pixel in the rearranged image is within the parallax range corresponding to the current diopter of the wearer's eyeball. In this way, after the rearranged image is synthesized and displayed by the microlens array, the wearer can see a clear image without the need to re-customize the lens, and multiple users can also use the head-mounted display device at different times, improving the universality of the head-mounted display device, reducing the use cost of the head-mounted display device, and greatly improving the user experience.

[0033] Some embodiments of the present application will be described in detail below with reference to the drawings. The following embodiments and features of the embodiments can be combined with each other in the case of no conflict.

[0034] Please refer to FIG. 1, which is a structural schematic block diagram of a computer device according to an embodiment of the present application.

[0035] As shown in FIG. 1, the computer device 100 includes a processor 101 and a memory 102, which are connected through a bus 103, such as an I2C (Inter-integrated Circuit) bus. The computer device 100 can include a head-mounted display device, a server or a mobile terminal. The head-mounted display device can include an AR (Augmented Reality) glasses, a VR (Virtual Reality) glasses, a MR (Mixed Reality) glasses, an AR helmet, a VR helmet and a MR helmet, etc. The server can be a stand-alone server or a cloud server providing cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communication, middleware services, domain name services, security services, CDN (Content Delivery Network), and basic cloud computing services such as big data and artificial intelligence platforms, etc. The mobile terminal can include a smart phone, a notebook computer, a tablet computer or a PC, etc.

[0036] Specifically, the processor 101 is configured to provide computing and control capabilities to support the operation of the entire computer device. The processor 101 can be a CPU (Central Processing Unit), and can also be other general-purpose processors, DSPs (Digital Signal Processors), ASICs (Application Specific Integrated Circuits), FPGAs (Field-Programmable Gate Arrays) or other programmable logic devices, discrete gates or transistor logic devices, discrete hardware components, etc. The general-purpose processor can be a microprocessor or any conventional processor, etc. Specifically, the memory 102 can be a Flash chip, a ROM (Read-Only Memory) disk, an optical disk, a U disk or a mobile hard disk, etc.

[0037] Those skilled in the art can understand that the structure shown in FIG. 1 is only a block diagram of part of the structure related to the embodiment scheme of the present application, and does not constitute a limitation on the computer device to which the embodiment scheme of the present application is applied. The specific computer device can include more or fewer components than those shown in the figure, or combine certain components, or have a different component arrangement.

[0038] The processor 101 is configured to run a computer program stored in the memory 102, and implement any one of the image processing methods provided by the embodiments of the present application when the computer program is executed.

[0039] In an embodiment, the computer device is a head-mounted display device, the head-mounted display device includes a microlens array, the microlens array includes N*M microlenses, N and M are positive integers, and at least one of N and M is greater than or equal to 2, and the processor 101 is configured to run a computer program stored in the memory, and implement the following steps when the computer program is executed:

[0040] Obtain the current refractive power of the wearer's eyeball of the head-mounted display device, and obtain the parallax range corresponding to the current refractive power;

[0041] Obtain each first image corresponding to each microlens, and generate a parallax map between each first image corresponding to each microlens;

[0042] According to the parallax map and the parallax range, determine a to-be-processed image block in the first image, each pixel in the to-be-processed image block corresponding to a parallax outside the parallax range;

[0043] Perform rearrangement processing on each pixel in the to-be-processed image block in each first image to obtain N*M second images, so that each pixel in each second image corresponds to a parallax within the parallax range;

[0044] Control the head-mounted display device to display N*M second images, so that the microlens array synthesizes and displays N*M second images.

[0045] In an embodiment, when the processor 101 implements the step of determining a to-be-processed image block in the first image according to the parallax map and the parallax range, it is configured to implement:

[0046] Obtain the line-of-sight focus of the wearer, and determine a region of interest of the wearer in each first image according to the line-of-sight focus;

[0047] Determine a to-be-processed image block in the region of interest in the first image according to the parallax map and the parallax range.

[0048] In an embodiment, the processor 101, when implementing the rearrangement processing on each pixel in the to-be-processed image block in each of the first images, is configured to:

[0049] determine a respective disparity deviation value of each pixel in the to-be-processed image block in each of the first images according to the disparity map and the disparity range;

[0050] perform the rearrangement processing on each pixel in the to-be-processed image block in each of the first images according to the respective disparity deviation value of each pixel in the to-be-processed image block in each of the first images.

[0051] In an embodiment, the disparity range includes a first disparity and a second disparity, the first disparity is smaller than the second disparity, and the processor 101, when implementing the determination of the respective disparity deviation value of each pixel in the to-be-processed image block in each of the first images according to the disparity map and the disparity range, is configured to:

[0052] for each pixel in the to-be-processed image block in each of the first images, determine a first deviation value between the disparity corresponding to the pixel and the first disparity and a second deviation value between the disparity corresponding to the pixel and the second disparity;

[0053] determine the disparity deviation value corresponding to the pixel according to the first deviation value and the second deviation value.

[0054] In an embodiment, the processor 101, when implementing the rearrangement processing on each pixel in the to-be-processed image block in each of the first images according to the respective target disparity deviation value of each pixel in the to-be-processed image block in each of the first images, is configured to:

[0055] for each to-be-processed image block in each of the first images, determine a respective position offset of each pixel in the to-be-processed image block according to the respective target disparity deviation value of each pixel in the to-be-processed image block;

[0056] adjust the position of each pixel in the to-be-processed image block according to the respective position offset of each pixel in the to-be-processed image block.

[0057] In an embodiment, the processor 101, when implementing the obtaining of the disparity range corresponding to the current refractive power of the wearer's eyeball, is configured to:

[0058] obtain the current refractive power of the wearer's eyeball and a conversion relationship between the refractive power and the disparity range;

[0059] According to the conversion relationship, a parallax range corresponding to the current diopter is determined.

[0060] In an embodiment, the computer device is a server, the server is in communication connection with a head-mounted display device, the head-mounted display device includes a microlens array, the microlens array includes N*M microlenses, N and M are positive integers, and at least one of N and M is greater than or equal to 2, and the processor 101 is configured to run a computer program stored in the memory and implement the following steps when executing the computer program.

[0061] An image processing request sent by the head-mounted display device is obtained, the image processing request is triggered by the head-mounted display device when detecting that the diopter of the wearer's eyeball changes;

[0062] The current diopter of the wearer's eyeball is obtained from the image processing request, and a parallax range corresponding to the current diopter is obtained;

[0063] A first image corresponding to each of the microlenses is obtained, and a parallax map between the first images corresponding to each of the microlenses is generated;

[0064] According to the parallax map and the parallax range, a to-be-processed image block in the first image is determined, each pixel in the to-be-processed image block corresponds to a parallax outside the parallax range;

[0065] Each pixel in the to-be-processed image block in each of the first images is rearranged to obtain N*M second images, so that each pixel in each of the second images corresponds to a parallax within the parallax range;

[0066] N*M second images are sent to the head-mounted display device for the head-mounted display device to display N*M second images, so that the microlens array synthesizes and displays N*M second images.

[0067] It should be noted that those skilled in the art can clearly understand that, for the convenience and brevity of description, the specific working process of the computer device described above can refer to the corresponding process in the following image processing method embodiments, which will not be described here.

[0068] Please refer to FIG. 2, which is a step flowchart of an image processing method provided by an embodiment of the present application. The image processing method can be applied to a head-mounted display device.

[0069] As shown in FIG. 2, the image processing method includes steps S101-S105.

[0070] In step S101, the current diopter of the wearer's eyeball of the head-mounted display device is obtained, and a parallax range corresponding to the current diopter is obtained.

[0071] In this embodiment, the current diopter of the wearer's eyeball can be a diopter input by the wearer in advance, or can be obtained by analyzing the eye image of the wearer through a diopter measurement algorithm, or can be displayed on a virtual visual acuity chart, and the wearer manually confirms the position on the virtual visual acuity chart that he can see clearly, and then determines the diopter corresponding to the position as the current diopter of the wearer's eyeball.

[0072] In some embodiments, the head-mounted display device can be used only by the same wearer, or can be used by different wearers. Specifically, obtaining the current diopter of the wearer's eyeball of the head-mounted display device can include: obtaining iris information and / or fingerprint information of the wearer; and determining the diopter corresponding to the iris information and / or fingerprint information of the wearer as the current diopter of the wearer's eyeball. The diopter relationship table can be pre-stored or configured, and the diopter relationship table stores iris information and / or fingerprint information of different users and corresponding diopters. When the wearer uses the head-mounted display device, the iris information and / or fingerprint information of the wearer can be obtained in real time, and then the diopter relationship table is queried to obtain the current diopter of the wearer's eyeball.

[0073] In some embodiments, obtaining the parallax range corresponding to the current diopter can include: obtaining a conversion relationship between the diopter and the parallax range; and determining the parallax range corresponding to the current diopter according to the conversion relationship. The conversion relationship between the diopter and the parallax range is pre-established, and the conversion relationship between the diopter and the parallax range can include a conversion relationship table between the diopter and the parallax range. The conversion relationship between the diopter and the parallax range can accurately and quickly determine the parallax range corresponding to the current diopter.

[0074] In some embodiments, obtaining the current diopter of the wearer's eyeball of the head-mounted display device and obtaining the parallax range corresponding to the current diopter can include: obtaining the current diopter of the wearer's eyeball of the head-mounted display device and a preset diopter, the preset diopter being a diopter input by the wearer previously; determining that the diopter of the wearer's eyeball has changed when a difference between the current diopter and the preset diopter is greater than a preset difference, and determining that the diopter of the wearer's eyeball has not changed when the difference between the current diopter and the preset diopter is less than or equal to the preset difference; obtaining the parallax range corresponding to the current diopter when the diopter of the wearer's eyeball has changed; and not processing when the diopter of the wearer's eyeball has not changed.

[0075] In step S102, the first image corresponding to each microlens is obtained, and a parallax map between the first images corresponding to each microlens is generated.

[0076] In this embodiment, the head-mounted display device includes a microlens array, and the microlens array includes N*M microlenses, where N and M are positive integers, and at least one of N and M is greater than or equal to 2. The head-mounted display device can include two microlens arrays. For example, as shown in FIG. 3, the head-mounted display device is AR glasses 110, which includes a first lens 111, a second lens 112, a first microlens array 113 located on the inner side (close to the human eye) of the first lens 111, and a second microlens array 114 located on the inner side (close to the human eye) of the second lens 112. The first microlens array 113 is the same as the second microlens array 114.

[0077] In some embodiments, obtaining the first image corresponding to each microlens can include: obtaining a to-be-displayed image and a view angle of the to-be-displayed image; determining a view angle deviation corresponding to each microlens according to a preset view angle corresponding to each microlens and the view angle of the to-be-displayed image; and adjusting the position of each pixel in the to-be-displayed image according to the view angle deviation corresponding to each microlens to obtain the first image corresponding to each microlens, and the view angle of the first image is the preset view angle corresponding to the microlens. The first image corresponding to each microlens is an image of a different view angle of the same scene, and there is a parallax between the first images corresponding to each microlens.

[0078] In step S103, a to-be-processed image block is determined in the first image according to the parallax map and the parallax range, and the parallax corresponding to each pixel in the to-be-processed image block is outside the parallax range.

[0079] In this embodiment, based on the parallax map between the first images corresponding to each microlens, the parallax corresponding to each pixel in each first image can be determined, and then the parallax corresponding to each pixel in each first image can be compared with the parallax range corresponding to the current diopter, and a to-be-processed image block is determined in each first image according to the comparison result. The to-be-processed image block can include one or more.

[0080] For example, the microlens array includes 4 microlenses, and the first images are also 4, which are respectively denoted as first image A, first image B, first image C and first image D. For the first image A, the parallax between each pixel point in the first image A and the corresponding pixel point in the first image B, the first image C and the first image D can be determined through the parallax map between the four first images. If the parallax between each pixel point in the image block A1 in the first image A and the corresponding pixel point in the first image B is out of the parallax range, the parallax between each pixel point in the image block A2 and the corresponding pixel point in the first image C is out of the parallax range, and the parallax between each pixel point in the first image A and the corresponding pixel point in the first image D is in the parallax range, the image block A1 and the image block A2 in the first image A can be determined as the to-be-processed image blocks. In a similar manner, one or more to-be-processed image blocks can be determined in the first image B, the first image C and the first image D.

[0081] In some embodiments, determining the to-be-processed image blocks in the first images according to the parallax map and the parallax range can include: obtaining the visual line focal point of the wearer, and determining the region of interest of the wearer in each first image according to the visual line focal point; and determining the to-be-processed image blocks in the region of interest in the first images according to the parallax map and the parallax range. The visual line focal point of the wearer can be detected by an eye movement sensor in the head-mounted display device. In this embodiment, the region of interest of the wearer in the image is determined according to the visual line focal point of the wearer, and the to-be-processed image blocks are determined in the region of interest, which can effectively reduce the calculation amount of the image blocks and improve the image processing speed.

[0082] In step S104, each pixel in each to-be-processed image block in each first image is subjected to rearrangement processing to obtain N*M second images, so that each pixel in each second image corresponds to a parallax in the parallax range.

[0083] For example, the parallax between each pixel point in the image block A1 in the first image A and the corresponding pixel point in the first image B is out of the parallax range, the parallax between each pixel point in the image block A2 and the corresponding pixel point in the first image C is out of the parallax range, and the parallax between each pixel point in the first image A and the corresponding pixel point in the first image D is in the parallax range. Each pixel point in the image block A1 and the image block A2 in the first image A is subjected to rearrangement processing to obtain the second image corresponding to the first image A, so that each pixel in the second image corresponds to a parallax in the parallax range.

[0084] In an embodiment, as shown in FIG. 4, step S104 includes sub-step S1041 to sub-step S1042.

[0085] In sub-step S1041, according to the disparity map and the disparity range, a respective disparity deviation value of each pixel in each image block to be processed in each first image is determined.

[0086] For example, the image block to be processed in the first image A includes image block A1, which respectively corresponds to image block B1 in the first image B, image block C1 in the first image C and image block D1 in the first image D, then the disparity between each pixel point in the image block A1 and the corresponding pixel point in the image block B1 can be obtained from the disparity map, and based on the disparity and the disparity range, the disparity deviation value between each pixel point in the image block A1 and the corresponding pixel point in the image block B1 is determined, in a similar manner, the disparity deviation value between each pixel point in the image block A1 and the corresponding pixel point in the image block C1 and the disparity deviation value between each pixel point in the image block A1 and the corresponding pixel point in the image block D1 can be determined.

[0087] In some embodiments, the disparity range includes a first disparity and a second disparity, the first disparity is smaller than the second disparity, and according to the disparity map and the disparity range, a respective disparity deviation value of each pixel in each image block to be processed in each first image is determined, which can include: for each pixel in each image block to be processed in each first image, determining a first deviation value between the disparity corresponding to the pixel and the first disparity and a second deviation value between the disparity corresponding to the pixel and the second disparity; and determining the disparity deviation value corresponding to the pixel according to the first deviation value and the second deviation value. Wherein, any disparity in the disparity range is greater than or equal to the first disparity and less than or equal to the second disparity, for example, the disparity range is [d1, d2], then the first disparity is d1 and the second disparity is d2. In this embodiment, the deviation between the disparity of the pixel and the first disparity and the deviation between the disparity of the pixel and the second disparity can accurately determine the disparity deviation value corresponding to the pixel.

[0088] For example, the disparity corresponding to the pixel a in the image block A1 includes the disparity between the pixel a in the image block A1 and the corresponding pixel in the image block B1, denoted as d1, the disparity between the pixel a in the image block A1 and the corresponding pixel in the image block C1, denoted as d2, the disparity between the pixel a in the image block A1 and the corresponding pixel in the image block D1, denoted as d3, and the first disparity is D1, then the first deviation value between the disparity corresponding to the pixel a and the first disparity includes d1-D1, d2-D1 and d3-D1, and the second disparity is D2, then the second deviation value between the disparity corresponding to the pixel a and the second disparity includes d1-D2, d2-D2 and d3-D2, and the disparity deviation value corresponding to the pixel a includes Δd1, Δd2 and Δd3, Δd1 is determined according to d1-D1 and d1-D2, Δd2 is determined according to d2-D1 and d2-D2, and Δd3 is determined according to d3-D1 and d3-D2.

[0089] In some embodiments, the determining the disparity deviation value corresponding to the pixel according to the first disparity deviation value and the second disparity deviation value can include: calculating an average disparity deviation value according to the first disparity deviation value and the second disparity deviation value, and determining the average disparity deviation value as the disparity deviation value corresponding to the pixel. Alternatively, the first disparity deviation value or the second disparity deviation value is determined as the disparity deviation value corresponding to the pixel, or any value between the first disparity deviation value and the second disparity deviation value is determined as the disparity deviation value corresponding to the pixel. For example, the first disparity deviation value corresponding to the pixel a includes d1-D1, d2-D1 and d3-D1, and the second disparity deviation value corresponding to the pixel a includes d1-D2, d2-D2 and d3-D2, and the disparity deviation value corresponding to the pixel a includes ((d1-D1)+(d1-D2)) / 2, ((d2-D1)+(d2-D2)) / 2 and ((d3-D1)+(d3-D2)) / 2.

[0090] The sub-step S1042 includes: performing rearrangement processing on each pixel in the to-be-processed image block in each first image according to the disparity deviation value corresponding to each pixel in the to-be-processed image block in each first image.

[0091] In the embodiment, according to the disparity deviation value corresponding to each pixel in the to-be-processed image block in each first image, the new position of each pixel in the to-be-processed image block can be determined, and then each pixel in the to-be-processed image block is moved to the new position, so as to complete the rearrangement processing of each pixel in the to-be-processed image block, so that the disparity corresponding to each pixel in the second image obtained after the rearrangement processing is within the disparity range.

[0092] For example, the microlens array includes 4 microlenses, and the first images are also 4, which are respectively denoted as first image A, first image B, first image C and first image D. For example, the to-be-processed image block in the first image A is image block A1, the to-be-processed image block in the first image B includes image block B1 and image block B2, the to-be-processed image block in the first image C is image block C1, and the to-be-processed image block in the first image D is image block D1. According to the parallax deviation value corresponding to each pixel in the image block A1, the rearrangement processing is performed on each pixel in the image block A1 in the first image A. According to the parallax deviation value corresponding to each pixel in the image block B1, the rearrangement processing is performed on each pixel in the image block B1 in the first image B. Meanwhile, according to the parallax deviation value corresponding to each pixel in the image block B2, the rearrangement processing is performed on each pixel in the image block B2 in the first image B. According to the parallax deviation value corresponding to each pixel in the image block C1, the rearrangement processing is performed on each pixel in the image block C1 in the first image C. And according to the parallax deviation value corresponding to each pixel in the image block D1, the rearrangement processing is performed on each pixel in the image block D1 in the first image D.

[0093] In some embodiments, the rearrangement processing on each pixel in the to-be-processed image block in each first image according to the parallax deviation value corresponding to each pixel in the to-be-processed image block can include: for the to-be-processed image block in each first image, determining the position offset corresponding to each pixel in the to-be-processed image block according to the parallax deviation value corresponding to each pixel in the to-be-processed image block; and adjusting the position of each pixel in the to-be-processed image block according to the position offset corresponding to each pixel in the to-be-processed image block, so that the parallax corresponding to each pixel in the to-be-processed image block after the position adjustment is within the parallax range.

[0094] In step S105, the N*M second images are displayed so that the microlens array synthesizes and displays the N*M second images.

[0095] In this embodiment, since the parallax corresponding to each pixel in each second image in the N*M second images is within the parallax range, after the N*M second images are synthesized and displayed by the microlens array, the wearer can see a clear image without the need to re-customize the lens, and multiple users can use the head-mounted display device at different times, improving the versatility of the head-mounted display device, reducing the use cost of the head-mounted display device, and greatly improving the user experience.

[0096] It can be understood that the resolution of the second image is W*H, and the resolution of the synthetic image is (NW) * (MH) when the number of the N*M microlenses is known. For example, as shown in FIG. 5, the microlens array in the head-mounted display device includes 4 microlenses, and the second image 11, the second image 12, the third image 13 and the fourth image 14 displayed simultaneously can be synthesized to obtain the synthetic image 20 through the 4 microlenses, and the resolution of the synthetic image 20 is 4 * (MH).

[0097] Referring to FIG. 6, FIG. 6 is a schematic flowchart of steps of an image processing method provided in an embodiment of the present application. The image processing method can be applied to a server, which is in communication connection with a head-mounted display device.

[0098] As shown in FIG. 6, the image processing method includes steps S201 to S206.

[0099] In step S201, an image processing request sent by the head-mounted display device is acquired. The image processing request is triggered by the head-mounted display device when detecting that the refractive power of the wearer's eyeball changes.

[0100] In the embodiment, the head-mounted display device acquires the current refractive power of the wearer's eyeball and a preset refractive power, the preset refractive power being the refractive power input by the wearer before; when the difference between the current refractive power and the preset refractive power is greater than a preset difference, it is determined that the refractive power of the wearer's eyeball changes, and at this time, the image processing request carrying the current refractive power of the wearer's eyeball is sent to the server.

[0101] In some embodiments, the head-mounted display device can be used only by the same wearer, or can be used by different wearers. Specifically, the head-mounted display device acquiring the current refractive power of the wearer's eyeball can include: the head-mounted display device acquiring the iris information and / or fingerprint information of the wearer; and determining the refractive power corresponding to the iris information and / or fingerprint information of the wearer as the current refractive power of the wearer's eyeball. The refractive power relationship table in which the iris information and / or fingerprint information of different users and the corresponding refractive power are stored can be stored or configured in advance, and when the wearer uses the head-mounted display device, the iris information and / or fingerprint information of the wearer can be acquired in real time, and then the refractive power relationship table is queried to obtain the current refractive power of the wearer's eyeball.

[0102] In step S202, the current refractive power of the wearer's eyeball is acquired from the image processing request, and the parallax range corresponding to the current refractive power is acquired.

[0103] In this embodiment, when the server obtains the image processing request sent by the head-mounted display device, the current refractive power of the wearer's eyeball is obtained from the image processing request, and the parallax range corresponding to the current refractive power is obtained. Specifically, the conversion relationship between the refractive power and the parallax range is obtained; according to the conversion relationship, the parallax range corresponding to the current refractive power is determined. The conversion relationship between the refractive power and the parallax range is pre-established, and the conversion relationship between the refractive power and the parallax range can include a conversion relationship table between the refractive power and the parallax range.

[0104] In step S203, the first image corresponding to each microlens is obtained, and a parallax map between the first images corresponding to each microlens is generated.

[0105] In this embodiment, the server obtaining the first image corresponding to each microlens can include: obtaining a to-be-displayed image and a field of view of the to-be-displayed image; determining a field of view deviation of each microlens according to a preset field of view corresponding to each microlens and the field of view of the to-be-displayed image; and adjusting the position of each pixel in the to-be-displayed image according to the field of view deviation of each microlens to obtain the first image corresponding to each microlens, and the field of view of the first image is the preset field of view corresponding to the microlens. The first image corresponding to each microlens is an image of a different perspective of the same scene, and there is a parallax between the first images corresponding to each microlens.

[0106] In step S204, the to-be-processed image block in the first image is determined according to the parallax map and the parallax range, and the parallax corresponding to each pixel in the to-be-processed image block is outside the parallax range.

[0107] In this embodiment, based on the parallax map between the first images corresponding to each microlens, the server can determine the parallax corresponding to each pixel in each first image, and then compare the parallax corresponding to each pixel in each first image with the parallax range corresponding to the current refractive power, and determine the to-be-processed image block in each first image according to the comparison result. The to-be-processed image block can include one or more.

[0108] In some embodiments, determining the to-be-processed image block in the first image according to the parallax map and the parallax range can include: obtaining the line-of-sight focus of the wearer, and determining the region of interest of the wearer in each first image according to the line-of-sight focus; and determining the to-be-processed image block in the region of interest in the first image according to the parallax map and the parallax range. The line-of-sight focus of the wearer can be detected by an eye movement sensor in the head-mounted display device. In this embodiment, the region of interest of the wearer in the image is determined through the line-of-sight focus of the wearer, and the to-be-processed image block is determined in the region of interest, which can effectively reduce the calculation amount of the image block and improve the image processing speed.

[0109] Step S205, performing rearrangement processing on each pixel in the to-be-processed image block in each first image to obtain N*M second images, so that each pixel in each second image corresponds to a respective parallax within the parallax range.

[0110] In this embodiment, the server can determine, according to the parallax map and the parallax range, a respective parallax deviation value of each pixel in the to-be-processed image block in each first image, and perform rearrangement processing on each pixel in the to-be-processed image block in each first image according to the respective parallax deviation value of each pixel in the to-be-processed image block in each first image.

[0111] In some embodiments, determining, according to the parallax map and the parallax range, a respective parallax deviation value of each pixel in the to-be-processed image block in each first image can include: for each pixel in the to-be-processed image block in each first image, determining a first deviation value between the parallax corresponding to the pixel and the first parallax and a second deviation value between the parallax corresponding to the pixel and the second parallax; and determining the parallax deviation value corresponding to the pixel according to the first deviation value and the second deviation value. This embodiment can accurately determine the parallax deviation value corresponding to the pixel by the deviation between the parallax of the pixel and the first parallax and the deviation between the parallax of the pixel and the second parallax.

[0112] In some embodiments, determining, according to the first deviation value and the second deviation value, the parallax deviation value corresponding to the pixel can include: calculating an average deviation value according to the first deviation value and the second deviation value, and determining the average deviation value as the parallax deviation value corresponding to the pixel. Alternatively, the first deviation value or the second deviation value is determined as the parallax deviation value corresponding to the pixel, or any value between the first deviation value and the second deviation value is determined as the parallax deviation value corresponding to the pixel.

[0113] Step S206, sending the N*M second images to the head-mounted display device, so that the head-mounted display device displays the N*M second images, so that the microlens array synthesizes and displays the N*M second images.

[0114] In this embodiment, the head-mounted display device sends an image processing request to the server when detecting that the refractive power of the wearer's eyeball changes, so that the server completes the processing of the to-be-displayed image based on the image processing request, and then returns the processed image to the head-mounted display device for display. The head-mounted display device does not need to perform image processing operations, can save the computing power of the head-mounted display device, and the parallax of each pixel in the processed image is within the parallax range. After the N*M second images are synthesized and displayed by the microlens array, the wearer can see a clear image without the need to customize the lens again. Multiple users can also use the head-mounted display device at different times, improving the versatility of the head-mounted display device, reducing the use cost of the head-mounted display device, and greatly improving the user experience.

[0115] The application further provides an image processing device applied to a head-mounted display device, the head-mounted display device comprising a microlens array, the microlens array comprising N*M microlenses, N and M being positive integers, and at least one of N and M being greater than or equal to 2, the image processing device comprising:

[0116] an acquisition module configured to acquire the current refractive power of the wearer's eyeball of the head-mounted display device, and acquire a parallax range corresponding to the current refractive power;

[0117] the acquisition module is further configured to acquire a first image corresponding to each microlens, and generate a parallax map between the first images corresponding to each microlens;

[0118] a determination module configured to determine a to-be-processed image block in the first image according to the parallax map and the parallax range, each pixel in the to-be-processed image block corresponding to a parallax outside the parallax range;

[0119] a processing module configured to perform rearrangement processing on each pixel in the to-be-processed image block in each first image to obtain N*M second images, so that each pixel in each second image corresponds to a parallax within the parallax range;

[0120] a control module further configured to control the head-mounted display device to display N*M second images, so that the microlens array synthesizes and displays N*M second images.

[0121] In some embodiments, the determination module is further configured to:

[0122] acquire a line-of-sight focal point of the wearer, and determine a region of interest of the wearer in each first image according to the line-of-sight focal point;

[0123] According to the disparity map and the disparity range, a to-be-processed image block in the region of interest in the first image is determined.

[0124] In some embodiments, the processing module is further configured to:

[0125] According to the disparity map and the disparity range, a respective disparity deviation value of each pixel in the to-be-processed image block in each of the first images is determined.

[0126] According to the respective disparity deviation value of each pixel in the to-be-processed image block in each of the first images, a rearrangement processing is performed on each pixel in the to-be-processed image block in each of the first images.

[0127] In some embodiments, the disparity range includes a first disparity and a second disparity, the first disparity is smaller than the second disparity, and the processing module is further configured to:

[0128] For each pixel in the to-be-processed image block in each of the first images, a first deviation value between the disparity corresponding to the pixel and the first disparity and a second deviation value between the disparity corresponding to the pixel and the second disparity are determined.

[0129] According to the first deviation value and the second deviation value, a disparity deviation value corresponding to the pixel is determined.

[0130] In some embodiments, the processing module is further configured to:

[0131] For each to-be-processed image block in each of the first images, according to the respective disparity deviation value of each pixel in the to-be-processed image block, a respective position offset of each pixel in the to-be-processed image block is determined.

[0132] According to the respective position offset of each pixel in the to-be-processed image block, a position of each pixel in the to-be-processed image block is adjusted.

[0133] In some embodiments, the obtaining module is further configured to:

[0134] Obtain a conversion relationship between diopter and disparity range.

[0135] According to the conversion relationship, a disparity range corresponding to the current diopter is determined.

[0136] In some embodiments, the image processing apparatus includes:

[0137] The sending module is configured to, in a case where the refractive power of the wearer's eyeball of the head-mounted display device changes, send an image processing request carrying a current refractive power of the wearer's eyeball to a server, so that the server acquires the current refractive power from the image processing request and acquires a parallax range corresponding to the current refractive power when the image processing request is received; acquires a first image corresponding to each of the microlenses respectively, and generates a parallax map between the first images corresponding to the microlenses respectively; determines a to-be-processed image block in the first image according to the parallax map and the parallax range, each pixel in the to-be-processed image block corresponding to a parallax outside the parallax range respectively; performs rearrangement processing on each pixel in the to-be-processed image block in each of the first images to obtain N*M second images, so that each pixel in each of the second images corresponds to a parallax within the parallax range respectively; and sends the N*M second images to the head-mounted display device.

[0138] The receiving module is configured to receive and display the N*M second images sent by the server, so that the microlens array synthesizes and displays the N*M second images.

[0139] It should be noted that, for the convenience and brevity of description, the specific working process of the image processing device described above can refer to the corresponding process in the foregoing image processing method embodiments, and will not be described here.

[0140] The image processing system provided in the embodiment of the present application comprises a head-mounted display device and a server, the head-mounted display device is in communication connection with the server, the head-mounted display device comprises a microlens array, the microlens array comprises N*M microlenses, N and M are positive integers, and at least one of N and M is greater than or equal to 2;

[0141] The head-mounted display device is configured to, in a case where the refractive power of the wearer's eyeball of the head-mounted display device changes, send an image processing request carrying a current refractive power of the wearer's eyeball to a server;

[0142] The server is configured to acquire the current diopter from the image processing request and acquire a parallax range corresponding to the current diopter when the image processing request is received; acquire a first image corresponding to each of the microlenses and generate a parallax map between the first images corresponding to each of the microlenses; determine a to-be-processed image block in the first image according to the parallax map and the parallax range, each pixel in the to-be-processed image block corresponding to a parallax outside the parallax range; perform rearrangement processing on each pixel in the to-be-processed image block in each of the first images to obtain N*M second images, so that each pixel in each of the second images corresponds to a parallax within the parallax range; and send the N*M second images to the head-mounted display device.

[0143] The head-mounted display device is further configured to receive and display the N*M second images sent by the server, so that the microlens array synthesizes and displays the N*M second images.

[0144] It should be noted that, for the convenience and brevity of description, the specific working process of the image processing system described above can refer to the corresponding process in the foregoing image processing method embodiments, which will not be described here.

[0145] The embodiment of the present application also provides a storage medium for computer readable storage, the storage medium storing one or more programs, the one or more programs being executable by one or more processors to implement any one of the image processing methods provided in the specification of the embodiment of the present application.

[0146] The storage medium can be an internal storage unit of the head-mounted display device or the server, for example, a hard disk or a memory of the head-mounted display device or the server. The storage medium can also be an external storage device of the head-mounted display device or the server, for example, a plug-in hard disk, a smart media card (SMC), a secure digital (SD) card, a flash card, etc.

[0147] Those skilled in the art can understand that all or some of the steps in the methods disclosed above, the functional modules / units in the systems and devices can be implemented by software, firmware, hardware, or a combination thereof. In hardware embodiments, the division between the functional modules / units mentioned in the above description does not necessarily correspond to the division of physical components; for example, one physical component can have multiple functions, or one function or step can be performed by several physical components in cooperation. Some or all of the physical components can be implemented as software executed by a processor, such as a central processing unit, a digital signal processor, or a microprocessor, or as hardware, or as an integrated circuit, such as an application-specific integrated circuit. Such software can be distributed on computer-readable media, which can include computer storage media (or non-transitory media) and communication media (or transitory media). As is well known to those skilled in the art, the term computer storage media includes volatile and non-volatile, removable and non-removable media implemented in any method or technology for storage of information such as computer readable instructions, data structures, program modules or other data. Computer storage media includes, but is not limited to, RAM, ROM, EEPROM, flash memory or other memory technology, CD-ROM, digital versatile disks (DVD) or other optical disk storage, magnetic cassettes, magnetic tapes, magnetic disk storage or other magnetic storage devices, or any other medium which can be used to store the desired information and which can be accessed by a computer. In addition, it is well known to those skilled in the art that communication media typically embodies computer readable instructions, data structures, program modules or other data in a modulated data signal such as a carrier wave or other transport mechanism and can include any information delivery media.

[0148] It should be understood that the term "and / or" as used herein, refers to any combination of associated listed items, including all possible combinations, and includes any of the associated listed items. It is to be noted that the terms "comprising", "including", or any other variant thereof, are intended to cover a non-exclusive inclusion, such that processes, methods, articles, or systems that comprise a list of elements do not include only those elements, but can also include other elements not expressly listed or inherent to such processes, methods, articles, or systems. Without more limitations, an element defined by the phrase "comprising a" does not exclude the existence of additional identical elements in the process, method, article, or system including the element.

[0149] The above-mentioned serial numbers of the embodiments of the present application are only for description, and do not represent the advantages and disadvantages of the embodiments. The above is only a specific embodiment of the present application, but the protection scope of the present application is not limited thereto. Any person skilled in the art can easily think of various equivalent modifications or replacements within the technical range disclosed by the present application, and these modifications or replacements should be covered in the protection scope of the present application. Therefore, the protection scope of the present application should be subject to the protection scope of the claims.

Claims

1. An image processing method, wherein, The method is applied to a head-mounted display device, the head-mounted display device comprising a microlens array, the microlens array comprising N*M microlenses, N and M being positive integers, and at least one of N and M being greater than or equal to 2, the method comprising: obtaining a current diopter of an eyeball of a wearer of the head-mounted display device, and obtaining a parallax range corresponding to the current diopter; obtaining a first image corresponding to each of the microlenses, and generating a parallax map between the first images corresponding to the microlenses; determining, according to the parallax map and the parallax range, a to-be-processed image block in the first image, each pixel in the to-be-processed image block corresponding to a parallax outside the parallax range; performing rearrangement processing on each pixel in the to-be-processed image block in each of the first images to obtain N*M second images, so that each pixel in each of the second images corresponds to a parallax within the parallax range; displaying the N*M second images so that the microlens array synthesizes and displays the N*M second images.

2. The image processing method of claim 1, wherein, The method further comprises: obtaining a line-of-sight focus of the wearer, and determining a region of interest of the wearer in each of the first images according to the line-of-sight focus; determining, according to the parallax map and the parallax range, a to-be-processed image block in the region of interest in the first image.

3. The image processing method of claim 1, wherein, The method further comprises: determining, according to the parallax map and the parallax range, a parallax deviation value corresponding to each pixel in the to-be-processed image block in each of the first images; performing rearrangement processing on each pixel in the to-be-processed image block in each of the first images according to the parallax deviation value corresponding to each pixel in the to-be-processed image block in each of the first images.

4. The image processing method of claim 3, wherein, The parallax range comprises a first parallax and a second parallax, the first parallax being smaller than the second parallax, and the method further comprises: for each pixel in the to-be-processed image block in each of the first images, determining a first deviation value between the parallax corresponding to the pixel and the first parallax, and a second deviation value between the parallax corresponding to the pixel and the second parallax; determining a parallax deviation value corresponding to the pixel according to the first deviation value and the second deviation value.

5. The image processing method of claim 3, wherein, The method further comprises: for each to-be-processed image block in each of the first images, determining a position offset corresponding to each pixel in the to-be-processed image block according to the parallax deviation value corresponding to each pixel in the to-be-processed image block. Adjust a position of each pixel in the to-be-processed image block according to a position offset corresponding to each pixel in the to-be-processed image block.

6. The image processing method of any one of claims 1-5, wherein, The obtaining of the disparity range corresponding to the current diopter includes: Obtaining a conversion relationship between a diopter and a disparity range; According to the conversion relationship, the disparity range corresponding to the current diopter is determined.

7. The image processing method of any one of claims 1-5, wherein, The obtaining of the current diopter of the eyeball of the wearer of the head-mounted display device includes: Obtaining iris information and / or fingerprint information of the wearer of the head-mounted display device; The diopter corresponding to the iris information and / or the fingerprint information of the wearer is determined as the current diopter of the eyeball of the wearer.

8. The image processing method of any one of claims 1-5, wherein, The obtaining of the first image corresponding to each microlens includes: Obtaining a to-be-displayed image and a viewing angle of the to-be-displayed image; According to a preset viewing angle corresponding to each microlens and the viewing angle of the to-be-displayed image, a viewing angle deviation corresponding to each microlens is determined; According to the viewing angle deviation corresponding to each microlens, a position of each pixel in the to-be-displayed image is adjusted to obtain a first image corresponding to each microlens.

9. The image processing method of any one of claims 1-5, wherein, The obtaining of the current diopter of the eyeball of the wearer of the head-mounted display device and the obtaining of the disparity range corresponding to the current diopter include: Obtaining a current diopter and a preset diopter of the eyeball of the wearer of the head-mounted display device, the preset diopter being a diopter input by the wearer before; In a case where a difference between the current diopter and the preset diopter is greater than a preset difference, it is determined that the diopter of the eyeball of the wearer changes, and the disparity range corresponding to the current diopter is obtained.

10. An image processing method, wherein, The method is applied to a server in communication connection with a head-mounted display device, the head-mounted display device includes a microlens array, the microlens array includes N*M microlenses, N and M are positive integers, and at least one of N and M is greater than or equal to 2, and the method includes: Obtaining an image processing request sent by the head-mounted display device, the image processing request being triggered by the head-mounted display device in a case where it is detected that the diopter of the eyeball of the wearer changes; Obtaining a current diopter of the eyeball of the wearer from the image processing request, and obtaining a disparity range corresponding to the current diopter; Obtaining a first image corresponding to each microlens, and generating a disparity map between the first images corresponding to the microlenses; According to the disparity map and the disparity range, a to-be-processed image block is determined in the first image, each pixel in the to-be-processed image block corresponding to a disparity outside the disparity range; Each pixel in the to-be-processed image block in each first image is rearranged to obtain N*M second images, so that each pixel in each second image corresponds to a disparity within the disparity range; N*M second images are sent to the head-mounted display device for displaying N*M second images by the head-mounted display device, so that N*M second images are synthesized and displayed by the microlens array.

11. The image processing method of claim 10, wherein, The determining the image block to be processed in the first image according to the disparity map and the disparity range comprises: obtaining a line-of-sight focus of the wearer, and determining a region of interest of the wearer in each of the first images according to the line-of-sight focus; determining the image block to be processed in the region of interest in the first image according to the disparity map and the disparity range.

12. The image processing method of claim 10, wherein, The rearranging each pixel in the image block to be processed in each of the first images comprises: determining a respective disparity deviation value of each pixel in the image block to be processed in each of the first images according to the disparity map and the disparity range; rearranging each pixel in the image block to be processed in each of the first images according to the respective disparity deviation value of each pixel in the image block to be processed in each of the first images.

13. The image processing method of claim 12, wherein, The disparity range comprises a first disparity and a second disparity, the first disparity is smaller than the second disparity, and the determining the respective disparity deviation value of each pixel in the image block to be processed in each of the first images according to the disparity map and the disparity range comprises: for each pixel in the image block to be processed in each of the first images, determining a first deviation value between the disparity corresponding to the pixel and the first disparity and a second deviation value between the disparity corresponding to the pixel and the second disparity; and determining the respective disparity deviation value of the pixel according to the first deviation value and the second deviation value.

14. The image processing method of claim 12, wherein, The rearranging each pixel in the image block to be processed in each of the first images according to the respective disparity deviation value of each pixel in the image block to be processed in each of the first images comprises: for the image block to be processed in each of the first images, determining a respective position offset of each pixel in the image block to be processed according to the respective disparity deviation value of each pixel in the image block to be processed; and adjusting the position of each pixel in the image block to be processed according to the respective position offset of each pixel in the image block to be processed.

15. The image processing method of any one of claims 10-14, wherein, The obtaining the disparity range corresponding to the current diopter comprises: obtaining a conversion relationship between a diopter and a disparity range; and determining the disparity range corresponding to the current diopter according to the conversion relationship.

16. The image processing method of any one of claims 10-14, wherein, The obtaining the first image corresponding to each of the microlenses comprises: obtaining a to-be-displayed image and a viewing angle of the to-be-displayed image; determining a respective viewing angle deviation of each of the microlenses according to a preset viewing angle of each of the microlenses and the viewing angle of the to-be-displayed image; and adjusting the position of each pixel in the to-be-displayed image according to the respective viewing angle deviation of each of the microlenses to obtain the first image corresponding to each of the microlenses.

17. The image processing method of any one of claims 10-14, wherein, The obtaining the current diopter of the wearer's eyeball of the head-mounted display device and the obtaining the disparity range corresponding to the current diopter comprise: Obtaining a current diopter of an eye of a wearer of a head-mounted display device and a preset diopter, the preset diopter being a diopter previously input by the wearer; In a case where a difference between the current diopter and the preset diopter is greater than a preset difference, determining that the diopter of the eye of the wearer has changed, and obtaining a parallax range corresponding to the current diopter.

18. A head-mounted display device, wherein, The head-mounted display device comprises a processor, a memory, a computer program stored on the memory and executable by the processor, and a data bus for realizing connection communication between the processor and the memory, wherein the computer program, when executed by the processor, realizes the following steps: Obtaining a current diopter of an eye of a wearer of a head-mounted display device and a parallax range corresponding to the current diopter, the head-mounted display device comprising a microlens array, the microlens array comprising N*M microlenses, N and M being positive integers, and at least one of N and M being greater than or equal to 2; Obtaining a first image corresponding to each of the microlenses, and generating a parallax map between the first images corresponding to each of the microlenses; According to the parallax map and the parallax range, determining a to-be-processed image block in the first image, each pixel in the to-be-processed image block corresponding to a parallax outside the parallax range; Performing rearrangement processing on each pixel in the to-be-processed image block in each of the first images to obtain N*M second images, so that each pixel in each of the second images corresponds to a parallax within the parallax range; Displaying N*M second images so that the microlens array synthesizes and displays the N*M second images.

19. A server, wherein, The server comprises a processor, a memory, a computer program stored on the memory and executable by the processor, and a data bus for realizing connection communication between the processor and the memory, wherein the computer program, when executed by the processor, realizes the following steps: Obtaining an image processing request sent by a head-mounted display device, the image processing request being triggered by the head-mounted display device in a case where it is detected that the diopter of an eye of a wearer has changed, the server being in communication connection with the head-mounted display device, the head-mounted display device comprising a microlens array, the microlens array comprising N*M microlenses, N and M being positive integers, and at least one of N and M being greater than or equal to 2; Obtaining a current diopter of an eye of a wearer from the image processing request, and obtaining a parallax range corresponding to the current diopter; Obtaining a first image corresponding to each of the microlenses, and generating a parallax map between the first images corresponding to each of the microlenses; According to the parallax map and the parallax range, determining a to-be-processed image block in the first image, each pixel in the to-be-processed image block corresponding to a parallax outside the parallax range; perform rearrangement processing on each pixel in the to-be-processed image block in each of the first images, to obtain N*M second images, so that each pixel in each of the second images corresponds to a disparity within the disparity range; send the N*M second images to the head-mounted display device, so that the head-mounted display device displays the N*M second images, so that the micro-lens array synthesizes and displays the N*M second images.

20. A storage medium for computer readable storage, wherein, The storage medium stores one or more programs, which can be executed by one or more processors to implement the following steps: obtain a current diopter of an eyeball of a wearer of a head-mounted display device, and obtain a disparity range corresponding to the current diopter, the head-mounted display device comprising a micro-lens array, the micro-lens array comprising N*M micro-lenses, N and M being positive integers, and at least one of N and M being greater than or equal to 2; obtain a first image corresponding to each of the micro-lenses, and generate a disparity map between the first images corresponding to each of the micro-lenses; determine, according to the disparity map and the disparity range, a to-be-processed image block in the first image, each pixel in the to-be-processed image block corresponding to a disparity outside the disparity range; perform rearrangement processing on each pixel in the to-be-processed image block in each of the first images, to obtain N*M second images, so that each pixel in each of the second images corresponds to a disparity within the disparity range; display the N*M second images, so that the micro-lens array synthesizes and displays the N*M second images.

Images