Parallax adjustment method, model training method, apparatus, device, and storage medium

Abstract

A parallax adjustment method. The method comprises: acquiring a human eye image (S101); estimating the parallax of the human eye image on the basis of a preset parallax estimation model to obtain a parallax to be adjusted (S102); determining a target adjusted parallax on the basis of a preset system configuration, a system configuration of a head-mounted display device, and the parallax to be adjusted (S103); and adjusting a display parameter of a micro-display on the basis of the target adjusted parallax (S104). According to the present application, the parallax to be adjusted is estimated by means of the preset parallax estimation model, and considering that a device for training the parallax estimation model may be different from the actually used device, in order to improve the accuracy of parallax adjustment, the target adjusted parallax is determined on the basis of the preset system configuration, the actually used system configuration, and the parallax to be adjusted, solving the technical problem of the parallax adjustment of the head-mounted display device needing to depend on a depth camera. Further provided are a model training method, an apparatus, a device, and a storage medium.

Description

Parallax adjustment method, model training method, device, equipment and storage medium

[0001] This application claims priority to the Chinese patent application filed with the China Patent Office on September 12, 2023, with application number 2023111763867, and invention name “Parallax adjustment method, model training method, device, equipment and storage medium”, the entire contents of which are incorporated by reference into this application. Technical Field

[0002] The present application relates to the technical field of head-mounted display devices, and in particular to a parallax adjustment method, a model training method, an apparatus, a device, and a storage medium. Background Art

[0003] The optical display system of a head-mounted display typically consists of a microdisplay and optical components. The optical components project the image displayed on the microdisplay into each of the user's left and right eyes, forming a virtual image at a fixed depth. Because the depth of the virtual image is fixed, when the user focuses on a real object, if the depth of the object differs from the depth of the virtual image, the eye muscles must adjust to focus on the virtual image. Prolonged switching of focus can cause eye fatigue.

[0004] Related technologies typically use a depth camera to determine the depth of a real object as viewed by the human eye, then adjust the parallax of the virtual image to correspond to that depth, thereby ensuring that the depth of the virtual content matches that of the real object. However, this reliance on a depth camera to adjust the parallax is not conducive to parallax adjustment for head-mounted display devices.

[0005] Summary of the Invention

[0006] The main purpose of this application is to provide a parallax adjustment method, a model training method, an apparatus, a device and a storage medium to solve the technical problem in the related art that the parallax adjustment of the head-mounted display device needs to rely on a depth camera.

[0007] In a first aspect, the present application provides a parallax adjustment method, which is applied to a head-mounted display device, wherein the head-mounted display device includes a microdisplay, including:

[0008] Acquire human eye images;

[0009] Based on a preset disparity estimation model, the disparity of the human eye image is estimated to obtain the disparity to be adjusted;

[0010] Determining a target adjustment parallax according to a preset system configuration, a system configuration of the head-mounted display device, and the parallax to be adjusted;

[0011] The parallax is adjusted according to the target, and the display parameters of the microdisplay are adjusted.

[0012] In a second aspect, the present application provides a method for training a disparity estimation model, which is applied to a head-mounted display device. The training method includes:

[0013] Acquire a plurality of training data, the training data including human eye images and annotated disparities corresponding to the human eye images;

[0014] estimating the disparity of the human eye image based on a disparity estimation model to obtain a current disparity;

[0015] adjusting model parameters of the disparity estimation model according to a deviation between the current disparity and the marked disparity;

[0016] The adjusted disparity estimation model is stored, and the stored disparity estimation model is used to estimate the disparity of the human eye image to obtain the disparity to be adjusted.

[0017] In a third aspect, the present application further provides a parallax adjustment device, which is applied to a head-mounted display device, wherein the head-mounted display device includes a microdisplay, including:

[0018] An acquisition module, used for acquiring human eye images;

[0019] an estimation module, configured to estimate the disparity of the human eye image based on a preset disparity estimation model to obtain a disparity to be adjusted;

[0020] a determination module, configured to determine a target adjustment parallax according to a preset system configuration, a system configuration of the head mounted display device, and the parallax to be adjusted;

[0021] The adjustment module is used to adjust the parallax according to the target and adjust the display parameters of the micro display.

[0022] In a fourth aspect, the present application further provides a head-mounted display device, the head-mounted display device comprising a memory and a processor;

[0023] The memory is used to store computer programs;

[0024] The processor is configured to execute the computer program and implement the above-mentioned disparity adjustment method or disparity estimation model training method when executing the computer program.

[0025] In a fifth aspect, the present application also provides a computer-readable storage medium, on which a computer program is stored. When the computer program is executed by a processor, the steps of the above-mentioned disparity adjustment method or disparity estimation model training method are implemented.

[0026] The present application provides a parallax adjustment method, a model training method, an apparatus, a device and a storage medium. The method includes: obtaining a human eye image; estimating the parallax of the human eye image based on a preset parallax estimation model to obtain the parallax to be adjusted; determining the target adjustment parallax according to the preset system configuration, the system configuration of the head-mounted display device and the parallax to be adjusted; and adjusting the display parameters of the microdisplay according to the target adjustment parallax. The present application estimates the parallax to be adjusted through a preset disparity estimation model without relying on a depth camera. The target adjustment parallax is determined based on the preset system configuration of the head-mounted display device for training the disparity estimation model, the system configuration of the head-mounted display device actually used and the parallax to be adjusted, and finally the display parameters of the microdisplay are adjusted according to the target adjustment parallax. This can further improve the accuracy of the parallax adjustment when the device for training the disparity estimation model is different from the device actually used, thereby solving the technical problem in the related art that the parallax adjustment of the head-mounted display device needs to rely on a depth camera. BRIEF DESCRIPTION OF THE DRAWINGS

[0027] In order to more clearly illustrate the technical solutions of the embodiments of the present application, the following is a brief introduction to the drawings required for use in the description of the embodiments. Obviously, the drawings described below are some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without any creative work.

[0028] FIG1 is a schematic flow chart of a parallax adjustment method provided in an embodiment of the present application;

[0029] FIG2 is a schematic diagram of a connection between a server and a terminal provided in an embodiment of the present application;

[0030] FIG3 is a flow chart of a method for training a disparity estimation model according to an embodiment of the present application;

[0031] FIG4 is a schematic block diagram of a parallax adjustment device provided in an embodiment of the present application;

[0032] FIG5 is a schematic block diagram of the structure of a head-mounted display device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0033] The following will be combined with the drawings in the embodiments of this application to clearly and completely describe the technical solutions in the embodiments of this application. Obviously, the embodiments described are part of the embodiments of this application, not all of them. Based on the embodiments in this application, all other embodiments obtained by ordinary technicians in this field without making creative efforts are within the scope of protection of this application.

[0034] The flowcharts shown in the accompanying drawings are for illustrative purposes only and do not necessarily include all contents and operations / steps, nor must they be executed in the order described. For example, some operations / steps may be decomposed, combined, or partially merged, so the actual execution order may vary depending on the actual situation.

[0035] The embodiments of the present application provide a parallax adjustment method, a model training method, an apparatus, a device, and a storage medium. Among them, a parallax adjustment method is applied to a head-mounted display device, which includes an optical element and a microdisplay, and uses the optical element to project the image displayed by the microdisplay into the left and right eyes of a person to form a virtual image. It should be noted that the head-mounted display device can be AR glasses or VR glasses. In addition, the parallax adjustment method can also be applied to a server, which can be a separate server or a cloud server that provides basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communications, middleware services, domain name services, security services, content delivery networks (CDNs), and big data and artificial intelligence platforms.

[0036] The following describes some embodiments of the present application in detail with reference to the accompanying drawings. In the absence of conflict, the following embodiments and features in the embodiments may be combined with each other.

[0037] Please refer to FIG. 1 , which is a flow chart of a parallax adjustment method provided in an embodiment of the present application.

[0038] It should be noted that the parallax adjustment method provided in the embodiment of the present application can be used in a head-mounted display device or in a server.

[0039] As shown in FIG2 , the parallax adjustment method is applied to a server, the server and the terminal are in communication connection, and the display parameters obtained by the parallax adjustment method can be sent to the terminal electronic device via the server.

[0040] In a specific implementation, the terminal includes a head-mounted display device, which can be smart glasses such as AR glasses and VR glasses; the server can be a separate server or a server cluster, or a cloud server that provides cloud computing services.

[0041] As shown in FIG. 1 , the parallax adjustment method includes steps S101 to S104 .

[0042] Step S101: Acquire a human eye image.

[0043] The camera on the head-mounted display device can capture the user's left and right eyes to obtain a human eye image, including left and right eye images. The camera can be an RGB camera, a monochrome camera, or a near-infrared camera. If a near-infrared camera is used to capture the human eye, an infrared LED can be used to illuminate the eye to ensure that the user's eye image is clearly visible on the near-infrared camera.

[0044] Step S102 : estimating the disparity of the human eye image based on a preset disparity estimation model to obtain the disparity to be adjusted.

[0045] Parallax refers to the difference in direction when observing the same object from two points at a certain distance. In other words, it refers to the difference in the frame captured by the viewfinder and lens in a camera. Because there is a distance between a person's left and right eyes, there is a slight difference in the perspective of the left and right eyes. This difference causes the scene observed by the left and right eyes to shift slightly. In other words, the distance and direction of the left and right eyes from the actual object are different. Therefore, there will be a certain difference between the left eye image and the right eye image obtained after the actual object is projected into the human eye. In practical applications, this difference can be reduced by performing parallax adjustment. Related technologies generally rely on depth cameras, that is, the depth camera is used to obtain the depth of the real object at which the human eye is looking, and then the parallax of the virtual image is adjusted to correspond to the depth. However, this embodiment does not require the use of a depth camera, but instead uses a disparity estimation model to estimate the disparity of the left and right eye images. Specifically, the disparity estimation model is pre-trained and optimized. In this embodiment, the acquired left and right eye images are input into a preset disparity estimation model to obtain a highly accurate parallax to be adjusted.

[0046] Step S103: determining a target adjustment parallax according to the preset system configuration, the system configuration of the head mounted display device, and the parallax to be adjusted.

[0047] Typically, different head-mounted display devices have different assembly conditions for their internal structural components. For example, different devices may have different positions and angles for observing objects under the same parallax standard. System configuration refers to the tolerances of the assembly of various components in a head-mounted display device, such as the resolution of the microdisplay, the placement and angle of the microdisplay, etc. The system configurations of head-mounted display devices produced under different production conditions are generally different. Based on this, when two devices have different system configurations and the same parallax setting conditions, the depth positions of the images obtained by the two devices under the parallax may be different. For example, the depth positions of the images obtained by head-mounted display device A based on its own system configuration and the images obtained by head-mounted display device B based on its own system configuration are different, resulting in differences in actual parallax. Based on this, considering that the head-mounted display device for training the disparity estimation model and the head-mounted display device actually used may have different system configurations, this embodiment can first determine whether the system configuration of the head-mounted display device for training the disparity estimation model and the system configuration of the head-mounted display device actually used are the same after estimating the disparity to be adjusted through the disparity estimation model, and then determine the target adjustment disparity in combination with the disparity to be adjusted.

[0048] Step S104: adjusting the parallax according to the target and adjusting the display parameters of the microdisplay.

[0049] Generally, a head-mounted display device may include two microdisplays, one for each user's left eye and one for each right eye. Therefore, once a target adjustment parallax is determined, the display parameters of the two microdisplays can be adjusted based on the target adjustment parallax. Specifically, the lateral coordinates of the images displayed by the two microdisplays can be adjusted so that, after the lateral coordinates of the two microdisplays are moved and adjusted, the parallax between the captured left-eye image and the right-eye image matches the target adjustment parallax.

[0050] The parallax adjustment method provided in the above embodiment includes acquiring a human eye image; estimating the parallax of the human eye image based on a preset disparity estimation model to obtain the disparity to be adjusted; determining the target adjustment disparity according to the preset system configuration, the system configuration of the head-mounted display device and the disparity to be adjusted; and adjusting the display parameters of the microdisplay according to the target adjustment disparity. The embodiment of the present application estimates the disparity to be adjusted through a preset disparity estimation model without relying on a depth camera. The target adjustment disparity is determined based on the preset system configuration of the head-mounted display device for training the disparity estimation model, the system configuration of the head-mounted display device actually used and the disparity to be adjusted, and finally the display parameters of the microdisplay are adjusted according to the target adjustment disparity. This can further improve the accuracy of the disparity adjustment when the device for training the disparity estimation model is different from the device actually used, thereby solving the technical problem in the related art that the disparity adjustment of the head-mounted display device needs to rely on a depth camera.

[0051] In one embodiment, step S103 may specifically include adjusting the parallax with the parallax to be adjusted as the target if the system configuration of the head mounted display device is the same as the preset system configuration.

[0052] In actual applications, head-mounted display devices may require system upgrades. Therefore, the system configurations of head-mounted display devices of different series are generally different, and the system configurations of head-mounted display devices produced at different times may also be different. If the system configurations of two devices are different, the depth positions of the images obtained by the two devices under the same parallax setting conditions may be different. If the system configurations of the two devices are the same, the depth positions of the images obtained by the two devices under the same parallax setting conditions are the same. Therefore, the parallax to be adjusted obtained by the parallax estimation model in this embodiment can be used as the target adjustment parallax, so as to adjust the display parameters of the two microdisplays based on the target adjustment parallax.

[0053] In one embodiment, step S103 may specifically include determining the target adjustment parallax corresponding to the parallax to be adjusted based on a preset correspondence between the target adjustment parallax and the parallax to be adjusted if the system configuration of the head mounted display device is different from the preset system configuration.

[0054] It should be noted that the correspondence between the preset target adjustment parallax and the parallax to be adjusted includes the correspondence between the parallax to be adjusted and the first depth, and the correspondence between the first depth and the target adjustment parallax. The correspondence between the parallax to be adjusted and the first depth corresponds to the preset system configuration, and the correspondence between the first depth and the target adjustment parallax corresponds to the system configuration of the head-mounted display device. Specifically, the correspondence between the parallax depths calibrated for head-mounted display devices with different system configurations is generally different. In the actual production process of head-mounted display devices, each head-mounted display device is generally calibrated with a correspondence between the parallax depths. For example, the parallax range of the head-mounted display device is [-30, 100], the depth corresponding to a parallax of -30 is 0.5 meters, the depth corresponding to a parallax of 20 is 0.8 meters, and the depth corresponding to a parallax of 50 is 2 meters. That is, within the parallax range of the head-mounted display device, each parallax corresponds to a depth, that is, the correspondence between the parallax depths is calibrated. In an embodiment of the present application, the head-mounted display device corresponding to the preset system configuration is calibrated with a correspondence between the parallax to be adjusted and the first depth. During actual use, the head-mounted display device corresponding to the system configuration is calibrated with a correspondence between the target adjustment parallax and the first depth.

[0055] When the system configuration of the head-mounted display device is different from the preset system configuration, that is, the system configuration of the head-mounted display device actually used is different from the system configuration of the head-mounted display device for training the disparity estimation model, the disparity depth conversion can be performed. It can be understood that each head-mounted display device has a built-in corresponding relationship between disparity and depth. Therefore, this embodiment can determine the target adjustment disparity corresponding to the disparity to be adjusted based on the preset correspondence between the target adjustment disparity and the disparity to be adjusted. The preset correspondence between the target adjustment disparity and the disparity to be adjusted includes the correspondence between the disparity to be adjusted and the first depth of the head-mounted display device corresponding to the training disparity estimation model, and the correspondence between the first depth and the target adjustment disparity of the head-mounted display device actually used.

[0056] In one embodiment, step S103 may include steps S1031 and S1032.

[0057] Step S1031: If the system configuration of the head mounted display device is different from the preset system configuration, determining the first depth according to the correspondence between the parallax to be adjusted and the first depth.

[0058] Step S1032: Determine the target adjustment disparity corresponding to the disparity to be adjusted according to the correspondence between the first depth and the target adjustment disparity.

[0059] When the system configuration of the head-mounted display device actually used is different from the system configuration of the trained disparity estimation model, the corresponding first depth can be found based on the correspondence between the disparity and the first depth in the head-mounted display device of the trained disparity estimation model to first determine the depth position of the first depth corresponding to the disparity to be adjusted, and then based on the correspondence between the first depth of the head-mounted display device actually used and the target adjustment disparity, the target adjustment disparity corresponding to the depth position of the first depth is determined, thereby determining the target adjustment disparity of the head-mounted display device actually used.

[0060] In one embodiment, step S104 may specifically be adjusting the horizontal coordinates of the human eye image displayed by the microdisplay according to the target adjustment parallax, so as to adjust the display parameters of the microdisplay.

[0061] The content to be displayed refers to the virtual image displayed by the microdisplay. After adjusting the display parameters of the two microdisplays according to the target adjustment parallax, the parallax between the left-eye image and the right-eye image captured by the two microdisplays matches the target adjustment parallax, allowing the virtual content to be displayed at the accurate depth position. Specifically, the lateral coordinates of the human eye image displayed by the microdisplay can be adjusted according to the target adjustment parallax, so that the content to be displayed is displayed at the depth position corresponding to the target adjustment parallax. The target adjustment parallax is determined by the disparity estimation model and the judgment system configuration, without relying on the depth camera, making parallax adjustment more convenient and faster.

[0062] Please refer to Figure 3, which is a schematic block diagram of a disparity estimation model training method provided in an embodiment of the present application. A disparity estimation model training method is applied to a head-mounted display device, and the training method includes steps S201 to S204.

[0063] Step S201: Acquire a plurality of training data, where the training data includes human eye images and annotated disparities corresponding to the human eye images.

[0064] Step S202 : Estimate the disparity of the human eye image based on the disparity estimation model to obtain the current disparity.

[0065] Step S203: adjusting the model parameters of the disparity estimation model according to the deviation between the current disparity and the marked disparity.

[0066] Step S204 : storing the adjusted disparity estimation model. The stored disparity estimation model is used to estimate the disparity of the human eye image to obtain the disparity to be adjusted.

[0067] Model parameters refer to the estimation capability of the disparity estimation model and are crucial to its estimation accuracy. The labeled disparity refers to the actual disparity of the human eye image used to train the disparity estimation model. During neural network-based disparity estimation model training, the model parameters generally need to be continuously adjusted to ensure that the current disparity estimated by the model is consistent with the labeled disparity, that is, the current disparity is consistent with the actual disparity of the training image. Therefore, if the current disparity obtained during training is very close to the labeled disparity corresponding to the training data, the disparity estimation model's estimation accuracy is relatively high. If the current disparity obtained during training differs significantly from the labeled disparity corresponding to the training data, the model parameters of the disparity estimation model need to be adjusted, and training repeated until the current disparity approaches the labeled disparity or the current disparity is identical to the labeled disparity. Upon completion of training, i.e., when the disparity estimation model converges, the adjusted disparity estimation model is stored and used to estimate the disparity of the human eye image to obtain the disparity to be adjusted. It can be understood that the parallax to be adjusted is obtained by obtaining a human eye image when the head mounted display device is in use and estimating the parallax of the human eye image based on a stored parallax estimation model.

[0068] In one embodiment, the human eye image includes a plurality of images at different first depths, and the parallax to be adjusted has a corresponding relationship with the first depth.

[0069] A head-mounted display device for training a disparity estimation model can acquire multiple images at different first depths, each image at the first depth having a corresponding disparity to be adjusted. In an embodiment of the present application, the system configuration corresponding to the head-mounted display device for training the disparity estimation model is a preset system configuration, and the head-mounted display device corresponding to the preset system configuration is calibrated with a correspondence between the disparity to be adjusted and the first depth. Using multiple images at different first depths as training data for the disparity estimation model, the disparity estimation model is continuously trained and optimized so that the current disparity obtained through training is close to the labeled disparity or the current disparity is the same as the labeled disparity.

[0070] The training method for the disparity estimation model provided in the above embodiment includes obtaining multiple training data, the training data including a human eye image and annotated disparity corresponding to the human eye image; estimating the disparity of the human eye image based on the disparity estimation model to obtain the current disparity; adjusting the model parameters of the disparity estimation model according to the deviation between the current disparity and the annotated disparity; and storing the adjusted disparity estimation model, wherein the stored disparity estimation model is used to estimate the disparity of the human eye image to obtain the disparity to be adjusted. The embodiment of the present application continuously trains and optimizes the disparity estimation model so that the current disparity estimated by the disparity estimation model is close to the annotated disparity or the current disparity is the same as the annotated disparity, and then stores the disparity estimation model that has been adjusted and can estimate the disparity to be adjusted, thereby improving the accuracy of the disparity estimation model in estimating the disparity.

[0071] Please refer to Figure 4, which is a schematic block diagram of a parallax adjustment device provided in an embodiment of the present application. The parallax adjustment device can be configured in a server or electronic device to execute the aforementioned parallax adjustment method.

[0072] As shown in FIG4 , the parallax adjustment device is applied to a head-mounted display device, which includes a micro display, and includes: an acquisition module 110 , an estimation module 120 , a determination module 130 and an adjustment module 140 .

[0073] The acquisition module 110 is used to acquire a human eye image.

[0074] The estimation module 120 is configured to estimate the disparity of the human eye image based on a preset disparity estimation model to obtain the disparity to be adjusted.

[0075] The determination module 130 is configured to determine a target adjustment parallax according to a preset system configuration, a system configuration of the head mounted display device, and the parallax to be adjusted.

[0076] The adjustment module 140 is configured to adjust the parallax according to the target and adjust the display parameters of the micro display.

[0077] In one embodiment, the determination module 130 may be configured to adjust the parallax with the parallax to be adjusted as the target if the system configuration of the head mounted display device is the same as the preset system configuration.

[0078] In one embodiment, the determination module 130 may be configured to determine the target adjustment parallax corresponding to the parallax to be adjusted based on a preset correspondence between the target adjustment parallax and the parallax to be adjusted if the system configuration of the head mounted display device is different from the preset system configuration.

[0079] It should be noted that the correspondence between the preset target adjustment parallax and the parallax to be adjusted includes the correspondence between the parallax to be adjusted and the first depth, and the correspondence between the first depth and the target adjustment parallax. The correspondence between the parallax to be adjusted and the first depth corresponds to the preset system configuration, and the correspondence between the first depth and the target adjustment parallax corresponds to the system configuration of the head-mounted display device.

[0080] In one embodiment, the determination module 130 may include a first determination submodule and a second determination submodule.

[0081] The first determining submodule is configured to determine the first depth according to a correspondence between the parallax to be adjusted and the first depth if the system configuration of the head mounted display device is different from the preset system configuration.

[0082] The second determining submodule is configured to determine a target adjustment disparity corresponding to the to-be-adjusted disparity according to a correspondence between the first depth and the target adjustment disparity.

[0083] In one embodiment, the adjustment module 140 is specifically configured to adjust the parallax according to the target, and adjust the horizontal coordinates of the human eye image displayed by the microdisplay, so as to adjust the display parameters of the microdisplay.

[0084] It should be noted that those skilled in the art can clearly understand that, for the convenience and brevity of description, the specific working processes of the above-described devices and modules and units can refer to the corresponding processes in the aforementioned method embodiments and will not be repeated here.

[0085] The method of the present application can be used in many general or special computer system environments or configurations. For example: personal computers, server computers, handheld devices or portable devices, tablet devices, multi-processor systems, microprocessor-based systems, set-top boxes, programmable consumer electronic devices, network PCs, minicomputers, mainframe computers, distributed computing environments including any of the above systems or devices, etc. The present application can be described in the general context of computer-executable instructions executed by a computer, such as program modules. Generally, program modules include routines, programs, objects, components, data structures, etc. that perform specific tasks or implement specific abstract data types. The present application can also be practiced in distributed computing environments, in which tasks are performed by remote processing devices connected through a communication network. In a distributed computing environment, program modules can be located in local and remote computer storage media, including storage devices.

[0086] Illustratively, the above-mentioned method and apparatus may be implemented in the form of a computer program, which may be run on a computer device.

[0087] Please refer to Figure 5, which is a schematic block diagram of the structure of a head-mounted display device provided in an embodiment of the present application. The head-mounted device can be smart glasses such as AR glasses and VR glasses.

[0088] As shown in FIG5 , the computer device includes a processor, a memory, and a network interface connected via a system bus, wherein the memory may include a storage medium and an internal memory.

[0089] The storage medium may store an operating system and a computer program. The computer program includes program instructions, which, when executed, may cause the processor to perform steps of any one of the disparity adjustment methods or the training method of a disparity estimation model.

[0090] The processor is used to provide computing and control capabilities and support the operation of the entire computer equipment.

[0091] The internal memory provides an environment for the operation of the computer program in the storage medium. When the computer program is executed by the processor, the processor can execute the steps of any disparity adjustment method or a disparity estimation model training method.

[0092] This network interface is used for network communication, such as sending assigned tasks.

[0093] Those skilled in the art will understand that the structure shown in FIG5 is merely a block diagram of a portion of the structure related to the solution of the present application, and does not constitute a limitation on the computer device to which the solution of the present application is applied. The specific computer device may include more or fewer components than shown in the figure, or combine certain components, or have a different arrangement of components.

[0094] It should be understood that the processor may be a central processing unit (CPU), other general-purpose processors, digital signal processors (DSP), application-specific integrated circuits (ASIC), field-programmable gate arrays (FPGA), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. The general-purpose processor may be a microprocessor or any conventional processor, etc.

[0095] In one embodiment, the processor is configured to execute a computer program and implement the following steps when executing the computer program:

[0096] Acquire human eye images;

[0097] Based on a preset disparity estimation model, the disparity of the human eye image is estimated to obtain the disparity to be adjusted;

[0098] Determining a target adjustment parallax according to a preset system configuration, a system configuration of the head-mounted display device, and the parallax to be adjusted;

[0099] Adjust the parallax according to the target and adjust the display parameters of the micro display.

[0100] In one embodiment, if the system configuration of the head mounted display device is the same as the preset system configuration, the parallax is adjusted with the parallax to be adjusted as the target.

[0101] In one embodiment, if the system configuration of the head mounted display device is different from the preset system configuration, the target adjustment parallax corresponding to the parallax to be adjusted is determined based on a preset correspondence between the target adjustment parallax and the parallax to be adjusted.

[0102] In one embodiment, the correspondence between the preset target adjustment parallax and the parallax to be adjusted includes the correspondence between the parallax to be adjusted and the first depth, and the correspondence between the first depth and the target adjustment parallax. The correspondence between the parallax to be adjusted and the first depth corresponds to the preset system configuration, and the correspondence between the first depth and the target adjustment parallax corresponds to the system configuration of the head-mounted display device.

[0103] If the system configuration of the head mounted display device is different from the preset system configuration, determining the first depth according to a correspondence between the parallax to be adjusted and the first depth;

[0104] The target adjustment disparity corresponding to the disparity to be adjusted is determined according to the correspondence between the first depth and the target adjustment disparity.

[0105] In one embodiment, adjusting the parallax according to the target and adjusting the display parameters of the microdisplay include:

[0106] The parallax is adjusted according to the target, and the horizontal coordinates of the human eye image displayed by the microdisplay are adjusted to adjust the display parameters of the microdisplay.

[0107] Accordingly, in one embodiment, the processor is configured to execute the computer program and implement the following steps when executing the computer program:

[0108] Acquire multiple training data, the training data including human eye images and annotated disparities corresponding to the human eye images;

[0109] Based on the disparity estimation model, the disparity of the human eye image is estimated to obtain the current disparity;

[0110] According to the deviation between the current disparity and the marked disparity, the model parameters of the disparity estimation model are adjusted; the adjusted disparity estimation model is stored, and the stored disparity estimation model is used to estimate the disparity of the human eye image to obtain the disparity to be adjusted.

[0111] In one embodiment, the human eye image includes a plurality of images at different first depths, and the parallax to be adjusted has a corresponding relationship with the first depth.

[0112] It should be noted that, those skilled in the art can clearly understand that, for the sake of convenience and brevity of description, the specific working process of the above description of disparity adjustment or disparity estimation model training can refer to the corresponding process in the embodiments of the aforementioned disparity adjustment method or disparity estimation model training method, and will not be repeated here.

[0113] An embodiment of the present application also provides a computer-readable storage medium, on which a computer program is stored. The method implemented when the computer program is executed by a processor can refer to the various embodiments of the disparity adjustment method or the disparity estimation model training method of the present application.

[0114] The computer-readable storage medium may be an internal storage unit of the computer device described in the aforementioned embodiment, such as a hard disk or memory of the computer device. The computer-readable storage medium may also be an external storage device of the computer device, such as a plug-in hard disk, a smart media card (SMC), a secure digital (SD) card, a flash card, etc., equipped on the computer device.

[0115] It should be understood that the terms used in this specification are only for the purpose of describing specific embodiments and are not intended to limit the present application. As used in this specification and the appended claims, the singular forms "a", "an", and "the" are intended to include the plural forms unless the context clearly indicates otherwise.

[0116] It should also be understood that the term "and / or" used in this specification and the appended claims refers to any combination of one or more of the associated listed items and all possible combinations, including these combinations. It should be noted that, in this article, the terms "include", "comprise" or any other variations thereof are intended to cover non-exclusive inclusion, so that a process, method, article or system that includes a series of elements includes not only those elements, but also other elements that are not explicitly listed, or also includes elements that are inherent to such process, method, article or system. In the absence of further restrictions, an element defined by the sentence "including a..." does not exclude the presence of other identical elements in the process, method, article or system that includes the element.

[0117] The serial numbers of the embodiments of the present application are for description only and do not represent the advantages or disadvantages of the embodiments. The above description is only a specific embodiment of the present application, but the scope of protection of the present application is not limited thereto. Any technician familiar with the technical field can easily think of various equivalent modifications or replacements within the technical scope disclosed in this application, and these modifications or replacements should be included in the scope of protection of this application. Therefore, the scope of protection of this application should be based on the scope of protection of the claims.

Claims

1. A parallax adjustment method, applied to a head-mounted display device, wherein the head-mounted display device includes a micro display, comprising: Acquire human eye images; Based on a preset disparity estimation model, the disparity of the human eye image is estimated to obtain the disparity to be adjusted; Determining a target adjustment parallax according to a preset system configuration, a system configuration of the head mounted display device, and the parallax to be adjusted; The parallax is adjusted according to the target, and the display parameters of the micro display are adjusted.

2. The parallax adjustment method according to claim 1, wherein: The step of determining a target adjustment parallax according to a preset system configuration, a system configuration of the head mounted display device, and the parallax to be adjusted includes: If the system configuration of the head mounted display device is the same as the preset system configuration, the parallax is adjusted with the parallax to be adjusted as a target.

3. The parallax adjustment method according to claim 1, wherein: The step of determining a target adjustment parallax according to a preset system configuration, a system configuration of the head mounted display device, and the parallax to be adjusted includes: If the system configuration of the head mounted display device is different from the preset system configuration, the target adjustment disparity corresponding to the disparity to be adjusted is determined based on a preset correspondence between the target adjustment disparity and the disparity to be adjusted.

4. The parallax adjustment method according to claim 3, wherein: The correspondence between the preset target adjustment parallax and the to-be-adjusted parallax includes a correspondence between the to-be-adjusted parallax and a first depth, and a correspondence between the first depth and the target adjustment parallax, the correspondence between the to-be-adjusted parallax and the first depth corresponds to the preset system configuration, and the correspondence between the first depth and the target adjustment parallax corresponds to the system configuration of the head mounted display device; If the system configuration of the head mounted display device is different from the preset system configuration, determining the target adjustment parallax corresponding to the parallax to be adjusted based on the preset correspondence between the target adjustment parallax and the parallax to be adjusted includes: If the system configuration of the head mounted display device is different from the preset system configuration, determining the first depth according to the correspondence between the parallax to be adjusted and the first depth; The target adjustment disparity corresponding to the to-be-adjusted disparity is determined according to the correspondence between the first depth and the target adjustment disparity.

5. The parallax adjustment method according to any one of claims 1 to 4, wherein: The adjusting the parallax according to the target and adjusting the display parameters of the microdisplay includes: The parallax is adjusted according to the target, and the lateral coordinates of the human eye image displayed by the microdisplay are adjusted to adjust the display parameters of the microdisplay.

6. A method for training a disparity estimation model, applied to a head mounted display device, the training method comprising: Acquire a plurality of training data, wherein the training data includes human eye images and annotated disparities corresponding to the human eye images; Based on a disparity estimation model, the disparity of the human eye image is estimated to obtain a current disparity; adjusting a model parameter of the disparity estimation model according to a deviation between the current disparity and the marked disparity; The adjusted disparity estimation model is stored, and the stored disparity estimation model is used to estimate the disparity of the human eye image to obtain the disparity to be adjusted.

7. The method for training a disparity estimation model according to claim 6, wherein: The human eye image includes a plurality of images with different first depths, and the parallax to be adjusted has a corresponding relationship with the first depth.

8. A parallax adjustment device, applied to a head-mounted display device, wherein the head-mounted display device includes a micro display, comprising: An acquisition module, used for acquiring human eye images; An estimation module, used for estimating the disparity of the human eye image based on a preset disparity estimation model to obtain a disparity to be adjusted; A determination module, configured to determine a target adjustment parallax according to a preset system configuration, a system configuration of the head mounted display device, and the parallax to be adjusted; The adjustment module is used to adjust the parallax according to the target and adjust the display parameters of the micro display.

9. The parallax adjustment device according to claim 8, wherein: The determining module is used for adjusting the parallax with the parallax to be adjusted as a target if the system configuration of the head mounted display device is the same as the preset system configuration.

10. The parallax adjustment device according to claim 8, wherein: The determination module is used to determine the target adjustment disparity corresponding to the to-be-adjusted disparity based on a preset correspondence relationship between the target adjustment disparity and the to-be-adjusted disparity if the system configuration of the head mounted display device is different from the preset system configuration.

11. The parallax adjustment device according to claim 10, wherein: The determination module includes a first determination submodule and a second determination submodule; The first determining submodule is configured to determine the first depth according to a correspondence between the parallax to be adjusted and the first depth if the system configuration of the head mounted display device is different from the preset system configuration; The second determining submodule is configured to determine a target adjustment disparity corresponding to the to-be-adjusted disparity according to a correspondence between the first depth and the target adjustment disparity.

12. The parallax adjustment device according to any one of claims 8 to 11, wherein: The adjustment module is used to adjust the parallax according to the target, and adjust the horizontal coordinates of the human eye image displayed by the microdisplay, so as to adjust the display parameters of the microdisplay.

13. A head mounted display device, the head mounted display device comprising a memory and a processor; The memory is used to store computer programs; The processor is used to execute the computer program and implement the disparity adjustment method as described in any one of claims 1 to 5 or the disparity estimation model training method as described in any one of claims 6 to 7 when executing the computer program.

14. A computer-readable storage medium having a computer program stored thereon, wherein when the computer program is executed by a processor, the computer program implements the steps of the disparity adjustment method described in any one of claims 1 to 5 or the steps of the disparity estimation model training method described in any one of claims 6 to 7.

Images