Parallax adjustment method and device, model training method and device, equipment and storage medium

Abstract

The invention relates to the technical field of head-mounted display equipment, and provides a parallax adjustment method and device, a model training method and device, equipment and a storage medium, and the method comprises the steps: obtaining a human eye image; estimating the parallax of the human eye image based on a preset parallax estimation model to obtain a to-be-adjusted parallax; determining a target adjustment parallax according to a preset system configuration, a system configuration of the head-mounted display device and the to-be-adjusted parallax; and adjusting display parameters of the micro-display according to the target adjustment parallax. According to the method, the to-be-adjusted parallax is estimated through the preset parallax estimation model, and the target adjustment parallax is determined according to the preset system configuration, the actually used system configuration and the to-be-adjusted parallax in order to improve the parallax adjustment accuracy by considering that the equipment for training the parallax estimation model is possibly different from the actually used equipment; the technical problem that parallax adjustment of the head-mounted display device needs to depend on a depth camera is solved.

Description

Technical Field

[0001] The present application relates to the technical field of head-mounted display devices, and in particular to a parallax adjustment method, a model training method, an apparatus, a device and a storage medium. Background Art

[0002] The optical display system of a head-mounted display device is usually composed of a microdisplay and optical elements. The optical elements are used to project the image displayed on the microdisplay into the left and right eyes of a person to form a virtual image of a fixed depth. Since the depth of the virtual image is fixed, when a person's eyes look at a real object, if the depth of the object is different from the depth of the virtual image, the eye muscles need to be adjusted to focus the gaze point on the virtual image. Switching the gaze point for a long time will cause eye fatigue.

[0003] Related technologies usually use a depth camera to obtain the depth of the real object that the human eye is looking at, and then adjust the parallax of the virtual image to the parallax corresponding to the depth, so that the depth of the virtual content is consistent with the depth of the real object. However, related technologies need to rely on the depth camera to adjust the parallax, which is not conducive to the parallax adjustment of the head-mounted display device. Summary of the invention

[0004] The main purpose of this application is to provide a parallax adjustment method, a model training method, an apparatus, a device and a storage medium, which are used to solve the technical problem in the related art that the parallax adjustment of the head-mounted display device needs to rely on a depth camera.

[0005] In a first aspect, the present application provides a parallax adjustment method, which is applied to a head-mounted display device, wherein the head-mounted display device includes a micro display, including:

[0006] Acquire human eye images;

[0007] Based on a preset disparity estimation model, the disparity of the human eye image is estimated to obtain the disparity to be adjusted;

[0008] Determining a target adjustment parallax according to a preset system configuration, a system configuration of the head mounted display device, and the parallax to be adjusted;

[0009] The parallax is adjusted according to the target to adjust the display parameters of the micro display.

[0010] In a second aspect, the present application provides a method for training a disparity estimation model, which is applied to a head-mounted display device, and the training method includes:

[0011] Acquire a plurality of training data, wherein the training data includes human eye images and annotated disparities corresponding to the human eye images;

[0012] Based on a disparity estimation model, the disparity of the human eye image is estimated to obtain a current disparity;

[0013] adjusting a model parameter of the disparity estimation model according to a deviation between the current disparity and the marked disparity;

[0014] The adjusted disparity estimation model is stored, and the stored disparity estimation model is used to estimate the disparity of the human eye image to obtain the disparity to be adjusted.

[0015] In a third aspect, the present application further provides a parallax adjustment device, which is applied to a head-mounted display device, wherein the head-mounted display device includes a micro display, including:

[0016] An acquisition module, used for acquiring human eye images;

[0017] An estimation module, used for estimating the disparity of the human eye image based on a preset disparity estimation model to obtain a disparity to be adjusted;

[0018] A determination module, configured to determine a target adjustment parallax according to a preset system configuration, a system configuration of the head mounted display device, and the parallax to be adjusted;

[0019] The adjustment module is used to adjust the parallax according to the target and adjust the display parameters of the micro display.

[0020] In a fourth aspect, the present application further provides a head-mounted display device, the head-mounted display device comprising a memory and a processor;

[0021] The memory is used to store computer programs;

[0022] The processor is used to execute the computer program and implement the above-mentioned disparity adjustment method or disparity estimation model training method when executing the computer program.

[0023] In a fifth aspect, the present application also provides a computer-readable storage medium, on which a computer program is stored. When the computer program is executed by a processor, the steps of the above-mentioned disparity adjustment method or disparity estimation model training method are implemented.

[0024] The present application provides a parallax adjustment method, a model training method, an apparatus, a device and a storage medium, the method comprising: obtaining a human eye image; estimating the parallax of the human eye image based on a preset parallax estimation model to obtain the parallax to be adjusted; determining the target adjustment parallax according to the preset system configuration, the system configuration of the head-mounted display device and the parallax to be adjusted; adjusting the display parameters of the microdisplay according to the target adjustment parallax. The present application estimates the parallax to be adjusted through a preset parallax estimation model without relying on a depth camera. And the target adjustment parallax is determined according to the preset system configuration of the head-mounted display device for training the parallax estimation model, the system configuration of the head-mounted display device actually used and the parallax to be adjusted, and finally the display parameters of the microdisplay are adjusted according to the target adjustment parallax, which can further improve the accuracy of the parallax adjustment when the device for training the parallax estimation model is different from the device actually used, thereby solving the technical problem in the related art that the parallax adjustment of the head-mounted display device needs to rely on the depth camera. BRIEF DESCRIPTION OF THE DRAWINGS

[0025] In order to more clearly illustrate the technical solutions of the embodiments of the present application, the drawings required for use in the description of the embodiments will be briefly introduced below. Obviously, the drawings described below are some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.

[0026] Figure 1 is a flow chart of a parallax adjustment method provided in an embodiment of the present application;

[0027] Figure 2 is a schematic diagram of the connection between the server and the terminal provided in the embodiment of the present application;

[0028] Figure 3 It is a flowchart of a method for training a disparity estimation model provided in an embodiment of the present application;

[0029] Figure 4 is a schematic block diagram of a parallax adjustment device provided in an embodiment of the present application;

[0030] Figure 5 It is a schematic block diagram of the structure of a head-mounted display device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0031] The following will be combined with the drawings in the embodiments of the present application to clearly and completely describe the technical solutions in the embodiments of the present application. Obviously, the described embodiments are part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of this application.

[0032] The flowcharts shown in the accompanying drawings are only examples and do not necessarily include all the contents and operations / steps, nor must they be executed in the order described. For example, some operations / steps may also be decomposed, combined or partially merged, so the actual execution order may change according to actual conditions.

[0033] The parallax adjustment method, model training method, device, equipment and storage medium provided in the embodiments of the present application. Among them, a parallax adjustment method is applied to a head-mounted display device, which includes an optical element and a microdisplay, and uses the optical element to project the image displayed by the microdisplay into the left and right eyes of a person to form a virtual image. It should be noted that the head-mounted display device can be AR glasses or VR glasses. In addition, the parallax adjustment method can also be applied to a server, which can be a separate server or a cloud server that provides basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communications, middleware services, domain name services, security services, content delivery networks (Content Delivery Network, CDN), and big data and artificial intelligence platforms.

[0034] In conjunction with the accompanying drawings, some embodiments of the present application are described in detail below. In the absence of conflict, the following embodiments and features in the embodiments can be combined with each other.

[0035] See also Figure 1 , Figure 1 It is a flowchart of a parallax adjustment method provided in an embodiment of the present application.

[0036] It should be noted that the parallax adjustment method provided in the embodiment of the present application can be used in a head-mounted display device or in a server.

[0037] like Figure 2 As shown, the parallax adjustment method is applied to a server, the server and the terminal are connected in communication, and the display parameters obtained according to the parallax adjustment method can be sent to the terminal electronic device through the server. Of course, it is not limited to this and is not limited here.

[0038] In a specific implementation, the terminal includes a head-mounted display device, which may be smart glasses such as AR glasses and VR glasses; the server may be a separate server or a server cluster, or a cloud server that provides cloud computing services.

[0039] like Figure 1 As shown, the parallax adjustment method includes steps S101 to S104.

[0040] Step S101, obtaining a human eye image.

[0041] The camera on the head mounted display device can be used to capture the user's left eye and right eye to obtain a human eye image including a left eye image and a right eye image. The camera can be an RGB camera, a monochrome camera, or a near infrared camera. If a near infrared camera is used to capture the human eye, an infrared LED light can be used to illuminate the human eye to ensure that the image of the user's eye on the near infrared camera is clearly visible.

[0042] Step S102: based on a preset disparity estimation model, estimate the disparity of the human eye image to obtain the disparity to be adjusted.

[0043] Parallax refers to the difference in direction when observing the same target from two points at a certain distance. In other words, the difference in the frame of view captured by the viewing window and the lens in the camera. Because there is a distance between the left eye and the right eye of a person, there is a slight difference in the perspective of the left eye and the right eye, and such a difference will cause the scene observed by the left eye and the right eye to have a little displacement, that is, the distance and direction of the left eye and the right eye of a person are different from the actual object. Therefore, there will be a certain difference between the left eye image and the right eye image obtained after the actual object is projected on the human eye. In practical applications, the difference can be reduced by performing parallax adjustment. The relevant technology generally needs to rely on a depth camera, that is, the depth of the real object that the human eye looks at is obtained by the depth camera, and then the parallax of the virtual image is adjusted to the parallax corresponding to the depth. However, this embodiment does not require the use of a depth camera, but estimates the parallax of the left eye image and the right eye image through a parallax estimation model. Specifically, the parallax estimation model is pre-trained and optimized. The embodiment of the present application inputs the obtained left eye image and right eye image into a preset parallax estimation model to obtain a parallax to be adjusted with higher accuracy.

[0044] Step S103: determining a target adjustment parallax according to a preset system configuration, a system configuration of the head mounted display device, and the parallax to be adjusted.

[0045] Normally, different head-mounted display devices have different assembly conditions of their internal structural components. For example, different devices may have different positions and angles of observing objects under the same parallax standard. System configuration refers to the assembly tolerances of various components in a head-mounted display device, such as the resolution of a microdisplay, the placement and angle of a microdisplay, etc. The system configurations of head-mounted display devices produced under different production conditions are generally different. Based on this, when the system configurations of two devices are different and the setting conditions of the parallax are the same, the depth positions of the images acquired by the two devices under the parallax may be different. For example, the depth positions of the images acquired by head-mounted display device A based on its own system configuration and the images acquired by head-mounted display device B based on its own system configuration are different, resulting in differences in actual parallax. Based on this, taking into account the situation that the head-mounted display device for training the disparity estimation model and the head-mounted display device actually used may have different system configurations, after estimating the disparity to be adjusted through the disparity estimation model, this embodiment can also first determine whether the system configuration of the head-mounted display device for training the disparity estimation model and the system configuration of the head-mounted display device actually used are the same, and then determine the target adjustment parallax in combination with the parallax to be adjusted.

[0046] Step S104: adjusting the parallax according to the target and adjusting the display parameters of the micro display.

[0047] Generally, the head mounted display device may be provided with two micro displays, and the two micro displays correspond to the left eye and the right eye of the user, respectively. Therefore, after the target adjustment parallax is determined, the display parameters of the two micro displays may be adjusted according to the target adjustment parallax, that is, the horizontal coordinates of the images displayed by the two micro displays may be adjusted so that after the horizontal coordinates of the two micro displays are moved and adjusted, the parallax between the captured left eye image and the right eye image matches the target adjustment parallax.

[0048] The parallax adjustment method provided in the above embodiment includes acquiring a human eye image; estimating the parallax of the human eye image based on a preset parallax estimation model to obtain the parallax to be adjusted; determining the target adjustment parallax according to the preset system configuration, the system configuration of the head-mounted display device and the parallax to be adjusted; and adjusting the display parameters of the microdisplay according to the target adjustment parallax. The embodiment of the present application estimates the parallax to be adjusted through a preset parallax estimation model without relying on a depth camera. And the target adjustment parallax is determined according to the preset system configuration of the head-mounted display device for training the parallax estimation model, the system configuration of the head-mounted display device actually used, and the parallax to be adjusted, and finally the display parameters of the microdisplay are adjusted according to the target adjustment parallax, which can further improve the accuracy of the parallax adjustment when the device for training the parallax estimation model is different from the device actually used, thereby solving the technical problem in the related art that the parallax adjustment of the head-mounted display device needs to rely on a depth camera.

[0049] In one embodiment, step S103 may specifically include adjusting the parallax with the parallax to be adjusted as the target if the system configuration of the head mounted display device is the same as the preset system configuration.

[0050] In actual applications, head-mounted display devices may require system upgrades. Therefore, the system configurations of head-mounted display devices of different series are generally different, and the system configurations of head-mounted display devices produced at different times may also be different. If the system configurations of two devices are different, and the setting conditions of the parallax are the same, the depth positions of the images acquired by the two devices under the parallax may be different. If the system configurations of the two devices are the same, and the setting conditions of the parallax are the same, the depth positions of the images acquired by the two devices under the parallax are the same. Therefore, the parallax to be adjusted obtained by the parallax estimation model in this embodiment can be used as the target adjustment parallax, so as to adjust the display parameters of the two microdisplays based on the target adjustment parallax.

[0051] In one embodiment, step S103 may specifically include determining the target adjustment parallax corresponding to the parallax to be adjusted based on a preset correspondence between the target adjustment parallax and the parallax to be adjusted if the system configuration of the head mounted display device is different from the preset system configuration.

[0052] It should be noted that the correspondence between the preset target adjustment parallax and the parallax to be adjusted includes the correspondence between the parallax to be adjusted and the first depth, and the correspondence between the first depth and the target adjustment parallax. The correspondence between the parallax to be adjusted and the first depth corresponds to the preset system configuration, and the correspondence between the first depth and the target adjustment parallax corresponds to the system configuration of the head-mounted display device. Specifically, the correspondence between the parallax depths calibrated by head-mounted display devices with different system configurations is generally different. In the actual production process of head-mounted display devices, each head-mounted display device is generally calibrated with a correspondence between the parallax depths. For example, the parallax range of the head-mounted display device is [-30, 100], the depth corresponding to the parallax of -30 is 0.5 meters, the depth corresponding to the parallax of 20 is 0.8 meters, and the depth corresponding to the parallax of 50 is 2 meters, that is, within the parallax range of the head-mounted display device, each parallax corresponds to a depth, that is, the correspondence between the calibrated parallax depths. In an embodiment of the present application, a head-mounted display device corresponding to a preset system configuration is calibrated with a correspondence between the parallax to be adjusted and the first depth. During actual use, a head-mounted display device corresponding to the system configuration is calibrated with a correspondence between the target adjustment parallax and the first depth.

[0053] When the system configuration of the head-mounted display device is different from the preset system configuration, that is, the system configuration of the head-mounted display device actually used is different from the system configuration of the head-mounted display device for training the disparity estimation model, the disparity depth conversion can be performed. It can be understood that each head-mounted display device has a corresponding built-in correspondence between the disparity and the depth. Therefore, the present embodiment can determine the target adjustment disparity corresponding to the disparity to be adjusted according to the preset correspondence between the target adjustment disparity and the disparity to be adjusted. The preset correspondence between the target adjustment disparity and the disparity to be adjusted includes the correspondence between the disparity to be adjusted and the first depth of the head-mounted display device corresponding to the training disparity estimation model, and the correspondence between the first depth and the target adjustment disparity of the head-mounted display device actually used.

[0054] In one embodiment, step S103 may include step S1031 to step S1032.

[0055] Step S1031: if the system configuration of the head mounted display device is different from the preset system configuration, determining the first depth according to the correspondence between the parallax to be adjusted and the first depth.

[0056] Step S1032: Determine the target adjustment disparity corresponding to the to-be-adjusted disparity according to the correspondence between the first depth and the target adjustment disparity.

[0057] When the system configuration of the head-mounted display device actually used is different from the system configuration of the trained disparity estimation model, the corresponding first depth can be found according to the correspondence between the disparity and the first depth in the head-mounted display device of the trained disparity estimation model, so as to first determine the depth position of the first depth corresponding to the disparity to be adjusted, and then determine the target adjustment disparity corresponding to the depth position of the first depth based on the correspondence between the first depth of the head-mounted display device actually used and the target adjustment disparity, thereby determining the target adjustment disparity of the head-mounted display device actually used.

[0058] In one embodiment, step S104 may specifically be to adjust the horizontal coordinates of the human eye image displayed by the microdisplay according to the target adjustment parallax, so as to adjust the display parameters of the microdisplay.

[0059] The content to be displayed refers to the virtual image displayed by the microdisplay. After adjusting the display parameters of the two microdisplays according to the target adjustment parallax, the parallax between the left-eye image and the right-eye image captured by the two microdisplays matches the target adjustment parallax, so that the virtual content can be displayed at an accurate depth position. Specifically, the lateral coordinates of the human eye image displayed by the microdisplay can be adjusted according to the target adjustment parallax, so that the content to be displayed is displayed at the depth position corresponding to the target adjustment parallax. The target adjustment parallax is obtained by the parallax estimation model and the judgment system configuration, without relying on the depth camera, making the parallax adjustment more convenient and faster.

[0060] See also Figure 3 , Figure 3 1 is a schematic block diagram of a method for training a disparity estimation model provided in an embodiment of the present application. A method for training a disparity estimation model is applied to a head mounted display device, and the training method includes steps S201 to S204.

[0061] Step S201: Acquire a plurality of training data, where the training data includes human eye images and annotated disparities corresponding to the human eye images.

[0062] Step S202: Estimate the disparity of the human eye image based on the disparity estimation model to obtain the current disparity.

[0063] Step S203: adjusting the model parameters of the disparity estimation model according to the deviation between the current disparity and the marked disparity.

[0064] Step S204 , storing the adjusted disparity estimation model, where the stored disparity estimation model is used to estimate the disparity of the human eye image to obtain the disparity to be adjusted.

[0065] Model parameters refer to the estimation ability of the disparity estimation model, and the model parameters of the disparity estimation model are very important for the accuracy of its estimation. The labeled disparity refers to the actual disparity of the human eye image used to train the disparity estimation model. In the process of training the disparity estimation model based on the neural learning network, it is generally necessary to continuously adjust the model parameters so that the current disparity estimated by the disparity estimation model is the same as the labeled disparity, that is, the current disparity is the same as the actual disparity of the human eye image used for training. Therefore, if the current disparity obtained by training is very close to the labeled disparity corresponding to the training data, it means that the estimation accuracy of the disparity estimation model is relatively high; if the current disparity obtained by training is significantly different from the labeled disparity corresponding to the training data, it is necessary to adjust the model parameters of the disparity estimation model, and then repeat the training until the current disparity is close to the labeled disparity or the current disparity is the same as the labeled disparity. When the training is completed, that is, when the disparity estimation model converges, the adjusted disparity estimation model is stored, and the stored disparity estimation model is used to estimate the disparity of the human eye image to obtain the disparity to be adjusted. It can be understood that the parallax to be adjusted is obtained by obtaining a human eye image when the head mounted display device is in use, and estimating the parallax of the human eye image based on a stored parallax estimation model.

[0066] In one embodiment, the human eye image includes a plurality of images with different first depths, and the parallax to be adjusted has a corresponding relationship with the first depth.

[0067] The head-mounted display device for training the disparity estimation model can obtain multiple images with different first depths, and each image with the first depth has a corresponding disparity to be adjusted. In an embodiment of the present application, the system configuration corresponding to the head-mounted display device for training the disparity estimation model is a preset system configuration, and the head-mounted display device corresponding to the preset system configuration is calibrated with a correspondence between the disparity to be adjusted and the first depth. Using multiple images with different first depths as training data for the disparity estimation model, the disparity estimation model is continuously trained and optimized so that the current disparity obtained through training is close to the annotated disparity or the current disparity is the same as the annotated disparity.

[0068] The training method of the disparity estimation model provided in the above embodiment includes obtaining a plurality of training data, the training data including a human eye image and an annotated disparity corresponding to the human eye image; estimating the disparity of the human eye image based on the disparity estimation model to obtain the current disparity; adjusting the model parameters of the disparity estimation model according to the deviation between the current disparity and the annotated disparity; storing the adjusted disparity estimation model, the stored disparity estimation model is used to estimate the disparity of the human eye image to obtain the disparity to be adjusted. . The embodiment of the present application continuously trains and optimizes the disparity estimation model, so that the current disparity estimated by the disparity estimation model is close to the annotated disparity or the current disparity is the same as the annotated disparity, and then stores the disparity estimation model that has been adjusted and can estimate the disparity to be adjusted, thereby improving the accuracy of the disparity estimation model in estimating the disparity.

[0069] See also Figure 4 , Figure 4 1 is a schematic block diagram of a parallax adjustment device provided in an embodiment of the present application. The parallax adjustment device can be configured in a server or an electronic device to execute the above-mentioned parallax adjustment method.

[0070] like Figure 4 As shown, the parallax adjustment device is applied to a head-mounted display device, and the head-mounted display device includes a micro display, including: an acquisition module 110, an estimation module 120, a determination module 130 and an adjustment module 140.

[0071] The acquisition module 110 is used to acquire a human eye image.

[0072] The estimation module 120 is used to estimate the disparity of the human eye image based on a preset disparity estimation model to obtain the disparity to be adjusted.

[0073] The determination module 130 is used to determine the target adjustment parallax according to the preset system configuration, the system configuration of the head mounted display device and the parallax to be adjusted.

[0074] The adjustment module 140 is used to adjust the parallax according to the target and adjust the display parameters of the micro display.

[0075] In one embodiment, the determination module 130 may be configured to adjust the parallax with the parallax to be adjusted as the target if the system configuration of the head mounted display device is the same as the preset system configuration.

[0076] In one embodiment, the determination module 130 may be configured to determine the target adjustment disparity corresponding to the disparity to be adjusted based on the correspondence between the preset target adjustment disparity and the disparity to be adjusted if the system configuration of the head mounted display device is different from the preset system configuration.

[0077] It should be noted that the correspondence between the preset target adjustment parallax and the parallax to be adjusted includes the correspondence between the parallax to be adjusted and the first depth, and the correspondence between the first depth and the target adjustment parallax. The correspondence between the parallax to be adjusted and the first depth corresponds to the preset system configuration, and the correspondence between the first depth and the target adjustment parallax corresponds to the system configuration of the head-mounted display device.

[0078] In one embodiment, the determination module 130 may include a first determination submodule and a second determination submodule.

[0079] The first determination submodule is configured to determine the first depth according to a correspondence between the parallax to be adjusted and the first depth if the system configuration of the head mounted display device is different from the preset system configuration.

[0080] The second determining submodule is configured to determine a target adjustment disparity corresponding to the to-be-adjusted disparity according to a correspondence between the first depth and the target adjustment disparity.

[0081] In one embodiment, the adjustment module 140 is specifically used to adjust the parallax according to the target, and adjust the horizontal coordinates of the human eye image displayed by the microdisplay, so as to adjust the display parameters of the microdisplay.

[0082] It should be noted that technicians in the relevant field can clearly understand that for the convenience and simplicity of description, the specific working processes of the above-described devices and modules and units can refer to the corresponding processes in the aforementioned method embodiments and will not be repeated here.

[0083] The method of the present application can be used in many general or special computer system environments or configurations. For example: personal computers, server computers, handheld devices or portable devices, tablet devices, multiprocessor systems, microprocessor-based systems, set-top boxes, programmable consumer electronic devices, network PCs, minicomputers, mainframe computers, distributed computing environments including any of the above systems or devices, etc. The present application can be described in the general context of computer-executable instructions executed by a computer, such as program modules. Generally, program modules include routines, programs, objects, components, data structures, etc. that perform specific tasks or implement specific abstract data types. The present application can also be practiced in distributed computing environments, in which tasks are performed by remote processing devices connected through a communication network. In a distributed computing environment, program modules can be located in local and remote computer storage media including storage devices.

[0084] Exemplarily, the above method and apparatus may be implemented in the form of a computer program, which may be run on a computer device.

[0085] See also Figure 5 , Figure 5 1 is a schematic block diagram of a head-mounted display device provided in an embodiment of the present application. The head-mounted device may be smart glasses such as AR glasses and VR glasses.

[0086] like Figure 5 As shown, the computer device includes a processor, a memory, and a network interface connected via a system bus, wherein the memory may include a storage medium and an internal memory.

[0087] The storage medium can store an operating system and a computer program. The computer program includes program instructions, and when the program instructions are executed, the processor can execute the steps of any disparity adjustment method or a disparity estimation model training method.

[0088] The processor is used to provide computing and control capabilities and support the operation of the entire computer equipment.

[0089] The internal memory provides an environment for the operation of the computer program in the storage medium. When the computer program is executed by the processor, the processor can execute the steps of any disparity adjustment method or a disparity estimation model training method.

[0090] The network interface is used for network communication, such as sending assigned tasks.

[0091] Those skilled in the art will understand that Figure 5The structure shown in the figure is only a block diagram of a part of the structure related to the solution of the present application, and does not constitute a limitation on the computer device to which the solution of the present application is applied. The specific computer device may include more or fewer components than those shown in the figure, or combine certain components, or have a different arrangement of components.

[0092] It should be understood that the processor may be a central processing unit (CPU), and the processor may also be other general-purpose processors, digital signal processors (DSP), application-specific integrated circuits (ASIC), field-programmable gate arrays (FPGA) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. Among them, the general-purpose processor may be a microprocessor or the processor may also be any conventional processor, etc.

[0093] In one embodiment, the processor is used to execute a computer program and implement the following steps when executing the computer program:

[0094] Acquire human eye images;

[0095] Based on a preset disparity estimation model, the disparity of the human eye image is estimated to obtain the disparity to be adjusted;

[0096] Determine a target adjustment parallax according to a preset system configuration, a system configuration of the head mounted display device, and the parallax to be adjusted;

[0097] The parallax is adjusted according to the target, and the display parameters of the microdisplay are adjusted.

[0098] In one embodiment, if the system configuration of the head mounted display device is the same as the preset system configuration, the parallax is adjusted with the parallax to be adjusted as the target.

[0099] In one embodiment, if the system configuration of the head mounted display device is different from the preset system configuration, the target adjustment disparity corresponding to the disparity to be adjusted is determined based on the preset correspondence between the target adjustment disparity and the disparity to be adjusted.

[0100] In one embodiment, the correspondence between the preset target adjustment parallax and the parallax to be adjusted includes the correspondence between the parallax to be adjusted and the first depth, and the correspondence between the first depth and the target adjustment parallax. The correspondence between the parallax to be adjusted and the first depth corresponds to the preset system configuration, and the correspondence between the first depth and the target adjustment parallax corresponds to the system configuration of the head-mounted display device.

[0101] If the system configuration of the head mounted display device is different from the preset system configuration, determining the first depth according to a correspondence between the parallax to be adjusted and the first depth;

[0102] According to the correspondence between the first depth and the target adjustment disparity, the target adjustment disparity corresponding to the to-be-adjusted disparity is determined.

[0103] In one embodiment, adjusting the parallax according to the target and adjusting the display parameters of the microdisplay include:

[0104] The parallax is adjusted according to the target, and the lateral coordinates of the human eye image displayed by the microdisplay are adjusted to adjust the display parameters of the microdisplay.

[0105] Correspondingly, in one embodiment, the processor is used to execute the computer program and implement the following steps when executing the computer program:

[0106] Acquire multiple training data, the training data including human eye images and annotated disparities corresponding to the human eye images;

[0107] Based on the disparity estimation model, the disparity of the human eye image is estimated to obtain the current disparity;

[0108] According to the deviation between the current disparity and the marked disparity, the model parameters of the disparity estimation model are adjusted; the adjusted disparity estimation model is stored, and the stored disparity estimation model is used to estimate the disparity of the human eye image to obtain the disparity to be adjusted.

[0109] In one embodiment, the human eye image includes a plurality of images with different first depths, and the parallax to be adjusted has a corresponding relationship with the first depth.

[0110] It should be noted that technical personnel in the relevant field can clearly understand that, for the convenience and conciseness of description, the specific working process of the above-mentioned disparity adjustment or disparity estimation model training can refer to the corresponding process in the embodiment of the aforementioned disparity adjustment method or the disparity estimation model training method, and will not be repeated here.

[0111] An embodiment of the present application also provides a computer-readable storage medium, on which a computer program is stored. The method implemented when the computer program is executed by a processor can refer to the various embodiments of the disparity adjustment method or the disparity estimation model training method of the present application.

[0112] The computer-readable storage medium may be an internal storage unit of the computer device described in the foregoing embodiment, such as a hard disk or memory of the computer device. The computer-readable storage medium may also be an external storage device of the computer device, such as a plug-in hard disk, a smart memory card (SmartMedia Card, SMC), a secure digital (Secure Digital, SD) card, a flash card (Flash Card), etc., equipped on the computer device.

[0113] It should be understood that the terms used in this application specification are only for the purpose of describing specific embodiments and are not intended to limit the application. As used in this application specification and the appended claims, unless the context clearly indicates otherwise, the singular forms "a", "an" and "the" are intended to include plural forms.

[0114] It should also be understood that the term "and / or" used in the specification of this application and the appended claims refers to any combination of one or more of the associated listed items and all possible combinations, including these combinations. It should be noted that, in this article, the terms "include", "comprise" or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, method, article or system including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or system. In the absence of further restrictions, an element defined by the sentence "including a..." does not exclude the presence of other identical elements in the process, method, article or system including the element.

[0115] The serial numbers of the embodiments of the present application are for description only and do not represent the advantages and disadvantages of the embodiments. The above description is only a specific implementation mode of the present application, but the protection scope of the present application is not limited thereto. Any technician familiar with the technical field can easily think of various equivalent modifications or replacements within the technical scope disclosed in the present application, and these modifications or replacements should be included in the protection scope of the present application. Therefore, the protection scope of the present application shall be based on the protection scope of the claims.

Claims

1. A parallax adjustment method, applied to a head-mounted display device, wherein the head-mounted display device includes a micro display, characterized in that: include: Acquire human eye images; Based on a preset disparity estimation model, the disparity of the human eye image is estimated to obtain the disparity to be adjusted; Determining a target adjustment parallax according to a preset system configuration, a system configuration of the head mounted display device, and the parallax to be adjusted; The parallax is adjusted according to the target to adjust the display parameters of the micro display.

2. The parallax adjustment method according to claim 1, characterized in that: The step of determining a target adjustment parallax according to a preset system configuration, a system configuration of the head mounted display device, and the parallax to be adjusted includes: If the system configuration of the head mounted display device is the same as the preset system configuration, the parallax is adjusted with the parallax to be adjusted as a target.

3. The parallax adjustment method according to claim 1, characterized in that: The step of determining a target adjustment parallax according to a preset system configuration, a system configuration of the head mounted display device, and the parallax to be adjusted includes: If the system configuration of the head mounted display device is different from the preset system configuration, the target adjustment disparity corresponding to the disparity to be adjusted is determined based on a preset correspondence between the target adjustment disparity and the disparity to be adjusted.

4. The parallax adjustment method according to claim 3, characterized in that: The correspondence between the preset target adjustment parallax and the to-be-adjusted parallax includes a correspondence between the to-be-adjusted parallax and a first depth, and a correspondence between the first depth and the target adjustment parallax, the correspondence between the to-be-adjusted parallax and the first depth corresponds to the preset system configuration, and the correspondence between the first depth and the target adjustment parallax corresponds to the system configuration of the head mounted display device; If the system configuration of the head mounted display device is different from the preset system configuration, determining the target adjustment parallax corresponding to the parallax to be adjusted based on the preset correspondence between the target adjustment parallax and the parallax to be adjusted includes: If the system configuration of the head mounted display device is different from the preset system configuration, determining the first depth according to the correspondence between the parallax to be adjusted and the first depth; The target adjustment disparity corresponding to the to-be-adjusted disparity is determined according to the correspondence between the first depth and the target adjustment disparity.

5. The parallax adjustment method according to any one of claims 1 to 4, characterized in that: The adjusting the parallax according to the target and adjusting the display parameters of the microdisplay includes: The parallax is adjusted according to the target, and the lateral coordinates of the human eye image displayed by the microdisplay are adjusted to adjust the display parameters of the microdisplay.

6. A method for training a disparity estimation model, characterized in that: Applied to head mounted display devices, the training methods include: Acquire a plurality of training data, wherein the training data includes human eye images and annotated disparities corresponding to the human eye images; Based on a disparity estimation model, the disparity of the human eye image is estimated to obtain a current disparity; adjusting a model parameter of the disparity estimation model according to a deviation between the current disparity and the marked disparity; The adjusted disparity estimation model is stored, and the stored disparity estimation model is used to estimate the disparity of the human eye image to obtain the disparity to be adjusted.

7. The method for training a disparity estimation model according to claim 6, characterized in that: The human eye image includes a plurality of images with different first depths, and the parallax to be adjusted has a corresponding relationship with the first depth.

8. A parallax adjustment device, applied to a head-mounted display device, wherein the head-mounted display device includes a micro display, characterized in that: include: An acquisition module, used for acquiring human eye images; An estimation module, used for estimating the disparity of the human eye image based on a preset disparity estimation model to obtain a disparity to be adjusted; A determination module, configured to determine a target adjustment parallax according to a preset system configuration, a system configuration of the head mounted display device, and the parallax to be adjusted; The adjustment module is used to adjust the parallax according to the target and adjust the display parameters of the micro display.

9. A head mounted display device, characterized in that: The head mounted display device comprises a memory and a processor; The memory is used to store computer programs; The processor is used to execute the computer program and implement the disparity adjustment method as described in any one of claims 1 to 5 or the disparity estimation model training method as described in any one of claims 6 to 7 when executing the computer program.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the steps of the disparity adjustment method described in any one of claims 1 to 5 or the steps of the disparity estimation model training method described in any one of claims 6 to 7 are implemented.

Images