Shooting method, video display method, electronic equipment and storage medium

Abstract

The embodiment of the invention discloses a shooting method, a video display method, electronic equipment and a storage medium, the shooting method is applied to the electronic equipment, the electronic equipment comprises a first camera module and a second camera module, the first camera module is used for shooting eyeball images of a user, and the second camera module is used for shooting eyeball images of the user. The second camera module is used for shooting an environment image in front of the user, and the method comprises the following steps: shooting an eyeball image of the user through the first camera module; determining eyeball state information of the user according to the eyeball image; according to the eyeball state information, a target value of a first shooting parameter of the second camera module is determined, and the first shooting parameter comprises at least one of an aperture and a focusing distance; and adjusting the numerical value of the first shooting parameter of the second camera module to the target numerical value, and after the adjustment is completed, controlling the second camera module to shoot an environment image in front of the user.

Description

Technical Field

[0001] This application relates to the technical field of device control, and particularly to a shooting method, a video display method, an electronic device, and a storage medium. Background Art

[0002] Currently, Video See Through (VST) technology is commonly used in electronic devices such as head-mounted Virtual Reality (VR), Augmented Reality (AR), and Mixed Reality (MR), so that users can still see the real world clearly even when their line of sight is blocked by the device. A color camera module (Camera Compact Module, CCM) is installed on the electronic device, and the external real-world image within the line of sight directly in front of the user wearing the electronic device is captured by the color camera module, digitally processed by an image processing chip, and finally the display screen of the electronic device replays the image in front of the user's eyes.

[0003] However, VST technology has relatively strict requirements for the color camera module: First, the replay image must be clear everywhere without out-of-focus blur of the lens, which requires the aperture of the lens to be as small as possible to obtain the maximum shooting depth of field; Second, during the movement of the user himself or the movement of real-world objects, motion blur should be avoided, which requires the shutter of the lens to be as fast as possible, that is, the single-frame exposure duration to be as short as possible. The combination of these two factors, a small aperture and a short exposure, will result in a very low light input of the color camera module, with a problem of too low signal-to-noise ratio and the image being prone to noise. Summary of the Invention

[0004] Embodiments of this application provide a shooting method, a video display method, an electronic device, and a storage medium, which can improve the signal-to-noise ratio of the captured image and reduce image noise.

[0005] According to a first aspect of this application, a shooting method is disclosed, which is applied to an electronic device. The electronic device includes a first camera module and a second camera module. The first camera module is used to capture an eye image of a user, and the second camera module is used to capture an environmental image directly in front of the user. The method includes:

[0006] Capture the eye image of the user through the first camera module;

[0007] Determine the eye state information of the user according to the eye image;

[0008] Determine a target value of a first shooting parameter of the second camera module according to the eye state information, where the first shooting parameter includes at least one of the following: aperture and focusing distance;

[0009] Adjust the value of the first shooting parameter of the second camera module to the target value, and after the adjustment is completed, control the second camera module to shoot an environmental image in front of the user.

[0010] According to a second aspect of the present application, a shooting device is disclosed, which is applied to an electronic device. The electronic device includes a first camera module and a second camera module. The first camera module is used to shoot an eye image of the user, and the second camera module is used to shoot an environmental image in front of the user. The device includes:

[0011] A first shooting module, configured to shoot the eye image of the user through the first camera module;

[0012] A first determination module, configured to determine the eye state information of the user according to the eye image;

[0013] A second determination module, configured to determine a target value of a first shooting parameter of the second camera module according to the eye state information, where the first shooting parameter includes at least one of the following: aperture and focusing distance;

[0014] A setting module, configured to adjust the value of the first shooting parameter of the second camera module to the target value;

[0015] A second shooting module, configured to control the second camera module to shoot an environmental image in front of the user after the adjustment is completed.

[0016] According to a third aspect of the present application, a video display method is disclosed, and the method includes:

[0017] Obtain an environmental image according to the shooting method described in the first aspect;

[0018] Perform digital processing on the environmental image to obtain a digitally processed image;

[0019] Display the digitally processed image on a display screen of the electronic device.

[0020] According to a fourth aspect of the present application, a video display device is disclosed, and the device includes:

[0021] An obtaining module, configured to obtain an environmental image according to the shooting device described in the second aspect;

[0022] A processing module, configured to perform digital processing on the environmental image to obtain a digitally processed image;

[0023] A display module, configured to display the digitally processed image on a display screen of an electronic device.

[0024] According to a fifth aspect of the present application, an electronic device is disclosed, including: a memory, a processor, and a computer program stored on the memory, where the processor executes the computer program to implement the shooting method in the first aspect or the video display method in the third aspect.

[0025] According to a sixth aspect of the present application, a computer-readable storage medium is disclosed, on which a computer program / instructions are stored, and when the computer program / instructions are executed by a processor, the shooting method in the first aspect or the video display method in the third aspect is implemented.

[0026] According to a seventh aspect of the present application, a computer program product is disclosed, including computer program / instructions, and when the computer program / instructions are executed by a processor, the shooting method in the first aspect or the video display method in the third aspect is implemented.

[0027] In an embodiment of the present application, an eyeball image of a user is captured by a first camera module; according to the eyeball image, the eyeball state information of the user is determined; according to the eyeball state information, a target value of a first shooting parameter of a second camera module is determined, where the first shooting parameter includes at least one of the following: aperture and focusing distance; the value of the first shooting parameter of the second camera module is adjusted to the target value, and after the adjustment is completed, the second camera module is controlled to capture an environmental image in front of the user. It can be seen that in the embodiment of the present application, since the eyeball state information of the user can directly reflect the user's perception of the current environment and the focus of attention, and the aperture and focusing distance of the second camera module are one of the key factors affecting the signal-to-noise ratio of the captured image, setting the value of the aperture and / or the value of the focusing distance of the second camera module according to the eyeball state information of the user can improve the signal-to-noise ratio of the captured image and reduce image noise. Description of the Drawings

[0028] Figure 1 is an exemplary diagram of an electronic device provided by an embodiment of the present application;

[0029] Figure 2 is a flowchart of a shooting method provided by an embodiment of the present application;

[0030] Figure 3 is an exemplary diagram of a shooting method provided by an embodiment of the present application;

[0031] Figure 4It is a flowchart of a video display method provided by an embodiment of the present application;

[0032] Figure 5 It is a schematic structural diagram of a shooting device provided by an embodiment of the present application;

[0033] Figure 6 It is a schematic structural diagram of a video display device provided by an embodiment of the present application;

[0034] Figure 7 It is a structural block diagram of an electronic device provided by an embodiment of the present application. Detailed implementation manners

[0035] To make the above objects, features, and advantages of the present application more obvious and understandable, the present application will be further described in detail below with reference to the accompanying drawings and specific implementation manners.

[0036] It should be noted that, for the method embodiments, for simplicity of description, they are all expressed as a series of action combinations. However, those skilled in the art should know that the embodiments of the present application are not limited by the described action sequences, because according to the embodiments of the present application, certain steps can be performed in other sequences or simultaneously. Secondly, those skilled in the art should also know that the embodiments described in the specification are all preferred embodiments, and the actions involved are not necessarily essential for the embodiments of the present application.

[0037] The embodiments of the present application provide a shooting method, a video display method, an electronic device, and a storage medium to improve the signal-to-noise ratio of the captured image and reduce image noise.

[0038] Next, with reference to the accompanying drawings, a shooting method and a video display method provided by the embodiments of the present application will be introduced.

[0039] It should be noted that the shooting method and the video display method provided by the embodiments of the present application are both applicable to an electronic device, which includes a first camera module and a second camera module. The first camera module is used to capture the eye image of the user, that is, the first camera module is essentially an eye movement camera module, and the second camera module is used to capture the environmental image in front of the user, that is, the second camera module is essentially a video perspective camera module. In practical applications, the electronic device may include: head-mounted display devices such as VR devices, AR devices, and MR devices. The embodiments of the present application do not limit this.

[0040] Exemplarily, when the electronic device is a head-mounted display device, as Figure 1 shown, the electronic device 10 is provided with a first camera module 11, a first camera module 12, a second camera module 13, and an adjustable strap 14.

[0041] Among them, the first camera module 11 and the first camera module 12 are arranged inside the electronic device 10. The first camera module 11 and the first camera module 12 are usually symmetrically arranged. The first camera module 11 is used to capture the eye image of the user's right eye, and the first camera module 12 is used to capture the eye image of the user's left eye;

[0042] The second camera module 13 is arranged outside the electronic device 10, usually at a position slightly above the exact middle of the outside of the electronic device 10. The second camera module 13 is used to capture the environmental image in front of the user.

[0043] In addition to the above structure, in another scenario, the second camera module may include two camera modules. Among them, one camera module is arranged at the position corresponding to the right eye on the outside of the electronic device to capture the environmental image in front of the user's right eye; the other camera module is arranged at the position corresponding to the left eye on the outside of the electronic device to capture the environmental image in front of the user's left eye.

[0044] Figure 2 is a flowchart of a shooting method provided by an embodiment of the present application. As Figure 2 shown, the method may include the following steps: Step 201, Step 202, Step 203, and Step 204;

[0045] In step 201, the eye image of the user is captured through the first camera module.

[0046] In the embodiment of the present application, the eye image of a single eye of the user can be captured.

[0047] Exemplarily, after the user wears the electronic device, the eye image of the user's left eye can be captured through the first camera module corresponding to the left eye position in the electronic device. Or, after the user wears the electronic device, the eye image of the user's right eye can be captured through the first camera module corresponding to the right eye position in the electronic device.

[0048] In the embodiment of the present application, the eye images of both eyes of the user can be captured.

[0049] Exemplarily, after the user wears the electronic device, the eye image of the user's right eye can be captured through the first camera module corresponding to the right eye position in the electronic device, and at the same time, the eye image of the user's right eye can be captured through the first camera module corresponding to the right eye position in the electronic device.

[0050] In step 202, according to the eye image, the eye state information of the user is determined.

[0051] In the embodiment of the present application, in the case where a single eye image is captured, the monocular eye state information of the user is determined according to the single eye image.

[0052] In the embodiments of the present application, in the case where two eye images are captured, the binocular eye state information of the user is determined according to the two eye images.

[0053] In the embodiments of the present application, the eye state information of the user may include at least one of the following: the binocular line-of-sight focusing distance of the user and the changing trend of the pupil diameter of the user.

[0054] In the embodiments of the present application, the binocular line-of-sight focusing distance of the user refers to the distance from the focusing position of the user's binocular line of sight in the environment to the electronic device or the user's eyes. The binocular line-of-sight focusing distance of the user can be used to reflect the distance of the focus of the user's gaze in the current environment.

[0055] In the embodiments of the present application, the pupil refers to a small circular aperture in the human eye. It is surrounded by the iris and is the passage for light to enter the eye; when irradiated by light, the pupil will shrink, and when the light is relatively dim, the pupil will dilate. Correspondingly, the changing trend of the pupil diameter of the user includes the trend of pupil constriction and the trend of pupil dilation. The changing trend of the pupil diameter of the user can be used to reflect the user's perception of the brightness of the current environment.

[0056] It can be seen that in the embodiments of the present application, since the binocular line-of-sight focusing distance of the user can intuitively reflect the distance of the focus of the user's gaze in the current environment, and the changing trend of the pupil diameter of the user can intuitively reflect the user's perception of the brightness of the current environment, and the user's perception of the brightness of the current environment and the focus of the gaze can determine which contents in the current environment the user's eyes expect to view. Therefore, using at least one of the binocular line-of-sight focusing distance of the user and the changing trend of the pupil diameter of the user as the eye state information of the user and adjusting the shooting parameters of the second camera module according to the eye state information can ensure that the picture captured by the second camera module is a clear picture that the user expects to see.

[0057] In step 203, according to the eye state information, the target value of the first shooting parameter of the second camera module is determined, where the first shooting parameter includes at least one of the following: aperture and focusing distance.

[0058] In the embodiments of the present application, considering that the aperture and focusing distance of the second camera module are one of the key factors affecting the signal-to-noise ratio of the captured picture, the aperture and / or the focusing distance are determined as the first shooting parameter of the second camera module.

[0059] In some embodiments, the aperture value of the second camera module is fixed and the aperture value is greater than the first value; the eye state information is the binocular line-of-sight focusing distance of the user, and the first shooting parameter is the focusing distance; correspondingly, the above step 203 may include the following steps: step 2031;

[0060] In step 2031, according to the binocular line-of-sight focusing distance of the user, determine the target value of the focusing distance of the second camera module; wherein, the target value of the focusing distance is positively correlated with the binocular line-of-sight focusing distance.

[0061] In the embodiment of the present application, the second camera module is a camera module with a fixed large aperture.

[0062] In the embodiment of the present application, the binocular line-of-sight focusing distance of the user can be determined as the target value of the focusing distance of the second camera module, so as to ensure that the focusing distance of the image of the second camera module is consistent with the state of the user's eyes, and ensure that the area of the image focused by the human eyes is clear.

[0063] In the embodiment of the present application, although using a larger aperture will cause the image outside the focusing distance to be out of focus and blurred, this is consistent with the blurred effect when the human eyes directly view the external world, and will not bring an unclear or unnatural feeling to the user; in addition, due to the larger aperture bringing a larger amount of incident light, the signal-to-noise ratio of the second camera module is greatly improved, so the noise in the image can be well suppressed.

[0064] In some embodiments, the aperture value of the second camera module is variable and the lower limit value of the aperture value is greater than the second value; the eye state information is the change trend of the pupil diameter of the user, and the first shooting parameter is the aperture; correspondingly, the above step 203 may include the following steps: step 2032;

[0065] In step 2032, according to the change trend of the pupil diameter of the user, determine the target value of the aperture of the second camera module; wherein, the target value of the aperture is positively correlated with the change trend of the pupil diameter.

[0066] In the embodiment of the present application, the second camera module is a camera module with a variable aperture.

[0067] In the embodiment of the present application, when the change trend of the pupil diameter of the user is the change trend of the pupil diameter of a single eye, determine the target value of the aperture of the second camera module according to the change trend of the pupil diameter of the single eye; when the change trend of the pupil diameter of the user includes the change trend of the pupil diameter of both eyes, calculate the mean value of the change trend of the pupil diameter of both eyes, and determine the target value of the aperture of the second camera module according to the mean value.

[0068] It can be seen that in the embodiment of the present application, the change trend of the pupil diameter of a single eye can be used to determine the target value of the aperture of the second camera module, or the change trend of the pupil diameter of both eyes can be used to determine the target value of the aperture of the second camera module, and the flexibility is relatively high.

[0069] In the embodiments of the present application, when the change trend of the pupil diameter indicates that the pupil is shrinking, the smaller the target value of the aperture; when the change trend of the pupil diameter indicates that the pupil is enlarging, the larger the target value of the aperture.

[0070] In the embodiments of the present application, since the human eye pupil constantly changes according to factors such as light intensity, the distance of the focused object, and the sympathetic nerve excitation state, adjusting the aperture according to the change of the pupil diameter can obtain a visual experience that is more in line with the natural viewing of the external picture by the human eye; in addition, when the aperture of the second camera module changes with the change of the human eye pupil, it can reduce the aperture in a bright environment to provide a larger depth of field, and expand the aperture in a dark environment to provide a higher signal-to-noise ratio, which can well suppress the noise in the picture, thereby balancing the video perspective effect in different brightness environments.

[0071] In some embodiments, the aperture value of the second camera module is variable and the lower limit value of the aperture value is greater than the second value; the eye state information includes the binocular line-of-sight focusing distance of the user and the change trend of the user's pupil diameter, and the first shooting parameters include the aperture and the focusing distance; correspondingly, the above step 203 may include the following steps: step 2033 and step 2034;

[0072] In step 2033, according to the binocular line-of-sight focusing distance of the user, determine the target value of the focusing distance of the second camera module; wherein, the target value of the focusing distance is positively correlated with the binocular line-of-sight focusing distance.

[0073] In the embodiments of the present application, the second camera module is a camera module with a variable aperture.

[0074] In the embodiments of the present application, the binocular line-of-sight focusing distance of the user can be determined as the target value of the focusing distance of the second camera module.

[0075] In step 2034, according to the change trend of the user's pupil diameter, determine the target value of the aperture of the second camera module; wherein, the target value of the aperture is positively correlated with the change trend of the pupil diameter.

[0076] In the embodiments of the present application, when the change trend of the user's pupil diameter is the change trend of the pupil diameter of one eye, determine the target value of the aperture of the second camera module according to the change trend of the pupil diameter of one eye; when the change trend of the user's pupil diameter includes the change trend of the pupil diameter of both eyes, calculate the average value of the change trend of the pupil diameter of both eyes, and determine the target value of the aperture of the second camera module according to the average value.

[0077] The content of step 2033 and step 2034 in the embodiments of the present application is similar to the content of the above step 2031 and the above step 2032, and will not be elaborated here.

[0078] In the embodiments of the present application, the focusing distance of the second camera module can be adjusted according to the binocular line-of-sight focusing distance of the user, and at the same time, the aperture of the second camera module can be adjusted according to the changing trend of the user's pupil diameter, so as to achieve a clearer and more natural video see-through effect on the premise of ensuring the reduction of noise in the picture.

[0079] In step 204, the value of the first shooting parameter of the second camera module is adjusted to the target value, and after the adjustment is completed, the second camera module is controlled to shoot the environmental image in front of the user.

[0080] In the embodiments of the present application, when the first shooting parameter is the focusing distance, the value of the focusing distance of the second camera module is adjusted to the target value of the determined focusing distance, and after the adjustment is completed, the second camera module is controlled to shoot the environmental image in front of the user.

[0081] In the embodiments of the present application, when the first shooting parameter is the aperture, the value of the aperture of the second camera module is adjusted to the target value of the determined aperture, and after the adjustment is completed, the second camera module is controlled to shoot the environmental image in front of the user.

[0082] In the embodiments of the present application, when the first shooting parameter includes the focusing distance and the aperture, the value of the aperture of the second camera module is adjusted to the target value of the determined focusing distance, and the value of the focusing distance of the second camera module is adjusted to the target value of the determined aperture. After both adjustments are completed, the second camera module is controlled to shoot the environmental image in front of the user.

[0083] In the embodiments of the present application, the eye tracking technology in the electronic device can be utilized. Exemplarily, as Figure 3 shown, under the architecture of the existing eye tracking system and video see-through system, the eye camera module in the eye tracking system shoots the eye image of the user, and then through image processing and the neural network processor, the eye state information of the user is obtained; the video see-through camera module in the video see-through system adjusts its own shooting parameters according to the eye state information of the user. After the shooting parameters are adjusted, the environmental image of the external world is shot, and then through digital processing, the digitally processed image of the external world is displayed on the display screen of the electronic device. In the above process, the existing eye tracking system and video see-through system in the electronic device are reused, and the hardware cost of the electronic device will not be increased.

[0084] As can be seen from the above embodiments, in this embodiment, an eye image of the user is captured by the first camera module; according to the eye image, the eye state information of the user is determined; according to the eye state information, the target value of the first shooting parameter of the second camera module is determined, where the first shooting parameter includes at least one of the following: aperture and focusing distance; the value of the first shooting parameter of the second camera module is adjusted to the target value, and after the adjustment is completed, the second camera module is controlled to capture an environmental image in front of the user. It can be seen that in the embodiments of the present application, since the eye state information of the user can directly reflect the user's perception of the current environment and the focus of attention, and the aperture and focusing distance of the second camera module are one of the key factors affecting the signal-to-noise ratio of the captured image, setting the value of the aperture and / or the value of the focusing distance of the second camera module according to the eye state information of the user can improve the signal-to-noise ratio of the captured image and reduce image noise.

[0085] Figure 4 is a flowchart of a video display method provided by an embodiment of the present application, which is applied to an electronic device, such as Figure 4 shown, the method may include the following steps: step 401, step 402, and step 403;

[0086] In step 401, an environmental image captured by the second camera module of the electronic device is obtained.

[0087] In the embodiments of the present application, the environmental image is a series of images captured by the second camera module of the electronic device based on any of the above shooting methods; wherein, the content of each environmental image is a picture of the external real world within the line of sight in front of the user.

[0088] In step 402, the environmental image is digitally processed to obtain a digitally processed image.

[0089] In the embodiments of the present application, the image processing chip of the electronic device can digitally process the environmental image so as to convert it into an image format that can be displayed on the display screen of the electronic device.

[0090] In step 403, the digitally processed image is displayed on the display screen of the electronic device.

[0091] In the embodiments of the present application, the digitally processed image is displayed on the display screen of the electronic device, so that the picture of the external real world within the line of sight in front of the user can be played back in front of the user's eyes.

[0092] It can be seen that in the embodiments of the present application, since the obtained environmental image is an image with a high signal-to-noise ratio, the picture displayed on the display screen after the environmental image is digitally processed is also a high-quality picture, improving the user experience of using the electronic device.

[0093] Figure 5 It is a schematic structural diagram of a photographing device provided by an embodiment of the present application, which is applied to an electronic device. The electronic device includes a first camera module and a second camera module. The first camera module is used to photograph the eye image of the user, and the second camera module is used to photograph the environmental image in front of the user. As Figure 5 shown, the photographing device 500 may include: a first photographing module 501, a first determination module 502, a second determination module 503, a setting module 504, and a second photographing module 505;

[0094] The first photographing module 501 is used to photograph the eye image of the user through the first camera module;

[0095] The first determination module 502 is used to determine the eye state information of the user according to the eye image;

[0096] The second determination module 503 is used to determine the target value of the first shooting parameter of the second camera module according to the eye state information, where the first shooting parameter includes at least one of the following: aperture and focusing distance;

[0097] The setting module 504 is used to adjust the value of the first shooting parameter of the second camera module to the target value;

[0098] The second photographing module 505 is used to control the second camera module to photograph the environmental image in front of the user after the adjustment is completed.

[0099] As can be seen from the above embodiments, in this embodiment, the eye image of the user is photographed through the first camera module; the eye state information of the user is determined according to the eye image; the target value of the first shooting parameter of the second camera module is determined according to the eye state information, where the first shooting parameter includes at least one of the following: aperture and focusing distance; the value of the first shooting parameter of the second camera module is adjusted to the target value, and after the adjustment is completed, the second camera module is controlled to photograph the environmental image in front of the user. It can be seen that in the embodiment of the present application, since the eye state information of the user can directly reflect the user's perception of the current environment and the focus of attention, and the aperture and focusing distance of the second camera module are one of the key factors affecting the signal-to-noise ratio of the photographed image, setting the value of the aperture and / or the value of the focusing distance of the second camera module according to the eye state information of the user can improve the signal-to-noise ratio of the photographed image and reduce the image noise.

[0100] Optionally, as an embodiment, the eye state information may include at least one of the following: the binocular line-of-sight focusing distance of the user and the change trend of the pupil diameter of the user.

[0101] Optionally, as an embodiment, the aperture value of the second camera module is fixed and the aperture value is greater than a first value;

[0102] The eye state information is the binocular line-of-sight focusing distance of the user, and the first shooting parameter is the focusing distance;

[0103] The second determination module 503 may include:

[0104] A first determination sub-module, configured to determine a target value of the focusing distance of the second camera module according to the binocular line-of-sight focusing distance of the user; wherein, the target value of the focusing distance is positively correlated with the binocular line-of-sight focusing distance.

[0105] Optionally, as an embodiment, the aperture value of the second camera module is variable and the lower limit value of the aperture value is greater than a second value;

[0106] The eye state information is the change trend of the pupil diameter of the user, and the first shooting parameter is the aperture;

[0107] The second determination module 503 may include:

[0108] A second determination sub-module, configured to determine a target value of the aperture of the second camera module according to the change trend of the pupil diameter of the user; wherein, the target value of the aperture is positively correlated with the change trend of the pupil diameter.

[0109] Optionally, as an embodiment, the aperture value of the second camera module is variable and the lower limit value of the aperture value is greater than a second value;

[0110] The eye state information includes the binocular line-of-sight focusing distance of the user and the change trend of the pupil diameter of the user, and the first shooting parameters include the aperture and the focusing distance;

[0111] The second determination module 503 may include:

[0112] A third determination sub-module, configured to determine a target value of the focusing distance of the second camera module according to the binocular line-of-sight focusing distance of the user; wherein, the target value of the focusing distance is positively correlated with the binocular line-of-sight focusing distance;

[0113] A fourth determination sub-module, configured to determine a target value of the aperture of the second camera module according to the change trend of the pupil diameter of the user; wherein, the target value of the aperture is positively correlated with the change trend of the pupil diameter.

[0114] Optionally, as an embodiment, the first determination sub-module or the third determination sub-module may specifically be configured to:

[0115] Determine the focusing distance of the user's binocular line of sight as the target value of the focusing distance of the second camera module.

[0116] Optionally, as an embodiment, the second determination sub-module or the fourth determination sub-module may specifically be configured to:

[0117] When the change trend of the pupil diameter of the user is the change trend of the pupil diameter of one eye, determine the target value of the aperture of the second camera module according to the change trend of the pupil diameter of the one eye;

[0118] When the change trend of the pupil diameter of the user includes the change trend of the pupil diameter of both eyes, calculate the mean value of the change trend of the pupil diameter of both eyes, and determine the target value of the aperture of the second camera module according to the mean value.

[0119] Any step in the embodiments of the shooting method provided in this application and the specific operations in any step can be completed by the corresponding module in the shooting device. The process of the corresponding operations completed by each module in the shooting device refers to the process of the corresponding operations described in the embodiments of the shooting method.

[0120] Figure 6 It is a schematic structural diagram of a video display device provided in an embodiment of this application, as Figure 6 shown, the video display device 600 may include: an acquisition module 601, a processing module 602, and a display module 603;

[0121] The acquisition module 601 is configured to acquire the environmental image captured by the above shooting device;

[0122] The processing module 602 is configured to perform digital processing on the environmental image to obtain a digitally processed image;

[0123] The display module 603 is configured to display the digitally processed image on the display screen of the electronic device.

[0124] It can be seen that in the embodiments of this application, since the acquired environmental image is an image with a high signal-to-noise ratio, after the environmental image is digitally processed, the picture displayed on the display screen is also a high-quality picture, improving the user experience of using the electronic device.

[0125] Any step in the embodiments of the video display method provided in this application and the specific operations in any step can be completed by the corresponding modules in the video display device. The process of the corresponding operations completed by each module in the video display device refers to the process of the corresponding operations described in the embodiments of the video display method.

[0126] For the device embodiments, since they are basically similar to the method embodiments, the description is relatively simple. For the relevant parts, please refer to the partial description of the method embodiments.

[0127] Figure 7 is a structural block diagram of an electronic device provided in an embodiment of the present application, as Figure 7 shown, the electronic device includes a processing component 722, which further includes one or more processors, and memory resources represented by a memory 732 for storing instructions executable by the processing component 722, such as application programs. The application programs stored in the memory 732 may include one or more modules each corresponding to a set of instructions. In addition, the processing component 722 is configured to execute instructions to perform the above-mentioned shooting method or the above-mentioned video display method.

[0128] The electronic device may further include a power supply component 726 configured to perform power management of the electronic device, a wired or wireless network interface 750 configured to connect the electronic device to a network, and an input / output (I / O) interface 758. The electronic device may operate based on an operating system stored in the memory 732, such as Windows ServerTM, MacOS XTM, UnixTM, LinuxTM, FreeBSDTM or the like.

[0129] Optionally, the electronic device is a head-mounted display device, and the first camera module and the second camera module are installed on the electronic device, wherein the first camera module is used to capture an eye image of the user, and the second camera module is used to capture an environmental image in front of the user.

[0130] According to still another embodiment of the present application, the present application further provides a computer-readable storage medium, on which computer programs / instructions are stored, and when the computer programs / instructions are executed by a processor, the steps in the shooting method described in any of the above embodiments are implemented, or the steps in the above video display method are implemented.

[0131] According to still another embodiment of the present application, the present application further provides a computer program product, including computer programs / instructions, and when the computer programs / instructions are executed by a processor, the steps in the shooting method described in any of the above embodiments are implemented, or the steps in the above video display method are implemented.

[0132] Each embodiment in this specification is described in a progressive manner. The key point of each embodiment is to illustrate the differences from other embodiments. For the same or similar parts among the embodiments, reference can be made to each other.

[0133] Those skilled in the art should understand that the embodiments of the present application can be provided as methods, devices, or computer program products. Therefore, the embodiments of the present application can take the form of all-hardware embodiments, all-software embodiments, or embodiments combining software and hardware aspects. Moreover, the embodiments of the present application can take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk memories, CD-ROMs, optical memories, etc.) containing computer-usable program codes.

[0134] The embodiments of the present application are described with reference to the flowcharts and / or block diagrams of methods, terminal devices (systems), and computer program products according to the embodiments of the present application. It should be understood that each flow and / or block in the flowchart and / or block diagram, as well as the combination of flows and / or blocks in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to the processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing terminal devices to generate a machine, such that the instructions executed by the processor of the computer or other programmable data processing terminal devices generate a device for implementing the functions specified in Figure 1 one or more of the flows Figure 1 or blocks or the combination of multiple flows and / or blocks.

[0135] These computer program instructions can also be stored in a computer-readable memory that can direct a computer or other programmable data processing terminal devices to work in a specific manner, such that the instructions stored in the computer-readable memory generate a manufactured article including an instruction device, and the instruction device implements the functions specified in Figure 1 one or more of the flows Figure 1 or blocks or the combination of multiple flows and / or blocks.

[0136] Although the preferred embodiments of the embodiments of the present application have been described, those skilled in the art can make additional changes and modifications once they know the basic creative concepts. Therefore, the appended claims are intended to be construed to include the preferred embodiments and all changes and modifications falling within the scope of the embodiments of the present application.

[0137] Finally, it should also be noted that in this text, relational terms such as first and second are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any actual relationship or order between these entities or operations. Moreover, the term "comprising", "including" or any other variant thereof is intended to cover non-exclusive inclusion, such that a process, method, article or terminal device comprising a series of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such process, method, article or terminal device. Without further limitation, an element defined by the statement "comprising an..." does not exclude the presence of additional identical elements in the process, method, article or terminal device comprising the element.

[0138] The above has introduced in detail a shooting method, a video display method, an electronic device and a storage medium provided by the present application. Specific examples are used in this text to elaborate on the principle and implementation manner of the present application. The description of the above embodiments is only used to help understand the method and its core idea of the present application; at the same time, for those of ordinary skill in the art, according to the idea of the present application, there will be changes in the specific implementation manner and application scope. In summary, the content of this specification should not be construed as a limitation to the present application.

Claims

1. A shooting method applied to an electronic device, the electronic device including a first camera module and a second camera module, the first camera module being used for shooting an eye image of a user, and the second camera module being used for shooting an environmental image directly in front of the user, characterized in that, The method includes: capturing the eye image of the user through the first camera module; determining the eye state information of the user according to the eye image; determining a target value of a first shooting parameter of the second camera module according to the eye state information, where the first shooting parameter includes at least one of the following: aperture and focusing distance; adjusting the value of the first shooting parameter of the second camera module to the target value, and after the adjustment is completed, controlling the second camera module to capture an environmental image in front of the user.

2. The method according to claim 1, wherein the eye state information includes at least one of the following: the binocular line-of-sight focusing distance of the user and the change trend of the pupil diameter of the user.

3. The method according to claim 2, wherein The aperture value of the second camera module is fixed and the aperture value is greater than a first value; the eye state information is the binocular line-of-sight focusing distance of the user, and the first shooting parameter is the focusing distance; The determining the target value of the first shooting parameter of the second camera module according to the eye state information includes: determining a target value of the focusing distance of the second camera module according to the binocular line-of-sight focusing distance of the user; wherein the target value of the focusing distance is positively correlated with the binocular line-of-sight focusing distance.

4. The method according to claim 2, wherein The aperture value of the second camera module is variable and the lower limit value of the aperture value is greater than a second value; the eye state information is the change trend of the pupil diameter of the user, and the first shooting parameter is the aperture; The determining the target value of the first shooting parameter of the second camera module according to the eye state information includes: determining a target value of the aperture of the second camera module according to the change trend of the pupil diameter of the user; wherein the target value of the aperture is positively correlated with the change trend of the pupil diameter.

5. The method according to claim 2, wherein The aperture value of the second camera module is variable and the lower limit value of the aperture value is greater than a second value; the eye state information includes the binocular line-of-sight focusing distance of the user and the change trend of the pupil diameter of the user, and the first shooting parameter includes the aperture and the focusing distance; The determining the target value of the first shooting parameter of the second camera module according to the eye state information includes: determining a target value of the focusing distance of the second camera module according to the binocular line-of-sight focusing distance of the user; wherein the target value of the focusing distance is positively correlated with the binocular line-of-sight focusing distance; determining a target value of the aperture of the second camera module according to the change trend of the pupil diameter of the user; wherein the target value of the aperture is positively correlated with the change trend of the pupil diameter.

6. The method according to claim 3 or 5, characterized in that The determining the target value of the focusing distance of the second camera module according to the binocular line-of-sight focusing distance of the user includes: determining the binocular line-of-sight focusing distance of the user as the target value of the focusing distance of the second camera module.

7. The method according to claim 4 or 5, characterized in that The determining the target value of the aperture of the second camera module according to the change trend of the pupil diameter of the user includes: When the change trend of the pupil diameter of the user is the change trend of the pupil diameter of one eye, determine the target value of the aperture of the second camera module according to the change trend of the pupil diameter of the one eye; When the change trend of the pupil diameter of the user includes the change trend of the pupil diameter of both eyes, calculate the average value of the change trend of the pupil diameter of both eyes, and determine the target value of the aperture of the second camera module according to the average value.

8. A video display method, characterized in that, The method includes: Obtain an environmental image according to the shooting method of any one of claims 1-7; Perform digital processing on the environmental image to obtain a digitally processed image; Display the digitally processed image on the display screen of the electronic device.

9. An electronic device, comprising: A memory, a processor, and a computer program stored on the memory, wherein the processor executes the computer program to implement the shooting method of any one of claims 1-7, or implement the video display method of claim 8.

10. The electronic device according to claim 9, wherein The electronic device is a head-mounted display device, and a first camera module and a second camera module are installed on the electronic device, wherein the first camera module is used to capture an eye image of the user, and the second camera module is used to capture an environmental image in front of the user.

11. A computer-readable storage medium having computer programs / instructions stored thereon, characterized in that, When the computer program / instructions are executed by the processor, the shooting method of any one of claims 1-7 is implemented, or the video display method of claim 8 is implemented.

Images