Shooting method, head-mounted display device and storage medium

Abstract

The embodiment of the invention provides a shooting method, head-mounted display equipment and a storage medium, and belongs to the field of shooting. The method comprises the following steps: acquiring a pre-shot image of a shooting device of the head-mounted display equipment; performing scene type identification on the pre-shot image through a preset scene type identification model to obtain a current scene type; according to the current scene type, controlling the head-mounted display device to output corresponding style recommendation information, the style recommendation information being used for prompting a wearer of the head-mounted display device with a target style suitable for the current scene; and in response to a photographing operation of the wearer, controlling the photographing device to photograph based on the photographing parameters corresponding to the target style to obtain a target image. According to the technical scheme provided by the embodiment of the invention, the intelligence and convenience of shooting various styles of images by the head-mounted display equipment are improved.

Description

Technical Field

[0001] This invention relates to the field of photography technology, and more particularly to a photography method, a head-mounted display device, and a storage medium. Background Technology

[0002] With the rapid popularization of head-mounted display devices, these portable image acquisition and information interaction devices have been widely used in various scenarios such as outdoor recording, daily life photography, work recording, and remote collaboration. Currently, head-mounted display devices mainly use fixed shooting parameters, resulting in images with a single style. If users want to capture images with different styles, they need to manually configure the shooting parameters in advance, which is not convenient or intelligent, and leads to a poor user experience. Summary of the Invention

[0003] This invention provides a shooting method, a head-mounted display device, and a storage medium, aiming to improve the intelligence and convenience of shooting images of various styles using a head-mounted display device.

[0004] In a first aspect, embodiments of the present invention provide a shooting method, including: Acquire pre-captured images from the camera device of the head-mounted display device; The scene type is identified by a preset scene type recognition model to obtain the current scene type; Based on the current scene type, the head-mounted display device is controlled to output corresponding style recommendation information, which is used to suggest a target style suitable for the current scene to the wearer of the head-mounted display device. In response to the wearer's photo-taking operation, the camera is controlled to take a photo based on the shooting parameters corresponding to the target style to obtain the target image.

[0005] In a second aspect, embodiments of the present invention also provide a head-mounted display device, the head-mounted display device including a processor, a memory, a computer program stored in the memory and executable by the processor, and a data bus for implementing communication between the processor and the memory, wherein when the computer program is executed by the processor, it implements the shooting method as described in the first aspect.

[0006] Thirdly, embodiments of the present invention also provide a storage medium for computer-readable storage, wherein the storage medium stores one or more programs that can be executed by one or more processors to implement the shooting method as described in the first aspect.

[0007] This invention provides a shooting method, a head-mounted display device, and a storage medium. The invention uses a preset scene type recognition model to identify the scene type of a pre-captured image from the head-mounted display device's shooting device, obtaining the current scene type. Based on the current scene type, the head-mounted display device outputs corresponding style recommendation information to prompt the wearer to capture a target style suitable for the current scene. Thus, when the user triggers a photo-taking operation, the device can take a photo based on the shooting parameters corresponding to the target style, eliminating the need for the user to pre-configure shooting parameters and improving the intelligence and convenience of the head-mounted display device in capturing images of various styles. Attached Figure Description

[0008] To more clearly illustrate the technical solutions of the embodiments of this application, the drawings used in the description of the embodiments will be briefly introduced below. Obviously, the drawings described below are some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0009] Figure 1 This is a schematic diagram of a scene implementing the shooting method provided in the embodiments of the present invention; Figure 2 This is a schematic diagram of another scenario for implementing the shooting method provided in the embodiments of the present invention; Figure 3 This is a flowchart illustrating a shooting method provided in an embodiment of the present invention; Figure 4 yes Figure 3 A flowchart illustrating the sub-steps of the shooting method; Figure 5 This is a flowchart illustrating another shooting method provided in an embodiment of the present invention; Figure 6 This is a schematic block diagram of the structure of a head-mounted display device provided in an embodiment of the present invention. Detailed Implementation

[0010] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some, not all, of the embodiments of the present invention. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0011] The flowchart shown in the attached diagram is for illustrative purposes only and does not necessarily include all content and operations / steps, nor does it necessarily have to be performed in the order described. For example, some operations / steps can be broken down, combined, or partially merged, so the actual execution order may change depending on the actual situation.

[0012] It should be understood that the terminology used in this specification is for the purpose of describing particular embodiments only and is not intended to limit the invention. As used in this specification and the appended claims, the singular forms “a,” “an,” and “the” are intended to include the plural forms unless the context clearly indicates otherwise.

[0013] With the rapid popularization of head-mounted display devices, these portable image acquisition and information interaction devices have been widely used in various scenarios such as outdoor recording, daily life photography, work recording, and remote collaboration. Currently, head-mounted display devices mainly use fixed shooting parameters, resulting in images with a single style. If users want to capture images with different styles, they need to manually configure the shooting parameters in advance, which is not convenient or intelligent, and leads to a poor user experience.

[0014] To address the aforementioned problems, embodiments of the present invention provide a shooting method, a head-mounted display device, and a storage medium. The embodiments of the present invention use a preset scene type recognition model to identify the scene type of pre-captured images from the shooting device of the head-mounted display device, obtaining the current scene type. Based on the current scene type, the head-mounted display device outputs corresponding style recommendation information to prompt the wearer to capture a target style suitable for the current scene. Thus, when the user triggers a photo-taking operation, a photo can be taken based on the shooting parameters corresponding to the target style, eliminating the need for the user to pre-configure shooting parameters to obtain a target image suitable for the current scene. This improves the intelligence and convenience of the head-mounted display device in capturing images of various styles.

[0015] The following detailed description of some embodiments of the present invention is provided in conjunction with the accompanying drawings. Unless otherwise specified, the following embodiments and features can be combined with each other.

[0016] In some embodiments, the shooting method provided by the present invention can be applied to a head-mounted display device, which may include augmented reality (AR) glasses, AR helmets, mixed reality (MR) glasses, and MR helmets. For example, please refer to... Figure 1 , Figure 1 This is a schematic diagram of a scene for implementing the shooting method provided in the embodiments of the present invention.

[0017] like Figure 1 As shown, the head-mounted display device 100 is worn on the head 11 of the wearer. The head-mounted display device 100 includes a shooting device, a control device, and a display device. The display device includes an optical waveguide lens 101 and an optical engine (…). Figure 1(Not shown). For example, the head-mounted display device 100 acquires a pre-captured image from the shooting device; it performs scene type recognition on the pre-captured image using a preset scene type recognition model to obtain the current scene type; based on the current scene type, it controls the head-mounted display device to output corresponding style recommendation information, which is used to prompt the wearer of the head-mounted display device to capture a target style suitable for the current scene; in response to the wearer's photo-taking operation, it controls the shooting device to take a photo based on the shooting parameters corresponding to the target style to obtain the target image. The entire implementation process of the shooting method provided in this embodiment does not depend on external devices; it is all executed by the head-mounted display device 100. This avoids situations where the head-mounted display device 100 cannot output style recommendation information or needs to wait a long time to output style recommendation information due to communication breakdown between the head-mounted display device 100 and external devices, resulting in lower latency and a better user experience.

[0018] For example, the head-mounted display device 100 acquires a pre-captured image from the shooting device; determines a reference image matching the pre-captured image in a preset image library, and determines structural difference information between the pre-captured image and the reference image; performs scene type recognition on the pre-captured image using a preset scene type recognition model to obtain the current scene type; based on the current scene type, controls the head-mounted display device to output corresponding style recommendation information, which is used to prompt the wearer of the head-mounted display device to shoot a target style suitable for the current scene; based on the structural difference information, controls the head-mounted display device to output composition adjustment suggestions for shooting the target style image, so as to prompt the wearer to adjust the composition; in response to the wearer's shooting operation, controls the shooting device to take a picture based on the shooting parameters corresponding to the target style, and obtains the target image. This embodiment not only recommends a target style suitable for the current scene to the wearer, but also outputs composition adjustment suggestions to prompt the wearer on how to adjust the composition, making it more convenient for the wearer to shoot images of the corresponding style, and further improving the intelligence and convenience of the head-mounted display device in shooting images of various styles.

[0019] In some embodiments, the head-mounted display device 100 includes a sensor for acquiring attitude data of the head-mounted display device 100. Figure 1 (Not shown) and an eye-tracking detection device for detecting the wearer's eye movements. For example, sensors that acquire attitude data of the head-mounted display device 100 may include an inertial measurement unit (IMU), which may include an accelerometer, a gyroscope, and / or a magnetometer. For example, the inertial measurement unit includes a three-axis accelerometer, a three-axis gyroscope, and a three-axis magnetometer.

[0020] In some embodiments, the shooting method provided by this invention can be applied to a system consisting of a head-mounted display device and a server. The server can be a standalone server or a cloud server providing basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communication, middleware services, domain name services, security services, content delivery networks (CDNs), and big data and artificial intelligence platforms. Please refer to [link / reference]. Figure 2 , Figure 2 This is a schematic diagram of another scenario for implementing the shooting method provided in the embodiments of the present invention.

[0021] like Figure 2 As shown, the head-mounted display device 100 is communicatively connected to the server 200. The head-mounted display device 100 includes an imaging device, a control device, and a display device. The display device includes an optical waveguide lens 101 and an optical engine (…). Figure 1 (Not shown). In this embodiment, the head-mounted display device 100 acquires a pre-captured image from the shooting device and sends it to the server 200. The server 200 uses a preset scene type recognition model to identify the scene type of the pre-captured image, obtains the current scene type, and sends the current scene type to the head-mounted display device 100. Upon receiving the current scene type, the head-mounted display device 100 controls the output of corresponding style recommendation information based on the current scene type. This style recommendation information is used to prompt the wearer of the head-mounted display device to capture a target style suitable for the current scene. In response to the wearer's photo-taking operation, the head-mounted display device 100 controls the shooting device to take a photo based on the shooting parameters corresponding to the target style, thereby obtaining the target image. In this embodiment, the server 200 runs the scene type recognition model to identify the scene type of the pre-captured image, resulting in faster calculation speed, improved efficiency in outputting style recommendation information, and reduced computational power requirements for the head-mounted display device 100 to execute the shooting method provided in this embodiment. This approach offers greater versatility and scalability.

[0022] In some embodiments, the head-mounted display device 100 acquires a pre-captured image from the shooting device and sends the pre-captured image to the server 200; the head-mounted display device 100 performs scene type recognition on the pre-captured image using a preset scene type recognition model to obtain the current scene type; based on the current scene type, the head-mounted display device 100 controls the head-mounted display device to output corresponding style recommendation information, which is used to prompt the wearer of the head-mounted display device to shoot a target style suitable for the current scene; the server 200 determines a reference image matching the pre-captured image in a preset image library, determines the structural difference information between the pre-captured image and the reference image, and sends the structural difference information to the head-mounted display device 100; based on the structural difference information, the head-mounted display device 100 outputs composition adjustment suggestions for shooting an image of the target style; in response to the wearer's shooting operation, the head-mounted display device 100 controls the shooting device to take a picture based on the shooting parameters corresponding to the target style to obtain the target image. In this embodiment, the server 200 performs image matching and determines structural difference information, eliminating the need for the head-mounted display device 100 to store a preset image library. This reduces the storage requirements of the head-mounted display device 100 in executing the shooting method provided in this embodiment of the invention, and provides greater versatility and scalability.

[0023] In some embodiments, the head-mounted display device 100 acquires a pre-captured image from the shooting device and sends the pre-captured image to the server 200. The server 200 performs scene type recognition on the pre-captured image using a preset scene type recognition model to obtain the current scene type and a reference image matching the pre-captured image in a preset image library, and determines the structural difference information between the pre-captured image and the reference image. The server 200 sends the current scene type and structural difference information to the head-mounted display device 100. When the head-mounted display device 100 receives the current scene type, it controls the head-mounted display device to output corresponding style recommendation information based on the current scene type. The style recommendation information is used to prompt the wearer of the head-mounted display device to shoot a target style suitable for the current scene. When the head-mounted display device 100 receives the structural difference information, it outputs composition adjustment suggestions for shooting an image of the target style based on the structural difference information. In response to the wearer's shooting operation, the head-mounted display device 100 controls the shooting device to take a picture based on the shooting parameters corresponding to the target style to obtain the target image. In this embodiment, the server 200 runs a scene type recognition model to identify the scene type of the pre-captured image, and the server 200 performs image matching and determines structural difference information. This eliminates the need for the head-mounted display device 100 to store a preset image library, reducing the storage and computing power requirements of the head-mounted display device 100 in executing the shooting method provided in this embodiment of the invention, and thus has higher versatility and scalability.

[0024] The following will combine Figure 1 or Figure 2The scene described below illustrates the shooting method provided by the embodiments of the present invention. It should be noted that... Figure 1 or Figure 2 The scenarios described are only used to explain the shooting method provided in the embodiments of the present invention, but do not constitute a limitation on the application scenarios of the shooting method provided in the embodiments of the present invention.

[0025] Please see Figure 3 , Figure 3 This is a flowchart illustrating a shooting method provided in an embodiment of the present invention.

[0026] like Figure 1 As shown, the shooting method includes steps S101 to S104.

[0027] Step S101: Acquire a pre-captured image from the camera device of the head-mounted display device.

[0028] In this embodiment, the pre-captured image of the camera device of the head-mounted display device is the image corresponding to the real-time view of the camera device. The camera device can be mounted on the temples, nose pads, or frames of the head-mounted display device, etc., and this embodiment of the invention does not specifically limit its placement.

[0029] In some embodiments, acquiring a pre-captured image of the camera device of the head-mounted display device includes: acquiring a pre-captured image of the camera device of the head-mounted display device in response to the head-mounted display device being in smart photo mode. The head-mounted display device includes a mode switch button for turning the smart photo mode on or off. When the mode switch button is in the on state, the head-mounted display device is in smart photo mode; when the mode switch button is in the off state, the head-mounted display device is not in smart photo mode. The user can control the head-mounted display device to enter or exit smart photo mode by switching the state of the mode switch button.

[0030] In some embodiments, the shooting method provided by the present invention further includes: the head-mounted display device receiving an activation command for a smart photography mode sent by a mobile terminal, and responding to the activation command to control the head-mounted display device to enter the smart photography mode. The mobile terminal includes an application (APP) for controlling the head-mounted display device. This application includes a mode switch button for turning the smart photography mode on or off. When the mode switch button is in the on state, the head-mounted display device is in the smart photography mode; when the mode switch button is in the off state, the head-mounted display device is not in the smart photography mode. The user can control the head-mounted display device to enter or exit the smart photography mode by switching the state of the mode switch button.

[0031] Step S102: Use a preset scene type recognition model to identify the scene type of the pre-captured image to obtain the current scene type.

[0032] In this embodiment, the scene type recognition model is obtained by pre-training a machine learning model based on multiple training samples. The training samples include sample images and labeled scene types. The machine learning model may include a neural network model and a random forest classification model. Specifically, the process of identifying the scene type of a pre-captured image using the preset scene type recognition model to obtain the current scene type may include: extracting the image feature vector of the pre-captured image, inputting the image feature vector into the preset scene type recognition model for scene type recognition, and thus obtaining the current scene type.

[0033] In some embodiments, the preset scene type recognition model is deployed on the head-mounted display device or on a server. Specifically, when the preset scene type recognition model is deployed on the head-mounted display device, the head-mounted display device can use the preset scene type recognition model to perform scene type recognition on the pre-captured image to obtain the current scene type; when the preset scene type recognition model is deployed on the server, the head-mounted display device sends the pre-captured image to the server, and the server uses the preset scene type recognition model to perform scene type recognition on the pre-captured image to obtain the current scene type, and then sends the current scene type to the head-mounted display device.

[0034] Step S103: Based on the current scene type, control the head-mounted display device to output corresponding style recommendation information. The style recommendation information is used to suggest the target style suitable for the current scene to the wearer of the head-mounted display device.

[0035] In this embodiment, based on the current scene type, a pre-stored relationship table between scene types and style identifiers can be queried to obtain the style identifier corresponding to the current scene type. Then, the head-mounted display device is controlled to output style recommendation information bound to the style identifier. The image style may include HDR enhancement, transparency style, shadow enhancement, and cinematic feel, etc.

[0036] In some embodiments, controlling the head-mounted display device to output corresponding style recommendation information may include at least one of the following: controlling the head-mounted display device to display a style indicator of the target style, the style indicator being used to indicate the target style recommended to the wearer; controlling the head-mounted display device to display style recommendation text, the style recommendation text including the text corresponding to the style identifier of the target style recommended to the wearer; controlling the head-mounted display device to broadcast style recommendation voice, the style recommendation voice including the voice corresponding to the style identifier of the target style recommended to the wearer.

[0037] Step S104: In response to the wearer's photo-taking operation, control the shooting device to take a photo based on the shooting parameters corresponding to the target style to obtain the target image.

[0038] In this embodiment, different styles correspond to different shooting parameters. For example, the image style may include High Dynamic Range Imaging (HDR) enhancement, transparent style, shadow enhancement, and cinematic style, etc., and the shooting parameters corresponding to HDR enhancement, transparent style, shadow enhancement, and cinematic style are all different. The shooting parameters include at least one of the following: exposure time, aperture value, ISO sensitivity, shutter speed, white balance, and exposure compensation.

[0039] In some embodiments, in response to the wearer's photo-taking operation, the shooting device is controlled to take a photo based on shooting parameters corresponding to the target style to obtain a target image. This includes: when the brightness of the environment in which the head-mounted display device is currently located is greater than or equal to a preset brightness value, detecting the wearer's eye movement behavior through the eye-tracking detection device of the head-mounted display device; and in response to the eye movement behavior being a first preset eye movement behavior, controlling the shooting device to take a photo based on shooting parameters corresponding to the target style to obtain a target image. The brightness of the environment in which the head-mounted display device is currently located can be obtained by a brightness sensor in the head-mounted display device, and the preset brightness can be set based on actual conditions; this embodiment of the invention does not specifically limit this. In this embodiment, when the brightness of the environment in which the head-mounted display device is currently located is greater than or equal to the preset brightness value, the accuracy of the eye-tracking detection device is high. At this time, an eye-tracking control mechanism is introduced, enabling the wearer to control the photo-taking process using eye movements (without physical contact or voice commands), significantly improving interaction efficiency.

[0040] In some embodiments, the shooting method provided by the present invention further includes: in response to the eye movement behavior being a second preset eye movement behavior, switching the style indicator corresponding to the target style to the style indicator corresponding to the next style of the target style; or in response to the eye movement behavior being a third preset eye movement behavior, switching the style indicator corresponding to the target style to the style indicator corresponding to the previous style of the target style, wherein the first preset eye movement behavior, the second preset eye movement behavior, and the third preset eye movement behavior are different. This embodiment introduces an eye movement control mechanism, enabling the wearer to switch styles by eye movement (without physical contact or voice commands) when they do not want to use the recommended style for taking photos, significantly improving interaction efficiency.

[0041] It should be noted that the first, second, and third preset eye-tracking behaviors can be set based on actual conditions, and this embodiment of the invention does not impose specific limitations on them. For example, the first preset eye-tracking behavior includes blinking twice consecutively, the second preset eye-tracking behavior includes blinking once and looking to the right, and the third preset eye-tracking behavior includes blinking once and looking to the left.

[0042] In some embodiments, in response to the wearer's photo-taking operation, the shooting device is controlled to take a photo based on shooting parameters corresponding to the target style to obtain a target image. This includes: when the brightness of the environment in which the head-mounted display device is currently located is less than a preset brightness value, in response to the wearer's first operation on a preset physical button, the shooting device is controlled to take a photo based on shooting parameters corresponding to the target style to obtain a target image. In this embodiment, when the brightness of the environment in which the head-mounted display device is currently located is less than the preset brightness value, the accuracy of the eye-tracking detection device is low. At this time, the wearer cannot control the photo-taking by eye movement, but can control the photo-taking by the preset physical button, avoiding the situation where the photo-taking cannot be controlled and improving the user experience.

[0043] In some embodiments, the shooting method provided by the present invention further includes: in response to a second operation by the wearer on a preset physical button, switching the style indicator corresponding to the target style to the style indicator corresponding to the next style of the target style; or in response to a third operation by the wearer on a preset physical button, switching the style indicator corresponding to the target style to the style indicator corresponding to the previous style of the target style, wherein the first, second, and third operations are different. In this embodiment, when the brightness of the environment in which the head-mounted display device is currently located is less than a preset brightness value, the accuracy of the eye-tracking detection device is low. At this time, the wearer cannot switch styles by eye movement, but can switch styles through the preset physical button, thus avoiding the situation where style switching is not possible and improving the user experience.

[0044] It should be noted that the first, second, and third operations can be set based on actual circumstances, and this embodiment of the invention does not impose specific limitations on them. For example, the first operation includes a double-click operation, the second operation includes a single-click forward swipe operation, and the third operation includes a single-click backward swipe operation.

[0045] In some embodiments, such as Figure 4 As shown, step S104 includes sub-steps S1041 to S1042.

[0046] Sub-step S1041: Control the shooting device to take pictures based on multiple shooting parameters corresponding to the target style to obtain multiple frames of images, wherein the multiple shooting parameters are different, and one shooting parameter corresponds to one frame of image.

[0047] In this embodiment, the multiple frames are continuous, and the shooting parameters corresponding to the same style are different, as are the shooting parameters corresponding to different styles. For example, the image style may include HDR enhancement, transparency style, shadow enhancement, and cinematic style, etc. Then, the shooting parameters corresponding to HDR enhancement are different, the shooting parameters corresponding to transparency style are different, the shooting parameters corresponding to shadow enhancement are different, the shooting parameters corresponding to cinematic style are different, and the shooting parameters corresponding to HDR enhancement, transparency style, shadow enhancement, and cinematic style are all different.

[0048] Sub-step S1042: Determine the target weighting coefficient for each frame of the image, and perform weighted fusion of multiple frames of the image based on the target weighting coefficient for each frame of the image to obtain the target image.

[0049] This embodiment takes multiple images based on multiple shooting parameters corresponding to the target style. The multiple images are then weighted and fused according to the target weighting coefficient of each image, thereby improving the contrast, detail, and style consistency of the images. This ensures that the style of the fused target image not only meets the target style but also guarantees the quality of the target image, achieving a balance between image quality and personalized style.

[0050] The embodiments of the present invention form a complete closed loop of intelligent image processing, from scene recognition (step S102), style recommendation (step S103) to multi-frame acquisition (sub-step S1041) and stylized fusion output (sub-step S1042), which is suitable for image capture tasks of head-mounted display devices under various lighting scenarios.

[0051] In some embodiments, weighted fusion of multiple frames of images based on the target weighting coefficients of each frame to obtain a target image may include: summing the target weighting coefficients of each frame to obtain a total weighting coefficient; multiplying the target weighting coefficient of each frame by the corresponding image to obtain a multi-frame weighted image; summing the multi-frame weighted images and then dividing by the total weighting coefficient to obtain the target image. For example, the weighted fusion process of multiple frames of images can be represented by the following formula:

[0052] in, It is the target image. It is the i-th frame image in n frames. It is the target weighting coefficient of the i-th frame in n frames, where n is greater than or equal to 2.

[0053] In some embodiments, determining the target weighting coefficient for each frame image may include: determining a first weighting coefficient, a second weighting coefficient, and a third weighting coefficient for each frame image, wherein the first weighting coefficient is related to the brightness of the image, the second weighting coefficient is related to the saturation of the image, and the third weighting coefficient is related to the structural information of the image; determining a fourth weighting coefficient for each frame image under the target style based on the exposure type of each frame image; and multiplying the corresponding first weighting coefficient, second weighting coefficient, third weighting coefficient, and fourth weighting coefficient for each frame image to obtain the target weighting coefficient for each frame image.

[0054] The first weighting coefficient of the image can be expressed by the formula... =exp( The first weighting coefficient is determined to measure how close a pixel's brightness is to a mid-gray level, thus preventing overexposed or underexposed pixels from participating in the fusion process. It is the first weighting coefficient of the i-th frame in n frames. It is the average brightness of the i-th frame out of n frames. This is the brightness tolerance coefficient, an empirical value that can be set as needed, for example, to 0.2. The second weighting coefficient of the image can be calculated using the formula... =std([ The second weighting coefficient is determined to emphasize brightly colored areas in the image. It is the second weighting coefficient of the i-th frame in n frames, std([ The standard deviation of the RGB channels of the i-th frame out of n frames represents the color intensity. A larger standard deviation indicates more vibrant colors in the i-th frame. The third weighting coefficient of the image can be calculated using the formula... (x,y)= Sure, (x, y) represents the third weighting coefficient of the i-th frame in n frames. This third weighting coefficient is used to highlight regions with rich structural details in the image, and the Sobel operator is used to extract the image gradient. and These are the horizontal and vertical gradient responses calculated by the Sobel operator.

[0055] In some embodiments, determining the fourth weighting coefficient of each frame image under the target style based on the exposure type of each frame image includes: querying a weighting coefficient relationship table based on the exposure type of each frame image to obtain the fourth weighting coefficient of each frame image under the target style. The weighting coefficient relationship table includes each exposure type among multiple different exposure types under different styles and its corresponding weighting coefficient. The exposure types include long exposure, medium exposure, and short exposure. The exposure time range corresponding to short exposure is, for example, 1 / 500 second to 1 / 8000 second; the exposure time range corresponding to medium exposure is, for example, 1 / 30 second to 1 / 250 second; and the exposure time range corresponding to long exposure is, for example, 1 second to several hours.

[0056] The weighting coefficients for the same exposure type under different styles can be different or the same, and the weighting coefficients for different exposure types under the same style can be different. For example, if the image styles include HDR enhancement, transparent style, shadow enhancement, and cinematic style, then the relationship between the weighting coefficients for HDR enhancement, transparent style, shadow enhancement, and cinematic style can be shown in Table 1.

[0057] Table 1

[0058] It should be noted that the weighting coefficients shown in Table 1 are merely exemplary, and the embodiments of the present invention do not impose specific limitations on them.

[0059] In some embodiments, such as Figure 5 As shown, after step S101, the following steps are also included: Step S105: Determine a reference image that matches the pre-captured image in the preset image library, and determine the structural difference information between the pre-captured image and the reference image.

[0060] In this embodiment, a preset image library stores multiple reference images. Each reference image is a sample image with optimal composition and is pre-collected. Structural difference information between the pre-captured image and the reference images is used to describe the structural differences between them. Specifically, the structural similarity between each reference image in the preset image library and the pre-captured image can be calculated, and the reference image with the highest structural similarity is determined as the matching reference image with the pre-captured image.

[0061] Step S106: Based on the structural difference information, control the head-mounted display device to output composition adjustment suggestions for the target style image corresponding to the captured reference image.

[0062] This embodiment outputs composition adjustment suggestions for shooting images of the target style, allowing the wearer to adjust the composition according to the suggestions. This makes it easier for the wearer to shoot images with the target style, increases the success rate of shooting images of the target style, and further enhances the intelligence and convenience of head-mounted display devices for shooting images of various styles.

[0063] For example, composition adjustment suggestions could include "look up," "shift left," "shift right," "move closer to the target," and "move further away from the target." It should be noted that... Figure 5 The execution order of each step is merely exemplary. Steps S105 and S106 may be executed before or after steps S102 and / or 103, or simultaneously with steps S102 and / or 103. This embodiment of the invention does not impose any specific limitations on this.

[0064] Please see Figure 6 , Figure 6 This is a schematic block diagram of the structure of a head-mounted display device provided in an embodiment of the present invention.

[0065] like Figure 6 As shown, the head-mounted display device 100 includes a processor 101 and a memory 102, which are connected via a bus 103, such as an I2C (Inter-integrated Circuit) bus.

[0066] Specifically, processor 101 provides computing and control capabilities to support the operation of the entire head-mounted display device. Processor 101 can be a Central Processing Unit (CPU), but it can also be other general-purpose processors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. The general-purpose processor can be a microprocessor or any conventional processor.

[0067] Specifically, the memory 102 can be a Flash chip, a read-only memory (ROM) disk, an optical disk, a USB flash drive, or a portable hard drive, etc.

[0068] Those skilled in the art will understand that Figure 6 The structures shown are merely block diagrams of some structures related to the embodiments of the present invention, and do not constitute a limitation on the head-mounted display devices on which the embodiments of the present invention are applied. Specific head-mounted display devices may include more or fewer components than those shown in the figures, or combine certain components, or have different component arrangements.

[0069] The processor 101 is used to run a computer program stored in the memory 102, and implements any of the shooting methods provided in the embodiments of the present invention when executing the computer program.

[0070] In one embodiment, the processor 101 is configured to run a computer program stored in a memory, and when executing the computer program, to perform the following steps: Acquire pre-captured images from the camera device of the head-mounted display device; The scene type is identified by a preset scene type recognition model to obtain the current scene type; Based on the current scene type, the head-mounted display device is controlled to output corresponding style recommendation information, which is used to suggest a target style suitable for the current scene to the wearer of the head-mounted display device. In response to the wearer's photo-taking operation, the camera is controlled to take a photo based on the shooting parameters corresponding to the target style to obtain the target image.

[0071] In one embodiment, after acquiring the pre-captured image from the camera device of the head-mounted display device, the processor 101 is further configured to: A reference image matching the pre-captured image is determined from a preset image library, and structural difference information between the pre-captured image and the reference image is determined. Based on the structural difference information, the head-mounted display device is controlled to output composition adjustment suggestions for capturing images of the target style corresponding to the reference image.

[0072] In one embodiment, when the processor 101 controls the shooting device to take a picture based on shooting parameters corresponding to the target style in response to the wearer's photo-taking operation, and obtains the target image, it is configured to: When the brightness of the environment in which the head-mounted display device is currently located is greater than or equal to a preset brightness value, the eye movement behavior of the wearer is detected by the eye movement detection device of the head-mounted display device; In response to the eye movement behavior being a first preset eye movement behavior, the camera is controlled to take a picture based on the shooting parameters corresponding to the target style to obtain the target image.

[0073] In one embodiment, the processor 101 is further configured to implement: In response to the eye movement behavior being a second preset eye movement behavior, the style indicator corresponding to the displayed target style is switched to the style indicator corresponding to the next style of the target style; or In response to the eye movement behavior being the third preset eye movement behavior, the style indicator corresponding to the target style will be switched to the style indicator corresponding to the previous style of the target style. The first preset eye movement behavior, the second preset eye movement behavior, and the third preset eye movement behavior are different.

[0074] In one embodiment, when the processor 101 controls the shooting device to take a picture based on shooting parameters corresponding to the target style in response to the wearer's photo-taking operation, and obtains the target image, it is configured to: When the brightness of the environment in which the head-mounted display device is currently located is less than a preset brightness value, in response to the wearer's first operation on the preset physical button, the camera is controlled to take a picture based on the shooting parameters corresponding to the target style to obtain the target image.

[0075] In one embodiment, the processor 101 is further configured to implement: In response to the wearer's second operation on the preset physical button, the displayed style indicator corresponding to the target style will be switched to the style indicator corresponding to the next style of the target style; or In response to the wearer's third operation on the preset physical button, the style indicator corresponding to the target style will be switched to the style indicator corresponding to the previous style of the target style. The first operation, the second operation, and the third operation are different.

[0076] In one embodiment, when the processor 101 controls the shooting device to take a picture based on the shooting parameters corresponding to the target style to obtain the target image, it is configured to: The shooting device is controlled to take pictures based on multiple shooting parameters corresponding to the target style to obtain multiple frames of images, wherein the multiple shooting parameters are different, and one shooting parameter corresponds to one frame of the image; Determine the target weighting coefficient for each frame of the image, and perform weighted fusion of multiple frames of the image based on the target weighting coefficient for each frame of the image to obtain the target image.

[0077] In one embodiment, the processor 101, when determining the target weighting coefficients for each frame of the image, performs the following: A first weighting coefficient, a second weighting coefficient, and a third weighting coefficient are determined for each frame of the image. The first weighting coefficient is related to the brightness of the image, the second weighting coefficient is related to the saturation of the image, and the third weighting coefficient is related to the structural information of the image. Based on the exposure type of each frame of the image, determine the fourth weighting coefficient of each frame of the image under the target style; For each frame of the image, the corresponding first weighting coefficient, second weighting coefficient, third weighting coefficient, and fourth weighting coefficient are multiplied together to obtain the target weighting coefficient for each frame of the image.

[0078] It should be noted that those skilled in the art will understand that, for the sake of convenience and brevity, the specific working process of the head-mounted display device described above can be referred to the corresponding process in the aforementioned shooting method embodiments, and will not be repeated here.

[0079] This invention also provides a storage medium for computer-readable storage, wherein the storage medium stores one or more programs that can be executed by one or more processors to implement any of the shooting methods provided in the specification of this invention.

[0080] The storage medium can be volatile or non-volatile. It can be an internal storage unit of the head-mounted display device described in the foregoing embodiments, such as the hard drive or memory of the head-mounted display device. Alternatively, it can be an external storage device of the head-mounted display device, such as a plug-in hard drive, smart media card (SMC), secure digital card (SD), flash card, etc., provided on the head-mounted display device.

[0081] Those skilled in the art will understand that all or some of the steps, systems, or apparatuses disclosed above, and their functional modules / units, can be implemented as software, firmware, hardware, or suitable combinations thereof. In hardware embodiments, the division between functional modules / units mentioned in the above description does not necessarily correspond to the division of physical components; for example, a physical component may have multiple functions, or a function or step may be performed collaboratively by several physical components. Some or all physical components may be implemented as software executed by a processor, such as a central processing unit, digital signal processor, or microprocessor, or as hardware, or as an integrated circuit, such as an application-specific integrated circuit. Such software may be distributed on a computer-readable medium, which may include computer storage media (or non-transitory media) and communication media (or transient media). As is known to those skilled in the art, the term computer storage media includes volatile and non-volatile, removable and non-removable media implemented in any method or technology for storing information (such as computer-readable instructions, data structures, program modules, or other data). Computer storage media include, but are not limited to, RAM, ROM, EEPROM, flash memory or other memory technologies, CD-ROM, digital versatile disc (DVD) or other optical disc storage, magnetic cartridges, magnetic tape, disk storage or other magnetic storage devices, or any other medium that can be used to store desired information and can be accessed by a computer. Furthermore, it is well known to those skilled in the art that communication media typically contain computer-readable instructions, data structures, program modules, or other data in modulated data signals such as carrier waves or other transmission mechanisms, and may include any information delivery medium.

[0082] It should be understood that the term "and / or" as used in this specification and the appended claims refers to any combination and all possible combinations of one or more of the associated listed items, and includes such combinations. It should be noted that, herein, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or system that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or system. Without further limitation, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or system that includes that element.

[0083] The sequence numbers of the above embodiments of the present invention are merely for descriptive purposes and do not represent the superiority or inferiority of the embodiments. The above descriptions are only specific embodiments of the present invention, but the scope of protection of the present invention is not limited thereto. Any person skilled in the art can easily conceive of various equivalent modifications or substitutions within the technical scope disclosed in the present invention, and these modifications or substitutions should all be covered within the scope of protection of the present invention. Therefore, the scope of protection of the present invention should be determined by the scope of the claims.

Claims

1. A shooting method, characterized in that, include: Acquire pre-captured images from the camera device of the head-mounted display device; The scene type is identified by a preset scene type recognition model to obtain the current scene type; Based on the current scene type, the head-mounted display device is controlled to output corresponding style recommendation information, which is used to suggest a target style suitable for the current scene to the wearer of the head-mounted display device. In response to the wearer's photo-taking operation, the camera is controlled to take a photo based on the shooting parameters corresponding to the target style to obtain the target image.

2. The shooting method according to claim 1, characterized in that, After acquiring the pre-captured image from the camera device of the head-mounted display device, the method further includes: A reference image matching the pre-captured image is determined from a preset image library, and structural difference information between the pre-captured image and the reference image is determined. Based on the structural difference information, the head-mounted display device is controlled to output composition adjustment suggestions for capturing images of the target style corresponding to the reference image.

3. The shooting method according to claim 1, characterized in that, The step of responding to the wearer's photo-taking operation by controlling the shooting device to take a photo based on the shooting parameters corresponding to the target style to obtain a target image includes: When the brightness of the environment in which the head-mounted display device is currently located is greater than or equal to a preset brightness value, the eye movement behavior of the wearer is detected by the eye movement detection device of the head-mounted display device; In response to the eye movement behavior being a first preset eye movement behavior, the camera is controlled to take a picture based on the shooting parameters corresponding to the target style to obtain the target image.

4. The shooting method according to claim 3, characterized in that, The method further includes: In response to the eye movement behavior being a second preset eye movement behavior, the style indicator corresponding to the displayed target style is switched to the style indicator corresponding to the next style of the target style; or In response to the eye movement behavior being the third preset eye movement behavior, the style indicator corresponding to the target style will be switched to the style indicator corresponding to the previous style of the target style. The first preset eye movement behavior, the second preset eye movement behavior, and the third preset eye movement behavior are different.

5. The shooting method according to claim 1, characterized in that, The step of responding to the wearer's photo-taking operation by controlling the shooting device to take a photo based on the shooting parameters corresponding to the target style to obtain a target image includes: When the brightness of the environment in which the head-mounted display device is currently located is less than a preset brightness value, in response to the wearer's first operation on the preset physical button, the camera is controlled to take a picture based on the shooting parameters corresponding to the target style to obtain the target image.

6. The shooting method according to claim 5, characterized in that, The method further includes: In response to the wearer's second operation on the preset physical button, the displayed style indicator corresponding to the target style will be switched to the style indicator corresponding to the next style of the target style; or In response to the wearer's third operation on the preset physical button, the style indicator corresponding to the target style will be switched to the style indicator corresponding to the previous style of the target style. The first operation, the second operation, and the third operation are different.

7. The shooting method according to any one of claims 1-6, characterized in that, The process of controlling the shooting device to take pictures based on the shooting parameters corresponding to the target style to obtain the target image includes: The shooting device is controlled to take pictures based on multiple shooting parameters corresponding to the target style to obtain multiple frames of images, wherein the multiple shooting parameters are different, and one shooting parameter corresponds to one frame of the image; Determine the target weighting coefficient for each frame of the image, and perform weighted fusion of multiple frames of the image based on the target weighting coefficient for each frame of the image to obtain the target image.

8. The shooting method according to claim 7, characterized in that, Determining the target weighting coefficients for each frame of the image includes: A first weighting coefficient, a second weighting coefficient, and a third weighting coefficient are determined for each frame of the image. The first weighting coefficient is related to the brightness of the image, the second weighting coefficient is related to the saturation of the image, and the third weighting coefficient is related to the structural information of the image. Based on the exposure type of each frame of the image, determine the fourth weighting coefficient of each frame of the image under the target style; For each frame of the image, the corresponding first weighting coefficient, second weighting coefficient, third weighting coefficient, and fourth weighting coefficient are multiplied together to obtain the target weighting coefficient for each frame of the image.

9. A head-mounted display device, characterized in that, The head-mounted display device includes a processor, a memory, a computer program stored in the memory and executable by the processor, and a data bus for enabling communication between the processor and the memory, wherein when the computer program is executed by the processor, it implements the steps of the shooting method as described in any one of claims 1 to 8.

10. A storage medium for computer-readable storage, characterized in that, The storage medium stores one or more programs, which can be executed by one or more processors to implement the steps of the shooting method according to any one of claims 1 to 8.

Images