Image shooting method and device, electronic equipment, storage medium and program product

Abstract

The invention discloses an image shooting method and device, electronic equipment, a storage medium and a program product, and belongs to the technical field of image.The method comprises the steps that N face images in a shooting preview interface are obtained, and N is an integer larger than 1; inputting the N face images into a first model, and obtaining texture feature information of the N face images; the textural feature information of the N face images is subjected to chemical processing through a first model, N first probability values are output, and each first probability value represents the probability that one face image is the face image of a real object; and focusing the face image in the shooting preview interface based on the N first probability values.

Description

Technical Field

[0001] The present application belongs to the field of image technology, and specifically relates to an image shooting method, device, electronic equipment, storage medium and program product. Background Art

[0002] With the development of shooting functions of electronic devices, users can shoot through electronic devices at any time to record their lives through the captured images.

[0003] Typically, during the shooting process, when a plurality of facial images are included in the shooting preview interface, the electronic device can identify each facial image based on a face recognition algorithm, and then can focus based on these facial images, thereby completing the shooting according to the user's input.

[0004] However, during the shooting preview process, if the collected objects include objects existing in the real environment and objects detected from certain pictures (such as posters), the electronic device may focus on the objects in the picture during the focusing process. As a result, the image of the object in the real environment is blurred because the object focused by the electronic device does not meet the user's needs, thus resulting in poor quality of the image taken by the electronic device. Summary of the invention

[0005] The purpose of the embodiments of the present application is to provide an image shooting method, device, electronic device, storage medium and program product, which can improve the quality of images shot by the electronic device.

[0006] In a first aspect, an embodiment of the present application provides an image shooting method, which includes: obtaining N facial images in a shooting preview interface, where N is an integer greater than 1; inputting the N facial images into a first model to obtain texture feature information of the N facial images; processing the texture feature information of the N facial images through the first model, and outputting N first probability values, each first probability value representing the probability that a facial image is a facial image of a real object; and focusing on the facial image in the shooting preview interface based on the N first probability values.

[0007] In a second aspect, an embodiment of the present application provides an image capturing device, which includes: an acquisition module, a processing module and an execution module; the acquisition module is used to acquire N facial images in a shooting preview interface, where N is an integer greater than 1; the processing module is used to input the N facial images acquired by the acquisition module into a first model to acquire texture feature information of the N facial images; and through the first model, the texture feature information of the N facial images is processed to output N first probability values, each first probability value representing the probability that a facial image is a facial image of a real object; the execution module is used to focus on the facial image in the shooting preview interface based on the N first probability values ​​output by the processing module.

[0008] In a third aspect, an embodiment of the present application provides an electronic device, which includes a processor and a memory, wherein the memory stores programs or instructions that can be run on the processor, and when the program or instructions are executed by the processor, the steps of the method described in the first aspect are implemented.

[0009] In a fourth aspect, an embodiment of the present application provides a readable storage medium, on which a program or instruction is stored, and when the program or instruction is executed by a processor, the steps of the method described in the first aspect are implemented.

[0010] In a fifth aspect, an embodiment of the present application provides a chip, comprising a processor and a communication interface, wherein the communication interface is coupled to the processor, and the processor is used to run a program or instruction to implement the method described in the first aspect.

[0011] In a sixth aspect, an embodiment of the present application provides a computer program / program product, which is stored in a storage medium and is executed by at least one processor to implement the method described in the first aspect.

[0012] In an embodiment of the present application, the electronic device can obtain N facial images in the shooting preview interface, and input the N facial images into the first model to obtain the texture feature information of the N facial images. Then, the electronic device can normalize the texture feature information of the N facial images through the first model, and output N first probability values, wherein each first probability value represents the probability that a facial image is a facial image of a real object, so that the electronic device can focus on the facial image in the shooting preview interface based on the N first probability values. In this scheme, since the first model can provide the texture feature information of the facial image based on the input facial image, the electronic device can determine the probability value of each facial image being a facial image of a real object based on the texture feature information of the N facial images, and further, the electronic device can determine which of the N facial images are facial images of a real object according to the output probability value, and then focus the facial image based on the facial image of the real object. In this way, the electronic device can focus on the face of the real object more conveniently, thereby improving the quality of the image captured by the electronic device. BRIEF DESCRIPTION OF THE DRAWINGS

[0013] Figure 1 This is one of the flow charts of an image capturing method provided in an embodiment of the present application;

[0014] Figure 2A This is one of the example schematic diagrams of a facial image frame provided in an embodiment of the present application;

[0015] Figure 2B This is a second example schematic diagram of a facial image frame provided in an embodiment of the present application;

[0016] Figure 3A This is one of the example schematic diagrams of a shooting preview interface provided in an embodiment of the present application;

[0017] Figure 3B This is a second example schematic diagram of a shooting preview interface provided in an embodiment of the present application;

[0018] Figure 3C is a schematic diagram of an example of a facial image provided in an embodiment of the present application;

[0019] Figure 4 This is the second flowchart of an image capturing method provided in an embodiment of the present application;

[0020] Figure 5 It is one of the structural schematic diagrams of a first model provided in an embodiment of the present application;

[0021] Figure 6 This is the third flowchart of an image capturing method provided in an embodiment of the present application;

[0022] Figure 7 This is the second structural diagram of a first model provided in an embodiment of the present application;

[0023] Figure 8 This is a third structural diagram of a first model provided in an embodiment of the present application;

[0024] Fig. 9 This is a fourth flowchart of an image capturing method provided in an embodiment of the present application;

[0025] Fig.10 This is a fourth structural diagram of a first model provided in an embodiment of the present application;

[0026] Fig.11 is a flow chart of a model training method provided in an embodiment of the present application;

[0027] Fig.12 is a structural schematic diagram of a second model provided in an embodiment of the present application;

[0028] Fig.13 is a structural schematic diagram of an image detection device provided in an embodiment of the present application;

[0029] Fig.14 It is a structural schematic diagram of a model training device provided in an embodiment of the present application;

[0030] Fig.15 This is one of the hardware structure diagrams of an electronic device provided in an embodiment of the present application;

[0031] Fig.16 This is the second schematic diagram of the hardware structure of an electronic device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0032] The following will be combined with the drawings in the embodiments of the present application to clearly describe the technical solutions in the embodiments of the present application. Obviously, the described embodiments are part of the embodiments of the present application, rather than all the embodiments. All other embodiments obtained by ordinary technicians in this field based on the embodiments in the present application belong to the scope of protection of this application.

[0033] The terms "first", "second", etc. in the specification of the present application are used to distinguish similar objects, and are not used to describe a specific order or sequence. It should be understood that the data used in this way can be interchangeable under appropriate circumstances, so that the embodiments of the present application can be implemented in an order other than those illustrated or described here, and the objects distinguished by "first", "second", etc. are usually a class, and the number of objects is not limited. For example, the first object can be one or more. In addition, "and / or" in the specification represents at least one of the connected objects, and the character " / " generally indicates that the objects associated with each other are in an "or" relationship.

[0034] The terms "at least one (item)", "at least one of" and the like in the specification of the present application refer to any one, any two or a combination of more than two of the objects included therein. For example, at least one (item) of a, b, and c can be represented by: "a", "b", "c", "a and b", "a and c", "b and c" and "a, b and c", where a, b, and c can be single or multiple. Similarly, "at least two (items)" refers to two or more, and its meaning is similar to that of "at least one (item)".

[0035] The image capturing method provided by the embodiment of the present application is described in detail below through specific embodiments and their application scenarios in conjunction with the accompanying drawings.

[0036] The image shooting method provided in the embodiment of the present application can be applied to a scene containing multiple facial images in the shooting preview interface, such as facial images of real objects and facial images in pictures.

[0037] The image capturing method provided in the embodiments of the present application is exemplarily described below using some specific scenarios as examples.

[0038] Scenario 1: The shooting preview interface includes a person and a poster, and the poster includes a facial image of the person.

[0039] In the process of shooting through an electronic device, if the shooting preview interface includes a person 1 and a poster, and the poster includes a facial image of a person 2, the electronic device can obtain the facial images included in the shooting preview interface to input into an image shooting model, obtain the texture feature information of each facial image, and can process the texture feature information of each facial image separately through the image shooting model, and input a probability value corresponding to each facial image, so that the electronic device can determine which facial image is a facial image of a real object, that is, the facial image of the above-mentioned person 1, and which facial image is not a facial image of a real object, that is, the facial image of the above-mentioned person 2 included in the poster, among the facial images included in the shooting preview interface according to the probability value corresponding to each facial image output by the image shooting model. Therefore, the electronic device can determine which facial image in the shooting preview interface to focus on according to the probability value.

[0040] Scenario 2: The shooting preview interface includes people, animals and posters, and the poster includes a facial image of the person.

[0041] In the process of shooting through an electronic device, if the shooting preview interface includes a person 1, an animal 1 and a poster, and the poster includes a facial image of a person 2, the electronic device can obtain the facial images included in the shooting preview interface to input them into an image shooting model to obtain the texture feature information of each facial image, and can process the texture feature information of each facial image separately through the image shooting model, and input a probability value corresponding to each facial image, so that the electronic device can determine which facial image is a facial image of a real object, i.e., the facial image of the above-mentioned person 1 or the facial image of the above-mentioned animal 1, and which facial image is not a facial image of a real object, i.e., the facial image of the above-mentioned person 1 or the facial image of the above-mentioned animal 1, among the facial images included in the shooting preview interface, according to the probability value corresponding to each facial image output by the image shooting model, and which facial image is not a facial image of a real object, i.e., the facial image of the above-mentioned person 2 included in the poster. Therefore, the electronic device can determine which facial image in the shooting preview interface to focus on according to the probability value.

[0042] It should be noted that the above-mentioned scenarios 1 and 2 are merely illustrative examples of some scenarios in which the embodiments of the present application may be applied. In actual implementation, the embodiments of the present application can also be applied to more possible scenarios such as photographing at least one of people, animals, people in pictures, and animals in pictures through electronic devices, and the embodiments of the present application are not limited here.

[0043] Based on the above scenario applied by the embodiment of the present application, the image shooting method provided by the embodiment of the present application, the electronic device can obtain N facial images in the shooting preview interface, and input the N facial images into the first model to obtain the texture feature information of the N facial images. Then, the electronic device can normalize the texture feature information of the N facial images through the first model, and output N first probability values, wherein each first probability value represents the probability that a facial image is a facial image of a real object, so that the electronic device can focus on the facial image in the shooting preview interface based on the N first probability values. In this scheme, since the first model can provide the texture feature information of the facial image based on the input facial image, the electronic device can determine the probability value of each facial image being a facial image of a real object based on the texture feature information of the N facial images, and further, the electronic device can determine which of the N facial images are facial images of the real object according to the output probability value, and then focus the facial image based on the facial image of the real object. In this way, the electronic device can focus on the face of the real object more conveniently, thereby improving the quality of the image shot by the electronic device.

[0044] The image capturing method provided in the embodiment of the present application is executed by an image capturing device, which may be an electronic device, or a functional module or entity in an electronic device, and the embodiment of the present application does not limit this. The image capturing method provided in the embodiment of the present application will be exemplarily described below using an electronic device as an example.

[0045] The present application embodiment provides an image capturing method. Figure 1 FIG. 1 is a flow chart of an image capturing method provided by an embodiment of the present application. Figure 1 As shown, the image capturing method provided in the embodiment of the present application may include the following steps 201 to 204.

[0046] Step 201: The electronic device obtains N facial images in a shooting preview interface.

[0047] In the embodiment of the present application, N is an integer greater than 1.

[0048] In some embodiments of the present application, the N facial images mentioned above may include but are not limited to at least one of the following: a facial image of a person, a facial image of an animal, or a facial image of a person or an animal contained in a picture.

[0049] In some embodiments of the present application, the above-mentioned pictures may include but are not limited to at least one of the following: photos, posters, and video screens.

[0050] In some embodiments of the present application, the electronic device may first obtain a preview image displayed in a shooting preview interface, and then obtain N facial images contained in the preview image through a face detection module.

[0051] In some embodiments of the present application, when the N facial images include facial images of people, the facial detection module may be a face location (FD) module.

[0052] In some embodiments of the present application, the preview image acquired by the electronic device may include, but is not limited to, any of the following: an image in RGB format, an image in YUV format.

[0053] It should be noted that an image in RGB format uses the values ​​of three color channels: red (Red, R), green (Green, G), and blue (Blue, B) to represent the color of the pixels in the image.

[0054] The YUV format image uses brightness and chromaticity to represent the color of the pixels in the image, where Y represents brightness (Luminance or Luma), that is, the grayscale value, and U and V represent chromaticity (Chrominance) and concentration (Chroma) to represent the color and saturation of the image, where U represents the blue chromaticity component and V represents the red chromaticity component;

[0055] For the same image, data in RGB format and data in YUV format can be converted to each other.

[0056] In some embodiments of the present application, when the face detection model obtains a preview image, the face detection model can first determine whether the preview image contains a facial image, and then after determining that the facial image is contained, the face detection module can further determine the position and size of the facial image contained in the preview image to obtain N facial images.

[0057] In some embodiments of the present application, after the electronic device detects N facial images, the electronic device may display N image frames in the shooting preview interface, wherein each of the N image frames may be used to identify a facial image.

[0058] In some embodiments of the present application, the shape of the image frame may be a square or a rectangle.

[0059] In some embodiments of the present application, the electronic device may first display N minimum image frames based on the detected N facial images to indicate the positions of the N facial images, and then, the electronic device may adaptively adjust the display size of the displayed minimum image frame so that the adjusted image frame can contain as much facial or head image information as possible and as little background image information unrelated to the facial image as possible.

[0060] In some embodiments of the present application, the above-mentioned minimum image frame may be an image frame with the smallest size that can represent the facial information of the object, and the facial minimum image frame may include facial image information of the eyes, nose and mouth of the object to be photographed.

[0061] In some implementations of the present application, the adjusted image frame may include facial or head image information such as eyebrows, ears, forehead, chin, etc. of the subject to be photographed, based on the facial image information contained in the smallest image frame.

[0062] In some embodiments of the present application, the display size of the above-adjusted image frame may be related to the display size of the corresponding minimum image frame. For example, the width of the adjusted image frame may be 1.2 times the width of the corresponding minimum image frame, and the height of the adjusted image frame may be 1.3 times the height of the corresponding minimum image frame.

[0063] It should be noted that the above-mentioned 1.2 times and 1.3 times are only used for examples, and the specific multiple relationship can be determined according to the image that needs to be processed, and the embodiment of the present application is not limited here.

[0064] In some embodiments of the present application, the display position of the above-mentioned adjusted image frame may be related to the display position of its corresponding minimum image frame.

[0065] For example, Figure 2A As shown, the image frame 10 may include images of the eyes, nose and mouth of the subject to be photographed, and the image frame 11 may include images of eyebrows, ears, forehead and chin based on the image frame 10. The image frame 11 may be extended in the upward and downward directions based on the image frame 10; the image frame 11 may be extended in the left and right directions based on the image frame 10.

[0066] For example, Figure 2BAs shown, the image frame 12 may include images of the eyes, nose and mouth of the subject to be photographed, and the image frame 13 may include an image of the face or head hair on the basis of the image frame 12. The bottom position of the image frame 13 may be the same as the bottom position of the image frame 12, that is, the image frame 13 may be extended upward on the basis of the image frame 12; the image frame 13 may be extended in both left and right directions on the basis of the image frame 12.

[0067] In some embodiments of the present application, the electronic device may obtain N facial images based on the image frame displayed in the above-mentioned shooting preview interface.

[0068] For example, the electronic device is a mobile phone. Figure 3A As shown, the shooting preview interface 20 displays a female teacher, a girl, a boy and a photo 21 containing an ancient figure. The mobile phone can obtain the preview image displayed in the shooting preview interface 21, and then through the FD module, after determining that the preview image contains a face image, as shown in FIG. Figure 3B As shown, an image frame is displayed at each face position of the person included in the preview image. Further, the mobile phone can crop and scale the preview image according to the four image frames displayed in the shooting preview interface 20 to obtain four face images, that is, Figure 3C The facial image shown is 1 corresponding to the girl, 2 corresponding to the boy, 3 corresponding to the female teacher and 4 corresponding to the ancient figure in the photo 21.

[0069] Step 202: The electronic device inputs N facial images into a first model to obtain texture feature information of the N facial images.

[0070] In the embodiment of the present application, the first model is trained based on texture feature information of facial images.

[0071] In some embodiments of the present application, the above-mentioned texture feature information may be the texture composed of fine structures such as skin and hair in the facial image.

[0072] In some embodiments of the present application, in the process of acquiring texture feature information through the first model, the first model may first perform image preprocessing on the acquired facial image, and then extract the texture feature information of the facial image.

[0073] In some embodiments of the present application, the above-mentioned image preprocessing may include but is not limited to performing one of the following: image denoising, image grayscale, histogram equalization, etc.

[0074] In some embodiments of the present application, since the facial orientations in the facial images may be different, before acquiring the texture feature information, the electronic device may first adjust each of the N facial images to a unified standard posture based on multiple key points of the face.

[0075] In some embodiments of the application, the key points of the face may include but are not limited to at least one of the following: eyebrows, eyes, nose, mouth, and ears.

[0076] In some embodiments of the present application, regarding the training process of the first model, please refer to the description of subsequent steps, and this application will not go into details here.

[0077] In some embodiments of the present application, the first model may be a classification network for living body detection.

[0078] In some embodiments of the present application, the first model may include a residual module and a dense-sparse decomposition module.

[0079] In some embodiments of the present application, Figure 1 ,like Figure 4 As shown, the above step 202 can be specifically implemented through the following steps 202a and 202b.

[0080] Step 202a: The electronic device performs downsampling processing on the N facial images through the residual module in the first model, and outputs first deep image feature information of the N facial images.

[0081] It should be noted that the above residual block (RSBlock) can solve the gradient vanishing and gradient exploding problems in deep neural network training by introducing skip connections or identity mapping.

[0082] In some embodiments of the present application, the above-mentioned residual module may include a convolution kernel of size T×T.

[0083] It should be noted that the value of T can be set according to actual needs, and the embodiment of the present application does not limit it. For example, the convolution kernel of the size of T×T can be a convolution kernel of the size of 3×3, or a convolution kernel of the size of 1×1.

[0084] In some embodiments of the present application, the electronic device may reduce the resolution of the facial image through down sampling processing.

[0085] For example, assuming that the resolution of a facial image is 256×256, the resolution of the facial image may be reduced to 128×128 after downsampling.

[0086] In this way, the electronic device reduces the size of the facial image through downsampling processing, thereby reducing the amount of model calculation and the number of parameters used in model calculation, while increasing the robustness and generalization ability of the model.

[0087] In some embodiments of the present application, the first model may include at least two residual modules to perform at least two downsampling processes on the facial image to obtain first deep image feature information of the facial image.

[0088] In some embodiments of the present application, the setting of the number of downsampling processes may be related to the size of the facial image. For example, the larger the size of the facial image, the more times the downsampling process needs to be performed on the facial image, and further, the more residual modules are set in the first model.

[0089] In some embodiments of the present application, the first deep layer image feature information may include abstract information, for example, information related to the semantic features of the image.

[0090] Step 202b: The electronic device performs redundant information removal processing on the first deep image feature information through the dense-sparse decomposition module in the first model, and outputs texture feature information of N facial images.

[0091] In some embodiments of the present application, the first model may include at least two dense-sparse decomposition modules (Dense-Sparse Block, DS Block).

[0092] In some embodiments of the present application, the DS Block may include a convolution layer consisting of a k×1 convolution kernel, a 1×k convolution kernel, and s 1×1 convolution kernels.

[0093] It should be noted that the above k×1 convolution kernel refers to a convolution kernel with a width of k and a height of 1. Since this convolution kernel has k weights in the horizontal direction and only one weight in the vertical direction, it is mainly used to extract features in the horizontal direction.

[0094] The above 1×k convolution kernel refers to a convolution kernel with a width of 1 and a height of k. Since this convolution kernel has k weights in the vertical direction and only one weight in the horizontal direction, it is mainly used to extract features in the vertical direction.

[0095] The s 1×1 convolution kernels mentioned above refer to s convolution kernels with width and height of 1. This convolution kernel is mainly used to change the number of channels of the input data, that is, the depth, without changing the spatial dimension of the data, that is, the width and height. The model can extract and combine features by stacking multiple 1×1 convolution kernels, and at the same time increase the nonlinearity of the network.

[0096] In some embodiments of the present application, the electronic device may first pass the first deep image feature information through a k×1 convolution kernel to extract features in the horizontal direction; then, the result output by the k×1 convolution kernel is input into a 1×k convolution kernel to extract features in the vertical direction. At this time, the two steps executed above may be equivalent to a depthwise separable convolution; then, the result output by the 1×k convolution kernel is input into s 1×1 convolution kernels to further extract and combine features, adjust the number of channels, and increase the nonlinearity of the network.

[0097] In this way, through the DS Block in the first model, feature maps of various sizes can be screened to retain important key feature information for subsequent liveness detection tasks, effectively removing redundant feature information, thereby improving the robustness and generalization of the model while reducing the amount of model calculation and accelerating the calculation.

[0098] In some embodiments of the present application, it is assumed that the first model includes x residual modules and y DS Blocks, wherein x and y are both integers greater than or equal to 2, and x is greater than or equal to y. When the electronic device inputs N facial images into the first model, each of the N facial images can first pass through xy residual modules to perform xy times of downsampling processing, and then the facial image after the downsampling processing can be passed through y combination modules composed of a residual module and a DS Block to perform corresponding processing, thereby outputting the first deep image feature information corresponding to each facial image.

[0099] It should be noted that if x=y, when the electronic device inputs N facial images into the first model, each of the N facial images can directly perform corresponding processing through y structures composed of a residual module and a DS Block, thereby outputting the first deep image feature information corresponding to each facial image.

[0100] In some embodiments of the present application, in the above-mentioned combination module composed of a residual module and a DS Block, the facial image can first be downsampled by the residual module in the combination module to obtain the original image feature information, and the original image feature information is copied and stored. Then, the original image feature information output by the residual module in the combination module can be input into the DS Block in the combination module to remove redundant image feature information in the original image feature information. Further, the first model can fuse the stored original image feature information and the image feature information after removing the redundant image feature information to output the texture feature information of the facial image, and input it into a subsequent detection module to execute the processing process corresponding to the subsequent detection module.

[0101] For example, in combination Figure 3C ,like Figure 5 As shown, it is assumed that the first model 30 includes 3 residual modules and 2 DSBlocks, wherein the 3 residual modules may be residual module 31, residual module 321 and residual module 331, wherein residual module 31 includes a convolution kernel of size n×n, residual module 321 includes a convolution kernel of size p×p, and residual module 331 includes a convolution kernel of size q×q; the 2 DS Blocks may be DS Block 322 and DS Block 332; residual module 321 and DSBlock 322 may be combined into a combined module 32, and residual module 331 and DS Block 332 may be combined into a combined module 33. After the mobile phone obtains four facial images and inputs the four facial images into the first model 30, the facial image 1 can be downsampled by the residual module 31 to output the image feature information a. Then, the first model 30 inputs the image feature information a into the residual module 321 in the combination module 32 for downsampling, outputs the image feature information b, and copies and stores it. Then, the first model 30 can input the image feature information b output by the residual module 321 into the DS Block 322 to remove the redundant image feature information in the image feature information b, and output the image feature information c. Further, the first model 30 can fuse the stored image feature information b with the image feature information c of the DS Block 322, that is, Figure 5 The step indicated by the identifier 323 is performed to output the image feature information d of the facial image.

[0102] Further, the first model can input the image feature information d into the residual module 331 in the combination module 33 for downsampling processing, output the image feature information e, and copy and store it; then, the first model 30 can input the image feature information e output by the residual module 331 into the DS Block 332 to remove the redundant image feature information in the image feature information e, and output the image feature information f. Further, the first model 30 can fuse the stored image feature information e with the image feature information f of the DS Block 322, that is, Figure 5 The step indicated by the mark 333 in the figure is used to output the image feature information g of the facial image, that is, the texture feature information of the facial image 1.

[0103] Similarly, facial image 2, facial image 3 and facial image 4 can perform the steps performed by the above-mentioned facial image 1, so that the mobile phone can obtain the texture feature information of facial image 1, the texture feature information of facial image 2, the texture feature information of facial image 3 and the texture feature information of facial image 4.

[0104] In some embodiments of the present application, the residual module and the dense-sparse decomposition module may be backbone network modules in a classification network for liveness detection.

[0105] In this way, the electronic device can obtain the first deep image feature information of the facial image through the residual module and the dense-sparse decomposition module in the first model, and remove the redundant information therein to output the texture feature information of the facial image, thereby improving the robustness of the first model and the accuracy of the output results based on the coordination between different modules.

[0106] Step 203: The electronic device processes the texture feature information of the N facial images through the first model and outputs N first probability values.

[0107] In some embodiments of the present application, the above-mentioned first model may include a global average pooling layer and a HEAD module.

[0108] In some embodiments of the present application, each of the N facial images includes texture feature information of multiple dimensions. Figure 1 ,like Figure 6 As shown, the above step 203 can be specifically implemented through the following steps 203a and 203b.

[0109] Step 203a: The electronic device fuses the texture feature information of multiple dimensions of each facial image through the global average pooling layer in the first model to obtain fused texture feature information of each facial image.

[0110] It should be noted that the above-mentioned global average pooling (GAP) layer can add and average all pixel values ​​contained in the feature map to obtain a numerical value, so that the corresponding feature map can be represented by the numerical value.

[0111] For example, in combination Figure 5 ,like Figure 7 As shown, the first model 30 also includes a GAP layer 34. The mobile phone can fuse the texture feature information of facial image 1, the texture feature information of facial image 2, the texture feature information of facial image 3 and the texture feature information of facial image 4 respectively to obtain a numerical value corresponding to facial image 1, facial image 2, facial image 3 and facial image 4 respectively, that is, the fused texture feature information of each of the above facial images.

[0112] Step 203b: The electronic device normalizes the fused texture feature information of each facial image through the HEAD module in the first model, and outputs N first probability values.

[0113] In some embodiments of the present application, the detection head (HEAD) module may utilize features extracted by the Backbone module in the first model to perform output for a specific task.

[0114] For example, in combination Figure 7 ,like Figure 8 As shown, the HEAD module 35 in the first model 30 can use the fused texture feature information of the facial image 1, the fused texture feature information of the facial image 2, the fused texture feature information of the facial image 3, and the fused texture feature information of the facial image 4 output by the GAP layer 34 to perform normalization processing respectively, so as to output the first probability values ​​corresponding to the facial image 1, the facial image 2, the facial image 3, and the facial image 4, respectively, such as the first probability value of 0.6 for the facial image 1, the first probability value of 0.6 for the facial image 2, the first probability value of 0.7 for the facial image 3, and the first probability value of 0.3 for the facial image 4.

[0115] In this way, the electronic device can output the probability value of whether the facial image is a facial image of a real object through the global average pooling layer and the HEAD module in the first model, thereby improving the robustness of the first model based on the coordination between different modules.

[0116] In some embodiments of the present application, the above-mentioned HEAD module may be composed of a fully connected layer (Fully Connected Layer, FC) and a normalized exponential (Softmax) function.

[0117] In some embodiments of the present application, Figure 6 ,like Fig. 9 As shown, the above step 203b can be specifically implemented through the following steps 203b1 and 203b2.

[0118] Step 203b1: The electronic device performs a weighted operation on the fused texture feature information of each facial image through the fully connected layer in the HEAD module, and outputs N logical value feature vectors.

[0119] Step 203b2: The electronic device transforms each logical value feature vector through the Softmax function in the HEAD module, and outputs N first probability values.

[0120] In some embodiments of the present application, the above-mentioned FC can be used to integrate the features extracted by the convolution layer or the pooling layer in the first model.

[0121] It should be noted that in classification tasks, the output dimension of FC can usually be equal to the number of categories.

[0122] In some embodiments of the present application, the above-mentioned logical value feature vector may be a logits feature, and the logits feature may be regarded as an unnormalized original prediction value output by the neural network.

[0123] In some embodiments of the present application, the Softmax function can convert the logits feature into a probability distribution, in which the probability value of each category is between 0 and 1, and the sum of the probability values ​​of all categories is 1.

[0124] In some embodiments of the present application, a probability array consisting of two probability values ​​may be first input into the Softmax function, wherein one of the two probability values ​​may represent the probability that a facial image is a facial image of a real object, and the other probability value may represent the probability that a facial image is not a facial image of a real object.

[0125] Exemplarily, the Softmax function may output a probability value p1 and a probability value p2, and p1+p2=1, wherein the probability value p1 may represent the probability that a facial image is a facial image of a real object, and the probability value p2 may represent the probability that a facial image is not a facial image of a real object.

[0126] For example, in combination Figure 8 ,like Fig.10As shown, after the GAP layer 34 outputs the fused texture feature information of the facial image 1, the fused texture feature information of the facial image 1 can be integrated through the FC 351 in the HEAD module 35 to output the logits feature of the facial image 1, and the logits feature of the facial image 1 is converted into a probability distribution, i.e. (0.6, 0.4), through the Softmax function 352, so that the first model 30 can output the first probability value 0.6 of the facial image 1. Similarly, the HEAD module 35 can obtain the probability distribution (0.6, 0.4) of the facial image 2, the probability distribution (0.7, 0.3) of the facial image 3, and the probability distribution (0.3, 0.7) of the facial image 4 through the FC 351 and the Softmax function 352, so that the first model 30 can output the first probability value 0.6 of the facial image 2, the first probability value 0.7 of the facial image 3, and the first probability value 0.3 of the facial image 4.

[0127] In this way, the electronic device can output the probability value of whether the facial image is a facial image of a real object through the FC and Softmax functions in the HEAD module, thereby improving the robustness of the first model and the accuracy of the output results based on the coordination between different modules.

[0128] In the embodiment of the present application, each of the N first probability values ​​represents the probability that a facial image is a facial image of a real object.

[0129] In one embodiment of the present application, the value range of the first probability value may be (0, 1).

[0130] Step 204: The electronic device focuses on the facial image in the shooting preview interface based on the N first probability values.

[0131] In some embodiments of the present application, the above step 204 can be specifically implemented by the following step 204a or step 204b.

[0132] Step 204a: The electronic device focuses on the facial image corresponding to the maximum value among the N first probability values ​​among the N facial images.

[0133] It should be noted that the maximum value among the above N first probability values ​​can indicate that the probability that the facial image corresponding to the maximum value is the facial image of the real object is the highest.

[0134] For example, in combination Figure 8 , when the first probability value of face image 1 is 0.6, the first probability value of face image 2 is 0.6, the first probability value of face image 3 is 0.7, and the first probability value of face image 4 is 0.3, the electronic device may determine Figure 3CThe probability that the facial image 3 is the facial image of a real object is the highest, so the electronic device can focus on the facial image 3 in the shooting preview interface.

[0135] Step 204b: The electronic device focuses on the facial images among the N facial images that correspond to the probability values ​​among the N first probability values ​​that are greater than a preset threshold.

[0136] In some embodiments of the present application, when the first probability value of a facial image is greater than 0.5, the electronic device can confirm that the facial image is a facial image of a real object; when the first probability value of a facial image is less than or equal to 0.5, the electronic device can confirm that the facial image is not a facial image of a real object.

[0137] For example, in combination Figure 8 , when the first probability value of face image 1 is 0.6, the first probability value of face image 2 is 0.6, the first probability value of face image 3 is 0.7, and the first probability value of face image 4 is 0.3, the electronic device may determine Figure 3C The face image 1, face image 2 and face image 3 in are face images of real objects. Figure 3C The face image 4 in FIG. 1 is not a face image of a real object. Thus, the electronic device can determine the focused object based on the face image 1, the face image 2 and the face image 3.

[0138] In some embodiments of the present application, the above step 204b can be specifically implemented through the following step 204b1.

[0139] Step 204b1: When the probability value greater than the preset threshold in the N first probability values ​​includes multiple probability values, the electronic device focuses on the facial image corresponding to one of the multiple probability values ​​based on the shooting parameters of the facial images corresponding to the multiple probability values.

[0140] In some embodiments of the present application, after the electronic device obtains N first probability values, the electronic device may focus based on the facial image of the real user included in the shooting preview interface.

[0141] In some embodiments of the present application, the shooting parameters of the above-mentioned facial image may include but are not limited to at least one of the following: the size of the image frame of the facial image, the distance between the image frame of the facial image and the center point of the camera used by the electronic device for shooting, and the angle between the image frame of the facial image and the center point of the camera used by the electronic device for shooting.

[0142] In some embodiments of the present application, the focusing method of the above-mentioned focusing may be: auto focus (Auto Focus, AF).

[0143] It should be noted that the above-mentioned autofocus can automatically measure the distance to the subject and adjust the lens focal length through specific sensors and computing systems to make the image clear.

[0144] In this way, the electronic device can determine which facial images contained in the shooting preview interface are of real objects and which are not facial images of real objects based on N first probability values, so that the electronic device can focus on the facial images of real objects to avoid the situation where the electronic device automatically focuses on facial images that are not real objects, thereby improving the quality of images taken by the electronic device.

[0145] In the image capture method provided in the embodiment of the present application, since the first model can provide the texture feature information of the facial image based on the input facial image, the electronic device can determine the probability value of each facial image being the facial image of a real object based on the texture feature information of the N facial images. Further, the electronic device can determine which of the N facial images are facial images of the real object based on the output probability value, and then focus the facial image based on the facial image of the real object. In this way, the electronic device can focus on the face of the real object more conveniently, thereby improving the quality of the image captured by the electronic device.

[0146] For various scenarios to which the embodiments of the present application can be applied, combined with the various implementation schemes of the embodiments of the present application described above, specific examples are given below to illustrate the implementation process of the embodiments of the present application in various scenarios.

[0147] Example 1: The preview interface includes a person and a poster, and the poster includes the person's face image

[0148] During shooting with an electronic device, if the shooting preview interface includes person 1 and a poster, and the poster includes a facial image of person 2, the electronic device can obtain facial image 1 of person 1 and facial image 2 of person 2 contained in the shooting preview interface.

[0149] The mobile phone can input the facial image 1 into the first model, and perform downsampling processing on the facial image 1 through the residual module to obtain the first deep image feature information of the facial image 1, and remove the redundant information in the first deep image feature information through the DS Block, and input the texture feature information of the facial image 1. Then, the texture feature information of the facial image 1 is fused through the GAP, and the fused texture feature information of the facial image 1 is input. Further, the logits feature of the facial image 1 is obtained through the FC in the HEAD module, and the first probability value of the facial image 1 is output according to the Softmax function in the HEAD module, such as 0.7. Similarly, the mobile phone can input the facial image 2 into the first model to perform the same operation to output the first probability value of the facial image 2, such as 0.2. Therefore, since the probability value of the facial image 1 is greater than the probability value of the facial image 2, the mobile phone can focus on the facial image 1 of the person 1.

[0150] Example 2: The shooting preview interface includes people, animals and posters, and the poster includes a facial image of the person.

[0151] In the process of shooting through an electronic device, if the shooting preview interface includes person 1, animal 1 and a poster, and the poster includes a facial image of person 2, the electronic device can obtain facial image 1 of person 1, facial image 2 of person 2 and facial image 3 of animal 1 contained in the shooting preview interface. The mobile phone can input facial image 1 into the first model, downsample facial image 1 through the residual module to obtain the first deep image feature information of facial image 1, and remove redundant information in the first deep image feature information through DS Block, and input the texture feature information of facial image 1. Then, the texture feature information of facial image 1 is fused through GAP in the first model, and the fused texture feature information of facial image 1 is input. Further, the logits feature of facial image 1 is obtained through FC in the HEAD module, and the first probability value of facial image 1, such as 0.7, is output according to the Softmax function in the HEAD module. Similarly, the mobile phone can input facial image 2 into the first model to perform the same operation to output the first probability value of facial image 2, such as 0.2; the mobile phone can input facial image 3 into the first model to perform the same operation to output the first probability value of facial image 3, such as 0.6. Therefore, since the probability values ​​of facial image 1 and facial image 3 are greater than 0.5, the mobile phone can determine that facial image 1 and facial image 3 are facial images of real objects; since the probability value of facial image 2 is less than 0.5, the mobile phone can determine that facial image 2 is not a facial image of a real object. Therefore, the mobile phone can determine the focused object from facial image 1 and facial image 3 based on the image information such as the position and size of the facial image in the shooting preview interface.

[0152] The present application embodiment provides a model training method. Fig.11 A flow chart of a model training method provided in an embodiment of the present application is shown, and the method can be applied to electronic devices. Fig.11 As shown, the model training method provided in the embodiment of the present application may include the following steps 401 to 404.

[0153] Step 401: The electronic device inputs a facial image sample into a second model to obtain texture feature information of the facial image sample.

[0154] Step 402: The electronic device processes the texture feature information of the facial image sample through a second model and outputs a second probability value.

[0155] In the embodiment of the present application, the second probability value represents the probability that the facial image sample is a facial image of a real object.

[0156] Step 403: The electronic device determines a loss value based on the second probability value and the target value.

[0157] In the embodiment of the present application, the above target value represents whether the facial image sample is a facial image of a real object.

[0158] Step 404: The electronic device trains the second model based on the loss value to obtain the first model.

[0159] In some embodiments of the present application, the facial image samples may be image samples with a sufficient number of samples, for example, 5,000 facial image samples.

[0160] It should be noted that the specific number of facial image samples can be determined according to the actual needs during the model training process, and this application does not limit it here.

[0161] In some embodiments of the present application, the facial image samples may include facial images at various angles and directions, and under different light angles.

[0162] In some embodiments of the present application, the structure of the above-mentioned second model is roughly the same as the structure of the above-mentioned first model. For details, please refer to the description of the above-mentioned embodiments, and the embodiments of the present application will not be repeated here.

[0163] In the embodiment of the present application, the above target value represents whether the facial image sample is a facial image of a real object.

[0164] Exemplarily, if the facial image sample is a facial image of a real object, the target value is 1; if the facial image sample is not a facial image of a real object, the target value is 0.

[0165] In some embodiments of the present application, for a facial image sample, the electronic device may input the facial image sample into a second model to determine whether the facial image sample is a facial image of a real object based on a detection result of the second model, i.e., a second probability value.

[0166] In some embodiments of the present application, the electronic device may perform rounding processing based on the second probability value to determine a detection value.

[0167] Exemplarily, when the second probability value of a facial image sample is greater than 0.5, the electronic device may round the second probability value to 1 to indicate that the second model has passed detection and determined that the facial image sample is a facial image sample of a real object; when the second probability value of a facial image sample is less than or equal to 0.5, the electronic device may round the second probability value to 0 to indicate that the second model has passed detection and determined that the facial image sample is not a facial image sample of a real object.

[0168] In some embodiments of the present application, the electronic device may determine the loss value by calculating through a cross entropy loss function based on the detection value of the facial image sample rounded based on the second probability value and the target value of the facial image sample.

[0169] It should be noted that the above cross entropy loss function measures the difference between the distribution of the true label and the distribution of the predicted label, that is, the difference between the target value and the test value. The smaller the value of the cross entropy loss function, the closer the predicted result is to the true result, and the better the performance of the model.

[0170] In some embodiments of the present application, the electronic device may use a back propagation algorithm to iteratively update the weight values ​​in the second model based on the loss value and the gradient loss until convergence, so as to train the first model.

[0171] For example, Fig.12As shown, a single facial image sample, such as facial image 5, is used as an example for illustration. After the electronic device obtains the facial image sample, the facial image sample can be input into the second model 40 to be downsampled by the residual module 41 to output image feature information, wherein the residual module 41 can be a residual module of multiple levels, so that the second detection module can perform downsampling processing on the facial image sample for a number of times corresponding to the number of residual modules, so as to finally obtain image feature information a. Then, the second model 40 can input the image feature information a into the residual module 421 in the combination module 42 for downsampling processing, output image feature information b, and copy and store it; then, the second model 40 can input the image feature information b output by the residual module 421 into the DS Block 422 to remove the redundant image feature information in the image feature information b, and output the image feature information c. Further, the second model 40 can fuse the stored image feature information b with the image feature information c of the DS Block 422, i.e., the step indicated by the identifier 423, to output the image feature information d of the facial image.

[0172] Further, the first model can input the image feature information d into the residual module 431 in the combination module 33 for downsampling processing, output the image feature information e, and copy and store it; then, the second model 40 can input the image feature information e output by the residual module 431 into the DS Block 432 to remove redundant image feature information in the image feature information e, and output the image feature information f. Further, the second model 40 can fuse the stored image feature information e and the image feature information f of the DS Block 422, that is, the step indicated by the identifier 433, to output the image feature information g of the facial image, that is, the texture feature information of the facial image 5.

[0173] Furthermore, the second model 40 may fuse the texture feature information of the facial image 5 through the GAP layer 44 to obtain the fused texture feature information of the facial image 5. Then, the second model 40 may integrate the fused texture feature information of the facial image 5 through the FC 451 in the HEAD module 45 to output the logits feature of the facial image 5, and convert the logits feature of the facial image 5 into a probability distribution through the Softmax function 452, so as to obtain the second probability value of the facial image 5.

[0174] Furthermore, the second model 40 can perform the above steps on the facial image samples in batches, and then the second model can calculate the loss value through the cross entropy loss function according to the second probability value of the facial image samples and the target value contained in the image information of the facial image samples. Then, based on the determined loss value, the weight value in the second model 40 is iteratively updated by using the back propagation algorithm until convergence, so as to train the first model.

[0175] In the model training method provided in the embodiment of the present application, since the first model can be trained based on the second model and combined with the texture feature information of the facial image, in the actual use process, the electronic device can input the acquired facial image into the first detection module to directly determine the texture feature information of the input facial image and determine the probability value that the facial image is the facial image of a real object. Furthermore, the electronic device can focus on the facial image based on the output probability value and the facial image of the real object. In this way, the electronic device can focus on the face of the real object more conveniently, thereby improving the quality of the image taken by the electronic device.

[0176] In some embodiments of the present application, the above step 401 can be specifically implemented through step 401a and step 401b.

[0177] Step 401: The electronic device downsamples the facial image samples through the residual module in the second model, and outputs second deep image feature information of the facial image samples.

[0178] Step 402: The electronic device performs redundant information removal processing on the second deep image feature information through the dense-sparse decomposition module in the second model, and outputs texture feature information of the facial image sample.

[0179] It should be noted that the model training method of the above steps 401a and 401b can be specifically referred to the description of steps 202a and 202b in the above embodiments, and this application will not elaborate on it here.

[0180] It should be noted that the above-mentioned method embodiments, or various possible implementation methods in each method embodiment, can be executed separately, or, under the premise that there is no contradiction, can also be executed in combination with each other. The specific implementation can be determined according to actual usage requirements, and the embodiments of the present application do not limit this.

[0181] It should be noted that the image capturing method provided in the embodiment of the present application can be executed by an image capturing device. In the embodiment of the present application, an image capturing device executing the image capturing method is taken as an example to illustrate the image capturing device provided in the embodiment of the present application.

[0182] Fig.13A possible structural diagram of an image capturing device involved in an embodiment of the present application is shown. Fig.13 As shown, the image capturing device 70 may include: an acquisition module 71, a processing module 72 and an execution module 73;

[0183] The acquisition module 71 is used to acquire N facial images in the shooting preview interface, where N is an integer greater than 1;

[0184] The processing module 72 is used to input the N facial images acquired by the acquisition module 71 into the first model to acquire texture feature information of the N facial images; and process the texture feature information of the N facial images through the first model to output N first probability values, each of which represents the probability that a facial image is a facial image of a real object;

[0185] An execution module 73, configured to focus the facial image in the shooting preview interface based on the N first probability values ​​output by the processing module 72;

[0186] The first model is trained based on texture feature information of facial images.

[0187] In a possible implementation, the processing module 72 is specifically used to input N facial images into the residual module in the first model, downsample the N facial images, and output first deep image feature information of the N facial images; and remove redundant information from the first deep image feature information through the dense sparse decomposition module in the first model, and output texture feature information of the N facial images.

[0188] In one possible implementation, each facial image includes texture feature information of multiple dimensions; the above-mentioned processing module 72 is specifically used to fuse the texture feature information of multiple dimensions of each facial image through the global average pooling layer in the first model to obtain the fused texture feature information of each facial image; and to normalize the fused texture feature information of each facial image through the HEAD module in the first model to output N first probability values.

[0189] In a possible implementation, the processing module 72 is specifically used to perform a weighted operation on the fused texture feature information of each facial image through the fully connected layer in the HEAD module, and output N logical value feature vectors; and transform each logical value feature vector through the Softmax function in the HEAD module, and output N first probability values.

[0190] In a possible implementation, the execution module 73 is specifically used to focus on a facial image corresponding to the maximum value of the N first probability values ​​among the N facial images; or to focus on a facial image corresponding to a probability value greater than a preset threshold among the N first probability values ​​among the N facial images.

[0191] In a possible implementation, the execution module 73 is specifically used to focus on a facial image corresponding to one of the multiple probability values ​​based on shooting parameters of the facial images corresponding to the multiple probability values ​​when the probability values ​​greater than the preset threshold in the N first probability values ​​include multiple probability values.

[0192] In one possible implementation, the image capture device provided by the embodiment of the present application also includes: a training module; the above-mentioned processing module 72 is also used to input the facial image sample into the second model to obtain the texture feature information of the facial image sample before the acquisition module 71 acquires the N facial images in the shooting preview interface; and the texture feature information of the facial image sample is processed through the second model to output a second probability value, and the second probability value represents the probability that the facial image sample is a facial image of a real object; and, based on the second probability value and the target value, determine the loss value, and the target value represents whether the facial image sample is a facial image of a real object; the above-mentioned training module is used to train the second model based on the loss value determined by the processing module 72 to obtain the first model.

[0193] In the image capture device provided in the embodiment of the present application, since the first model can provide the texture feature information of the facial image based on the input facial image, the image capture device can determine the probability value of each facial image being the facial image of a real object based on the texture feature information of the N facial images. Furthermore, the image capture device can determine which of the N facial images are facial images of the real object based on the output probability value, and then focus the facial image based on the facial image of the real object. In this way, the image capture device can more conveniently focus on the face of the real object, thereby improving the quality of the image captured by the image capture device.

[0194] Fig.14 A possible structural diagram of the model training device involved in the embodiment of the present application is shown. Fig.14 As shown, the model training device 80 may include: an acquisition module 81, a processing module 82, a determination module 83 and a training module 84;

[0195] The acquisition module 81 is used to input the facial image sample into the second model to obtain the texture feature information of the facial image sample;

[0196] A processing module 82 is used to process the texture feature information of the facial image sample acquired by the acquisition module through a second model, and output a second probability value, wherein the second probability value represents the probability that the facial image sample is a facial image of a real object;

[0197] A determination module 83, configured to determine a loss value based on the second probability value and the target value obtained by the processing module 82, wherein the target value represents whether the facial image sample is a facial image of a real object;

[0198] The training module 84 is used to train the second model based on the loss value determined by the determination module 83 to obtain the first model.

[0199] In a possible implementation, the acquisition module 81 is specifically used to downsample the facial image samples through the residual module in the second model, and output the second deep image feature information of the facial image samples; and to remove redundant information from the second deep image feature information through the dense sparse decomposition module in the second model, and output the texture feature information of the facial image samples.

[0200] In the model training device provided in the embodiment of the present application, since the first model can be trained based on the second model and combined with the texture feature information of the facial image, in the actual use process, the model training device can input the acquired facial image into the first detection module to directly determine the texture feature information of the input facial image and determine the probability value that the facial image is the facial image of a real object. Furthermore, the model training device can focus on the facial image based on the output probability value and the facial image of the real object. In this way, the model training device can focus on the face of the real object more conveniently, thereby improving the quality of the image taken by the model training device.

[0201] The image capture device in the embodiment of the present application can be an electronic device, or a component in the electronic device, such as an integrated circuit or a chip. The electronic device can be a terminal, or other devices other than a terminal. Exemplarily, the electronic device can be a mobile phone, a tablet computer, a laptop computer, a PDA, a vehicle-mounted electronic device, a mobile Internet device (Mobile Internet Device, MID), an augmented reality (augmented reality, AR) / virtual reality (virtual reality, VR) device, a robot, a wearable device, an ultra-mobile personal computer (ultra-mobile personal computer, UMPC), a netbook or a personal digital assistant (personal digital assistant, PDA), etc., and can also be a server, a network attached storage (Network Attached Storage, NAS), a personal computer (personal computer, PC), a television (television, TV), a teller machine or a self-service machine, etc., which is not specifically limited in the embodiment of the present application.

[0202] The image capture device in the embodiment of the present application may be a device having an operating system. The operating system may be an Android operating system, an iOS operating system, or other possible operating systems, which are not specifically limited in the embodiment of the present application.

[0203] The image capture device provided in the embodiment of the present application can implement each process implemented in the above method embodiment, and will not be described again here to avoid repetition.

[0204] Alternatively, if Fig.15 As shown, an embodiment of the present application further provides an electronic device 90, including a processor 91 and a memory 92, wherein the memory 92 stores programs or instructions that can be executed on the processor 91, and when the program or instructions are executed by the processor 91, the various steps of the above-mentioned image capture method embodiment are implemented, and the same technical effect can be achieved. To avoid repetition, they are not described here.

[0205] It should be noted that the electronic devices in the embodiments of the present application include the mobile electronic devices and non-mobile electronic devices mentioned above.

[0206] Fig.16 A schematic diagram of the hardware structure of an electronic device to implement an embodiment of the present application.

[0207] The electronic device 100 includes but is not limited to components such as a radio frequency unit 101, a network module 102, an audio output unit 103, an input unit 104, a sensor 105, a display unit 106, a user input unit 107, an interface unit 108, a memory 109, and a processor 110.

[0208] Those skilled in the art will appreciate that the electronic device 100 may also include a power source (such as a battery) for supplying power to various components, and the power source may be logically connected to the processor 110 through a power management system, thereby implementing functions such as managing charging, discharging, and power consumption management through the power management system. Fig.16 The electronic device structure shown in the figure does not constitute a limitation on the electronic device. The electronic device may include more or fewer components than shown in the figure, or combine certain components, or arrange the components differently, which will not be described in detail here.

[0209] Among them, the processor 110 is used to obtain N facial images in the shooting preview interface, where N is an integer greater than 1; and input the N facial images into the first model to obtain texture feature information of the N facial images; and, through the first model, process the texture feature information of the N facial images, output N first probability values, each first probability value represents the probability that a facial image is a facial image of a real object; and, based on the N first probability values, focus on the facial image in the shooting preview interface; wherein the first model is trained based on the texture feature information of the facial image.

[0210] Optionally, the processor 110 is specifically used to downsample the N facial images through the residual module in the first model, and output the first deep image feature information of the N facial images; and remove redundant information from the first deep image feature information through the dense sparse decomposition module in the first model, and output the texture feature information of the N facial images.

[0211] Optionally, each facial image includes texture feature information of multiple dimensions; the processor 110 is specifically used to fuse the texture feature information of multiple dimensions of each facial image through the global average pooling layer in the first model to obtain the fused texture feature information of each facial image; and normalize the fused texture feature information of each facial image through the HEAD module in the first model to output N first probability values.

[0212] Optionally, the processor 110 is specifically used to perform a weighted operation on the fused texture feature information of each facial image through the fully connected layer in the HEAD module, and output N logical value feature vectors; and transform each logical value feature vector through the Softmax function in the HEAD module, and output N first probability values.

[0213] Optionally, the processor 110 is specifically used to focus on a facial image among the N facial images that corresponds to the maximum value among the N first probability values; or, to focus on a facial image among the N facial images that corresponds to a probability value among the N first probability values ​​that is greater than a preset threshold.

[0214] Optionally, the processor 110 is specifically configured to focus a facial image corresponding to one of the multiple probability values ​​based on shooting parameters of the facial images corresponding to the multiple probability values ​​when the probability values ​​greater than a preset threshold in the N first probability values ​​include multiple probability values.

[0215] In the electronic device provided in the embodiment of the present application, since the first model can provide the texture feature information of the facial image based on the input facial image, the electronic device can determine the probability value of each facial image being the facial image of a real object based on the texture feature information of the N facial images. Further, the electronic device can determine which of the N facial images are facial images of the real object based on the output probability value, and then focus the facial image based on the facial image of the real object. In this way, the electronic device can focus on the face of the real object more conveniently, thereby improving the quality of the image taken by the electronic device.

[0216] Among them, the processor 110 is used to input the facial image samples into the second model before obtaining N facial images in the shooting preview interface to obtain texture feature information of the facial image samples; and process the texture feature information of the facial image samples through the second model to output a second probability value, and the second probability value represents the probability that the facial image sample is a facial image of a real object; and, based on the second probability value and the target value, determine the loss value, the target value represents whether the facial image sample is a facial image of a real object; and, based on the loss value, train the second model to obtain the first model.

[0217] Optionally, the processor 110 is specifically used to downsample the facial image samples through the residual module in the second model, and output the second deep image feature information of the facial image samples; and to remove redundant information from the second deep image feature information through the dense sparse decomposition module in the second model, and output the texture feature information of the facial image samples.

[0218] In the electronic device provided in the embodiment of the present application, since the first model can be trained based on the second model and combined with the texture feature information of the facial image, during actual use, the electronic device can input the acquired facial image into the first detection module to directly determine the texture feature information of the input facial image and determine the probability value that the facial image is the facial image of a real object. Furthermore, the electronic device can focus on the facial image based on the output probability value and the facial image of the real object. In this way, the electronic device can focus on the face of the real object more conveniently, thereby improving the quality of the image taken by the electronic device.

[0219] The electronic device provided in the embodiment of the present application can implement each process implemented in the above method embodiment and can achieve the same technical effect. To avoid repetition, it will not be described here.

[0220] The beneficial effects of various implementations in this embodiment can be specifically referred to the beneficial effects of the corresponding implementations in the above method embodiment. To avoid repetition, they will not be described again here.

[0221] It should be understood that in the embodiment of the present application, the input unit 104 may include a graphics processor (Graphics Processing Unit, GPU) 1041 and a microphone 1042, and the graphics processor 1041 processes the image data of a static picture or video obtained by an image capture device (such as a camera) in a video capture mode or an image capture mode. The display unit 106 may include a display panel 1061, and the display panel 1061 may be configured in the form of a liquid crystal display, an organic light emitting diode, etc. The user input unit 107 includes a touch panel 1071 and at least one of other input devices 1072. The touch panel 1071 is also called a touch screen. The touch panel 1071 may include two parts: a touch detection device and a touch controller. Other input devices 1072 may include, but are not limited to, a physical keyboard, function keys (such as volume control keys, switch keys, etc.), a trackball, a mouse, and a joystick, which will not be repeated here.

[0222] The memory 109 can be used to store software programs and various data. The memory 109 may mainly include a first storage area for storing programs or instructions and a second storage area for storing data, wherein the first storage area may store an operating system, an application program or instructions required for at least one function (such as a sound playback function, an image playback function, etc.), etc. In addition, the memory 109 may include a volatile memory or a non-volatile memory, or the memory 109 may include both volatile and non-volatile memories. Among them, the non-volatile memory may be a read-only memory (ROM), a programmable read-only memory (PROM), an erasable programmable read-only memory (EPROM), an electrically erasable programmable read-only memory (EEPROM), or a flash memory. The volatile memory may be a random access memory (RAM), a static random access memory (SRAM), a dynamic random access memory (DRAM), a synchronous dynamic random access memory (SDRAM), a double data rate synchronous dynamic random access memory (DDRSDRAM), an enhanced synchronous dynamic random access memory (ESDRAM), a synchronous link dynamic random access memory (SLDRAM) and a direct memory bus random access memory (DRRAM). The memory 109 in the embodiment of the present application includes but is not limited to these and any other suitable types of memory.

[0223] The processor 110 may include one or more processing units; optionally, the processor 110 integrates an application processor and a modem processor, wherein the application processor mainly processes operations related to an operating system, a user interface, and application programs, and the modem processor mainly processes wireless communication signals, such as a baseband processor. It is understandable that the modem processor may not be integrated into the processor 110.

[0224] An embodiment of the present application also provides a readable storage medium, on which a program or instruction is stored. When the program or instruction is executed by a processor, the various processes of the above-mentioned method embodiment are implemented and the same technical effect can be achieved. To avoid repetition, it will not be repeated here.

[0225] The processor is the processor in the electronic device described in the above embodiment. The readable storage medium includes a computer readable storage medium, such as a computer read-only memory ROM, a random access memory RAM, a magnetic disk or an optical disk.

[0226] An embodiment of the present application further provides a chip, which includes a processor and a communication interface, wherein the communication interface is coupled to the processor, and the processor is used to run programs or instructions to implement the various processes of the above-mentioned method embodiment, and can achieve the same technical effect. To avoid repetition, it will not be repeated here.

[0227] It should be understood that the chip mentioned in the embodiments of the present application can also be called a system-level chip, a system chip, a chip system or a system-on-chip chip, etc.

[0228] An embodiment of the present application provides a computer program product, which is stored in a storage medium. The program product is executed by at least one processor to implement the various processes of the above-mentioned method embodiment and can achieve the same technical effect. To avoid repetition, it will not be repeated here.

[0229] It should be noted that, in this article, the terms "comprise", "include" or any other variant thereof are intended to cover non-exclusive inclusion, so that the process, method, article or device including a series of elements includes not only those elements, but also includes other elements not explicitly listed, or also includes elements inherent to such process, method, article or device. In the absence of further restrictions, the elements defined by the sentence "comprise one..." do not exclude the presence of other identical elements in the process, method, article or device including the element. In addition, it should be pointed out that the scope of the methods and devices in the embodiments of the present application is not limited to performing functions in the order shown or discussed, and may also include performing functions in a substantially simultaneous manner or in reverse order according to the functions involved, for example, the described method may be performed in an order different from that described, and various steps may also be added, omitted, or combined. In addition, the features described with reference to certain examples may be combined in other examples.

[0230] Through the description of the above implementation methods, those skilled in the art can clearly understand that the above-mentioned embodiment methods can be implemented by means of software plus a necessary general hardware platform, and of course by hardware, but in many cases the former is a better implementation method. Based on such an understanding, the technical solution of the present application, or the part that contributes to the prior art, can be embodied in the form of a computer software product, which is stored in a storage medium (such as ROM / RAM, a disk, or an optical disk), and includes a number of instructions for a terminal (which can be a mobile phone, a computer, a server, or a network device, etc.) to execute the methods described in each embodiment of the present application.

[0231] The embodiments of the present application are described above in conjunction with the accompanying drawings, but the present application is not limited to the above-mentioned specific implementation methods. The above-mentioned specific implementation methods are merely illustrative and not restrictive. Under the guidance of the present application, ordinary technicians in this field can also make many forms without departing from the purpose of the present application and the scope of protection of the claims, all of which are within the protection of the present application.

Claims

1. An image capturing method, characterized in that: The method comprises: Get N facial images in the shooting preview interface, where N is an integer greater than 1; Inputting the N facial images into a first model to obtain texture feature information of the N facial images; Processing the texture feature information of the N facial images through the first model, and outputting N first probability values, each of which represents the probability that a facial image is a facial image of a real object; Based on the N first probability values, the facial image in the shooting preview interface is focused.

2. The method according to claim 1, characterized in that The step of inputting the N facial images into a first model to obtain texture feature information of the N facial images includes: Downsampling the N facial images through the residual module in the first model, and outputting first deep image feature information of the N facial images; The first deep image feature information is processed to remove redundant information through the dense-sparse decomposition module in the first model, and texture feature information of the N facial images is output.

3. The method according to claim 1 or 2, characterized in that: Each facial image includes texture feature information in multiple dimensions; The processing of the texture feature information of the N facial images by the first model and outputting N first probability values ​​includes: By using the global average pooling layer in the first model, the texture feature information of multiple dimensions of each facial image is fused to obtain fused texture feature information of each facial image; The fused texture feature information of each facial image is normalized by the detection head HEAD module in the first model, and the N first probability values ​​are output.

4. The method according to claim 3, characterized in that The HEAD module in the first model is used to normalize the fused texture feature information of each facial image and output the N first probability values, including: Through the fully connected layer in the HEAD module, a weighted operation is performed on the fused texture feature information of each facial image to output N logical value feature vectors; Each logical value feature vector is transformed through the normalized exponential Softmax function in the HEAD module to output the N first probability values.

5. The method according to claim 1, characterized in that The focusing of the facial image in the shooting preview interface based on the N first probability values ​​includes: Focusing on a facial image among the N facial images that corresponds to a maximum value among the N first probability values; or, Focusing is performed on a facial image among the N facial images that corresponds to a probability value among the N first probability values ​​that is greater than a preset threshold.

6. The method according to claim 5, characterized in that The focusing, among the N facial images, on a facial image corresponding to a probability value greater than a preset threshold among the N first probability values ​​comprises: When the probability values ​​greater than the preset threshold in the N first probability values ​​include multiple probability values, focusing is performed on a facial image corresponding to one of the multiple probability values ​​based on shooting parameters of the facial images corresponding to the multiple probability values.

7. A model training method, characterized in that: The method comprises: Inputting the facial image sample into the second model to obtain texture feature information of the facial image sample; Processing the texture feature information of the facial image sample by the second model and outputting a second probability value, wherein the second probability value represents the probability that the facial image sample is a facial image of a real object; determining a loss value based on the second probability value and a target value, wherein the target value indicates whether the facial image sample is a facial image of a real object; Based on the loss value, the second model is trained to obtain the first model.

8. The method according to claim 7, characterized in that The step of inputting the facial image sample into the second model to obtain texture feature information of the facial image sample includes: Downsampling the facial image samples through the residual module in the second model, and outputting second deep image feature information of the facial image samples; The second deep image feature information is processed to remove redundant information through the dense-sparse decomposition module in the second model, and texture feature information of the facial image sample is output.

9. An image capturing device, characterized in that: The image shooting device comprises: an acquisition module, a processing module and an execution module; The acquisition module is used to acquire N facial images in the shooting preview interface, where N is an integer greater than 1; and input the N facial images into the first model to acquire texture feature information of the N facial images; The processing module is used to process the texture feature information of the N facial images acquired by the acquisition module through the first model, and output N first probability values, each of which represents the probability that a facial image is a facial image of a real object; The execution module is used to focus the facial image in the shooting preview interface based on the N first probability values ​​output by the processing module.

10. A model training device, characterized in that: The model training device comprises: an acquisition module, a processing module, a determination module and a training module; The acquisition module is used to input the facial image sample into the second model to acquire the texture feature information of the facial image sample; The processing module is used to process the texture feature information of the facial image sample acquired by the acquisition module through the second model, and output a second probability value, wherein the second probability value represents the probability that the facial image sample is a facial image of a real object; The determination module is used to determine a loss value based on the second probability value and a target value obtained by the processing module, wherein the target value represents whether the facial image sample is a facial image of a real object; The training module is used to train the second model based on the loss value determined by the determination module to obtain the first model.

11. An electronic device, characterized in that: It includes a processor and a memory, wherein the memory stores a program or instruction that can be run on the processor, and when the program or instruction is executed by the processor, the steps of the image capture method as described in any one of claims 1 to 6, or the steps of the model training method as described in claim 7 or 8 are implemented.

12. A readable storage medium, characterized in that: The readable storage medium stores programs or instructions, and when the programs or instructions are executed by the processor, the steps of the image capture method as described in any one of claims 1 to 6, or the steps of the model training method as described in claim 7 or 8 are implemented.

13. A computer program product, characterized in that The computer program product is stored in a storage medium, and the program product is executed by at least one processor to implement the steps of the image capture method as described in any one of claims 1 to 6, or the steps of the model training method as described in claim 7 or 8.

Images