Focusing method and device, electronic equipment and storage medium

Abstract

The invention provides a focusing method and device, electronic equipment and a storage medium, and the method comprises the steps: obtaining a first face region according to the depth information of at least one face region in an original image; performing face size recognition on the first face area to obtain a second face area; and executing corresponding operation according to the second face area. According to the embodiment, through the face size recognition processing, the false face in the shooting scene can be removed to obtain the real face, the face recognition accuracy in the shooting process is improved, and the shooting experience is improved.

Description

Technical Field

[0001] The present disclosure relates to the field of image processing technology, and in particular to a focusing method and device, an electronic device, and a storage medium. Background Art

[0002] An electronic device is usually provided with a camera, which has multiple shooting modes, such as photo, portrait, video, etc., so as to enrich the user's shooting experience.

[0003] Taking portrait shooting as an example, the camera usually extracts the position of the face and quickly focuses on the face to give users a good user experience.

[0004] In actual applications, there may be backgrounds with faces such as billboards or display screens in the shooting scene. The camera may make recognition errors when recognizing faces, resulting in out-of-focus and affecting the user's physical examination. Summary of the invention

[0005] The present disclosure provides a focusing method and device, an electronic device, and a storage medium to solve the above technical problems.

[0006] According to a first aspect of the present disclosure, there is provided a focusing method, comprising:

[0007] Obtaining a first facial region according to depth information of at least one facial region in the original image;

[0008] Performing facial size recognition on the first facial region to obtain a second facial region;

[0009] A corresponding operation is performed according to the second facial region.

[0010] Optionally, performing facial size recognition on the first facial region to obtain a second facial region includes:

[0011] Performing facial classification and recognition processing on the first facial region to obtain a candidate facial region containing a real face;

[0012] The candidate facial region is processed for size recognition to obtain a second facial region containing a real face.

[0013] Optionally, performing facial classification and recognition processing on the first facial region to obtain a candidate facial region containing a real face includes:

[0014] Acquire depth information of a plurality of pixels in the first facial region;

[0015] Inputting the depth information into a preset classification model to obtain a face type output by the preset classification model, wherein the face type includes a real face and a fake face;

[0016] A first facial region whose facial type is a real face is determined as a candidate facial region containing a real face.

[0017] Optionally, acquiring depth information of a plurality of pixels in the first facial region includes:

[0018] Obtaining a center point of the first facial region;

[0019] Depth information of each pixel in a preset area with the center point as a reference point is obtained as depth information of multiple pixels in the first facial area.

[0020] Optionally, performing size recognition processing on the candidate facial region to obtain a second facial region containing a real face includes:

[0021] Obtaining the actual size of the face in the candidate face area;

[0022] Determine a face type of the candidate face region according to the actual size and a preset size range, wherein the face type includes a real face and a false face;

[0023] The candidate facial regions are screened according to the facial type to obtain a second facial region containing a real face.

[0024] Optionally, obtaining the actual size of the face in the candidate face region includes:

[0025] Obtaining shooting parameters corresponding to the original image;

[0026] The actual size of the face in the candidate face area is determined according to the shooting parameters.

[0027] Optionally, obtaining the first facial region according to depth information of at least one facial region in the original image includes:

[0028] Obtaining target depth information corresponding to the original image;

[0029] Acquire the facial regions within each initial recognition frame in the target depth information to obtain the first facial regions corresponding to each initial recognition frame.

[0030] Optionally, obtaining target depth information corresponding to the original image includes:

[0031] Acquire first depth information corresponding to the original image;

[0032] Acquire depth information of at least one pixel in the original image;

[0033] Correcting the first depth information based on the depth information of the at least one pixel to obtain second depth information;

[0034] The second depth information is aligned with the original image to obtain target depth information corresponding to the original image.

[0035] Optionally, obtaining first depth information corresponding to the original image includes:

[0036] Dividing the original image into a plurality of image blocks;

[0037] Obtaining the phase difference corresponding to each image block according to the parallax ranging method;

[0038] Depth information of each pixel in the original image is acquired according to the phase difference to obtain first depth information corresponding to the original image.

[0039] Optionally, obtaining depth information of at least one pixel in the original image includes:

[0040] Dividing the shooting area corresponding to the original image into at least one partition;

[0041] Depth information of the nearest point of each partition in the at least one partition is obtained as depth information of at least one pixel point in the original image.

[0042] Optionally, performing a corresponding operation according to the second facial area includes:

[0043] A focusing operation is performed on the next frame of image according to the second facial region to obtain an identification frame corresponding to the next frame of original image.

[0044] According to a second aspect of the present disclosure, there is provided a focusing device, comprising:

[0045] A first region acquisition module, configured to obtain a first facial region according to depth information of at least one facial region in an original image;

[0046] A second region acquisition module, configured to perform facial size recognition on the first facial region to obtain a second facial region;

[0047] An operation execution module is used to execute a corresponding operation according to the second facial area.

[0048] Optionally, the second region acquisition module includes:

[0049] A candidate region acquisition submodule, configured to perform facial classification and recognition processing on the first facial region to obtain a candidate facial region containing a real face;

[0050] The second region acquisition submodule is used to perform size recognition processing on the candidate facial region to obtain a second facial region containing a real face.

[0051] Optionally, the candidate region acquisition submodule includes:

[0052] A depth information acquisition unit, configured to acquire depth information of a plurality of pixels in the first facial region;

[0053] A depth information acquisition unit, used to input the depth information into a preset classification model to obtain a face type output by the preset classification model, wherein the face type includes a real face and a fake face;

[0054] The candidate region acquisition unit is used to determine the first facial region whose facial type is a real face as a candidate facial region containing the real face.

[0055] Optionally, the depth information acquiring unit includes:

[0056] A center point acquisition subunit, used to acquire the center point of the first facial area;

[0057] The depth information acquisition subunit is used to acquire the depth information of each pixel point in a preset area with the central point as a reference point as the depth information of multiple pixel points in the first facial area.

[0058] Optionally, the second region acquisition submodule includes:

[0059] An actual size acquisition unit, used to acquire the actual size of the face in the candidate face area;

[0060] A face type acquisition unit, configured to determine a face type of the candidate face region according to the actual size and a preset size range, wherein the face type includes a real face and a false face;

[0061] The second region acquisition unit is used to filter the candidate facial regions according to the facial type to obtain a second facial region containing a real face.

[0062] Optionally, the actual size acquisition unit includes:

[0063] A shooting parameter acquisition subunit, used to acquire shooting parameters corresponding to the original image;

[0064] The actual size acquisition subunit is used to determine the actual size of the face in the candidate face area according to the shooting parameters.

[0065] Optionally, the first region acquisition module includes:

[0066] The target information acquisition submodule is used to acquire the target depth information corresponding to the original image;

[0067] The first region acquisition submodule is used to acquire the facial region within each initial recognition frame in the target depth information, and obtain the first facial region corresponding to each initial recognition frame.

[0068] Optionally, the target information acquisition submodule includes:

[0069] A first information acquisition unit, configured to acquire first depth information corresponding to the original image;

[0070] A pixel information acquisition unit, used to acquire depth information of at least one pixel in the original image;

[0071] A second information acquisition unit, configured to perform correction processing on the first depth information based on the depth information of the at least one pixel point to obtain second depth information;

[0072] The target information acquisition unit is used to align the second depth information with the original image to obtain target depth information corresponding to the original image.

[0073] Optionally, the first information acquisition unit includes:

[0074] An image block division subunit, used for dividing the original image into a plurality of image blocks;

[0075] A phase difference acquisition subunit, used to acquire the phase difference corresponding to each image block according to the parallax ranging method;

[0076] The first information acquisition subunit is used to acquire the depth information of each pixel in the original image according to the phase difference, so as to obtain the first depth information corresponding to the original image.

[0077] Optionally, the pixel information acquisition unit includes:

[0078] A partition division subunit, used to divide the shooting area corresponding to the original image into at least one partition;

[0079] The depth information acquisition subunit is used to acquire the depth information of the nearest point of each partition in the at least one partition as the depth information of at least one pixel point in the original image.

[0080] Optionally, the operation execution module includes:

[0081] The recognition frame acquisition submodule is used to perform a focusing operation on the next frame of the image according to the second facial region to obtain a recognition frame corresponding to the next frame of the original image. Focusing method

[0082] According to a third aspect of the present disclosure, there is provided an electronic device, including:

[0083] Processor and memory;

[0084] The memory is used to store a computer program executable by the processor;

[0085] The processor is configured to execute the computer program in the memory to implement the method as described in any one of the first aspects.

[0086] According to a fourth aspect of the present disclosure, a non-transitory computer-readable storage medium is provided, and when an executable computer program in the storage medium is executed by a processor, the method described in any one of the first aspects can be implemented.

[0087] The technical solution provided by the embodiments of the present disclosure may have the following beneficial effects:

[0088] The solution of this embodiment can obtain a first facial region based on the depth information of at least one facial region in the original image; then, perform facial size recognition on the first facial region to obtain a second facial region; and then, perform corresponding operations based on the second facial region. In this way, this embodiment can eliminate false faces in the shooting scene and obtain real faces through facial size recognition processing, thereby improving the facial recognition accuracy of the shooting process and enhancing the shooting experience.

[0089] It is to be understood that the foregoing general description and the following detailed description are exemplary and explanatory only and are not restrictive of the present disclosure. BRIEF DESCRIPTION OF THE DRAWINGS

[0090] Figure 1 The present invention is a flowchart of a focusing method according to an embodiment of the present invention.

[0091] Figure 2 The present invention is a flowchart of obtaining an initial face depth image according to an embodiment of the present invention.

[0092] Figure 3 The present invention is a flowchart of obtaining a depth image of a target face according to an embodiment of the present invention.

[0093] Figure 4 The present invention is a flowchart of obtaining a candidate depth image according to an embodiment of the present invention.

[0094] Figure 5 The present invention is a flowchart of obtaining a target recognition frame according to an embodiment of the present invention.

[0095] Figure 6 A schematic diagram of a pinhole imaging model according to an embodiment of the present disclosure.

[0096] Figure 7 The present invention is a flowchart of determining an abnormal event according to an embodiment of the present invention.

[0097] Figure 8 The figure is a schematic diagram of the conversion between a camera coordinate system and an image coordinate system according to an embodiment of the present disclosure.

[0098] Fig. 9 The figure is a schematic diagram of adjusting the lens position according to an embodiment of the present disclosure.

[0099] Fig.10 The present invention is a flowchart of a focusing method according to an embodiment of the present invention.

[0100] Fig.11 The present invention is a block diagram of a focusing device according to an embodiment of the present invention.

[0101] Fig.12 The present invention is a block diagram of an electronic device according to an embodiment of the present invention. DETAILED DESCRIPTION

[0102] Exemplary embodiments will be described in detail herein, examples of which are shown in the accompanying drawings. When the following description refers to the drawings, the same numbers in different drawings represent the same or similar elements unless otherwise indicated. The embodiments described in the following exemplary embodiments do not represent all embodiments consistent with the present disclosure. Instead, they are merely examples of devices consistent with some aspects of the present disclosure as detailed in the appended claims.

[0103] The embodiments of the present disclosure provide a focusing method and device, an electronic device, and a storage medium. The above-mentioned focusing method can be applied to electronic devices, and the electronic devices can include but are not limited to devices with camera functions such as smart phones, tablet computers, and smart displays. For the convenience of description, the schemes of various embodiments are described below with the electronic device as the execution subject. In some embodiments, the electronic device takes a smart phone as an example, but it does not constitute a limitation on the scheme of the present disclosure.

[0104] Figure 1 Flow chart of a focusing method according to an embodiment of the present disclosure. Figure 1 , a focusing method, comprising steps 11 to 13.

[0105] In step 11, a first facial region is obtained according to depth information of at least one facial region in the original image.

[0106] In this embodiment, a camera and a camera application are provided in the electronic device. After the electronic device is turned on, a main interface can be displayed, and a camera application (or camera APP) can be displayed in the main interface. When there is a need to shoot, the user can trigger the camera application, and the electronic device switches from the main interface to the shooting interface.

[0107] After the camera application is started, the camera application will send a preview request according to the set period. The preview request is finally sent to the hardware layer. At this time, the image collected by the image sensor can be processed by image pre-processing, image post-processing and other operations to obtain a preview image, and the preview image is returned to the application layer as a result, and finally displayed in the shooting interface of the camera application.

[0108] In this embodiment, after obtaining the preview image or during the image post-processing process, the electronic device will enable the facial recognition function. For the convenience of description, in this embodiment, the image used to locate the facial area is unified as the original image. The facial recognition function can locate the facial area in the original image; generate an external frame of the facial area, and obtain the coordinate data and identification code of the external frame. In some embodiments, the coordinate data of the external frame can be the coordinate data of the two diagonal points of the external frame, or it can be the coordinate data of the four corners. Among them, the identification code of the external frame is the encoding of each external frame in the original image, such as 0, 1, 2, 3, etc., and the encoding can be used to uniquely identify an external frame.

[0109] It is understandable that in the process of taking portraits, each preview image to be displayed has coordinate data and an identification code of the bounding box of at least one facial area, that is, the processor can obtain the bounding box of at least one facial area in the original image, which is subsequently referred to as the initial identification frame to facilitate the description of the solution.

[0110] In another embodiment, the electronic device may be provided with a facial recognition model, which may be implemented using a neural network, such as a convolutional neural network, a recurrent neural network, etc. In the case where facial recognition can be implemented, the corresponding solution falls within the protection scope of the present disclosure. In this embodiment, after obtaining the preview image, it may be input into the facial recognition model, and the recognition frame of the facial area output by the facial recognition model may be obtained, and the processor may obtain the initial recognition frame of the original image.

[0111] In this embodiment, the processor can obtain the first facial region according to the depth information of at least one facial region in the original image. Figure 2 , including step 21 and step 22.

[0112] In step 21, target depth information corresponding to the original image is obtained.

[0113] In one example, the processor may obtain target depth information corresponding to the original image, including:

[0114] First, the processor can obtain the depth information corresponding to the original image. For example, the processor can divide the original image into multiple image blocks, and then obtain the phase difference corresponding to each image block according to the parallax ranging method; then, the depth information of each pixel in the original image is obtained according to the phase difference to obtain the depth information corresponding to the original image, which is hereinafter referred to as the first depth information.

[0115] Secondly, the processor can obtain the depth information of at least one pixel in the original image. Taking the time of flight (ToF) method to obtain the distance as an example, the coverage range of the TOF sensor and the camera can be pre-configured so that the TOF sensor acquisition range is the same as the camera shooting range, or the TOF sensor acquisition range and the camera shooting range are converted so that the two coverage ranges are the same. On this basis, the processor can divide the shooting area corresponding to the original image into at least one partition; the processor can control the TOF sensor to send a detection signal (ultrasonic signal or infrared signal, etc.) to each partition and receive the echo signal of the detection signal, and calculate the distance of each detection point (that is, the distance between the camera and the object being photographed). Then, the processor can obtain the depth information of the nearest point in each partition. When the number of detection points of the TOF sensor is greater than or equal to the number of pixels of the camera, the nearest point of each partition can be mapped one by one to the original image as the depth information of at least one pixel in the original image. In this way, the processor can obtain the depth information of at least one pixel with high accuracy.

[0116] Afterwards, the processor can correct the first depth information based on the depth information of at least one pixel in the original image, for example, updating the depth information of the pixel corresponding to the first depth information to the depth information detected by the TOF sensor; and obtaining the difference in depth information before and after the update of the depth information of the pixel, or the average value of the difference of at least one pixel; then, using the above difference or average value to compensate for the depth information of other pixels, the depth data of all pixels of the first depth image are corrected, and the second depth information can be obtained. In this way, this embodiment can use the depth information of at least one pixel with higher precision to correct the first depth information, which can improve the accuracy of the depth information in the first depth information, thereby obtaining more accurate second depth information.

[0117] Finally, the processor may align the second depth information with the original image in pixel alignment, or in other words, add the depth information of each pixel in the second depth information to the pixel information of the matching pixel in the original image to obtain the target depth information corresponding to the original image.

[0118] In another example, the processor may control the camera to capture the original image, and at the same time control the depth sensor or depth camera to collect depth information; then, the original image and the depth information may be fused through image post-processing operations to obtain second depth information.

[0119] It should be noted that technicians can select a corresponding method to obtain the second depth information according to the specific scenario, and the corresponding solution falls within the protection scope of this disclosure.

[0120] In step 22, the facial regions within each initial recognition frame in the target depth image are acquired to obtain the first facial regions corresponding to each initial recognition frame.

[0121] In this embodiment, the processor can obtain the facial area in each initial recognition frame in the target depth information, and obtain the first facial area corresponding to each initial recognition frame. It is understandable that at least one initial recognition frame in the original image is obtained in step 11, and each initial recognition frame includes coordinate data and an identification code. The processor can, based on the order of the identification codes of each initial recognition frame, sequentially crop the image portion in each initial recognition frame in the target depth information according to the coordinate data of each initial recognition frame, thereby obtaining the facial area corresponding to each initial recognition frame, which is subsequently referred to as the first facial area. In this way, the purpose of obtaining the first facial area in this embodiment is to reduce the amount of data processing without processing the entire image, which is conducive to improving processing efficiency.

[0122] In step 12, facial size recognition is performed on the first facial region to obtain a second facial region.

[0123] In this embodiment, the real face refers to the face of a real target object, for example, if the target object is a person, the face refers to the human face; for example, if the target object is an animal, the face refers to the animal face; and the fake face refers to a face similar to the real face in a picture, display screen, etc. Since the fake face and the real face are similar or identical in the two-dimensional original image, it is easy to identify the fake face as the real face. To this end, the processor can perform facial recognition processing on the first facial region to obtain a second facial region containing the real face, see Figure 3 , including step 31 and step 32.

[0124] In step 31, face classification and recognition processing is performed on the first face region to obtain a candidate face region containing a real face.

[0125] In this embodiment, the facial classification and recognition process refers to recognizing whether the face in the first facial region is a real face or a false face. For example, the face in the candidate facial region has 3D features.

[0126] For example, the processor may perform facial classification and recognition processing on the first facial region to obtain a candidate facial image containing a real face, see Figure 4 , including steps 41 to 43.

[0127] In step 41, depth information of a plurality of pixels in the first facial region is obtained.

[0128] In this step, the processor can obtain the center point of the first facial area. Since the shape of the first facial area is rectangular, its center point can be obtained in the form of the center point of the rectangle, that is, the ordinate of the center point of the long side of the rectangle and the abscissa of the center point of the wide side of the rectangle constitute the coordinate data of the center point of the rectangle.

[0129] In this step, the processor can obtain the depth information of each pixel in the preset area with the center point as the reference point as the depth information of multiple pixels in the first facial area. The preset area can be pre-set, for example, an N pixel * N pixel area with the center point as the center (or a vertex therein), such as 8 * 8 pixels, 16 * 16 pixels; for example, a circular area with the center point as the center and N pixels as the radius. In this way, the processor can form a preset area after obtaining the center point of the first facial area, and use the depth information of each pixel in the preset area as the depth information of multiple pixels in the first facial area.

[0130] In step 42, the depth information is input into a preset classification model to obtain a face type output by the preset classification model, where the face type includes a real face and a fake face.

[0131] In this step, a preset classification model is stored in the electronic device. The preset classification model can be implemented using a neural network model, such as a Densenet model. The network model can be selected according to the specific scenario, which is not limited here. The preset classification model is preset to complete training, and its input data is the depth information of multiple pixels. The depth information of the multiple pixels is different and can include 3D structural information of the face in the first facial area. For example, the 3D structural information may include facial features, etc.; the output data of the preset classification model is the facial type of whether the face in the first facial area is a real face or a false face.

[0132] The training process of the preset classification model includes:

[0133] First, multiple training sample images are obtained. In one example, the method for obtaining the training sample images can refer to the method for obtaining the first facial area. On this basis, the first facial area is annotated and the facial type is added, thereby obtaining a training sample image containing pixel data (RGB data), depth information and facial type. In another example, an original depth image containing a face can be obtained and annotated to add a facial type, thereby obtaining a training sample image containing pixel data (RGB data), depth information and facial type. Secondly, each training sample image is input into a preset classification model in turn to obtain the facial type output by the preset classification model, and the success rate of the facial type output by the preset classification model and the annotated facial type is counted; when the success rate is higher than the preset success rate, it is determined that the preset classification model has completed training.

[0134] It should be noted that the training method of the preset classification model can be adjusted according to the specific scenario. If it is able to identify whether a face is real or fake, the corresponding solution falls within the protection scope of the present disclosure.

[0135] In this step, the processor can call the above-mentioned preset classification model and input the depth information (and the pixel data in the first facial area) into the preset classification model to obtain the facial type output by the preset classification model, that is, whether the face in each first facial area is a real face or a false face.

[0136] In step 43, the first facial region whose facial type is a real face is determined as a candidate facial region containing a real face.

[0137] In step 32, the candidate facial region is subjected to size recognition processing to obtain a second facial region containing a real face.

[0138] In this embodiment, there may be misidentified areas in the candidate facial region, that is, a depth area where a false face may be identified as a real face. Therefore, the processor can perform size recognition processing on the candidate facial region to obtain a second facial region containing a real face, see Figure 5 , including steps 51 to 53.

[0139] In step 51, the actual size of the face in the candidate face region is obtained.

[0140] In this step, the processor can obtain the actual size of the face in the candidate facial region. The processor can obtain the shooting parameters corresponding to the original image, such as the object focal length f and the distance s'. Then, the processor can determine the actual size of the face in the candidate facial region based on the above shooting parameters.

[0141] It is understandable that the camera lens is a group of lenses. When light parallel to the main optical axis passes through the lens, the light converges to a point, namely the focus. The distance from the focus to the center of the lens is called the focal length f. In this example, the camera lens is equivalent to a convex lens. The photosensitive element is near the focus of the convex lens. When the focal length is approximated as the distance from the center of the convex lens to the photosensitive element, it becomes a pinhole imaging model, such as Figure 6 shown.

[0142] Figure 6 In the equation, AB represents the real height of the object, A'B' represents the height of the object in the original image, s represents the object distance, s' represents the phase distance, f represents the object focal length, and f' represents the phase focal length. It can be understood that the focal length of pinhole imaging refers to the image focal length f'. In practical applications, the object focal length and image focal length in the pinhole imaging model are equal in size but opposite in sign. According to the Gaussian formula of lens imaging, as shown in equation (1).

[0143]

[0144] It should be noted that during the shooting process, the light undergoes a transformation process from the world coordinate system, the camera coordinate system, and the image coordinate system to the pixel coordinate system, and finally projects the face to the pixel point at the corresponding position of the camera's image sensor. Therefore, the face size in the world coordinate system, that is, the actual size of the face, can be reversely deduced through the pixel position and size of the face in the candidate face area.

[0145] In this embodiment, when the candidate facial region is known, it can be converted to the depth information in the camera coordinate system, so it is necessary to find the relationship between the facial recognition frame and the facial size in the camera coordinate system. Figure 7 , the origin of the camera coordinate system is O (assumed to be the center point of the image sensor), the center point of the image frame is c, a point P (x, y, z) in the camera coordinate system is projected to the point P' = (x', y') in the image, according to similar triangles:

[0146]

[0147] The matrix is ​​represented as:

[0148]

[0149] See also Figure 8 , O 1 -xy represents the image coordinate system, the coordinates are continuous; 0 -uv represents the pixel coordinate system. The coordinates can only be integers greater than zero. The coordinates of the center pixel are (u 0 ,v 0), each pixel is slightly different depending on the image sensor, and the length and width of the pixel are dx and dy, which are the actual size of the pixel. The transformation from the image coordinate system to the pixel coordinate system is shown in equations (4) and (5):

[0150]

[0151]

[0152] The matrix is ​​represented as:

[0153]

[0154] Based on the above formula, the conversion formula between the coordinate position in the camera coordinate system and the coordinate position in the pixel coordinate system is shown in formula (7).

[0155]

[0156] In formula (7), the number of camera intrinsic parameters is 5, and Zc is the depth data of the center point of the candidate depth image.

[0157] In this embodiment, the horizontal and vertical coordinates of the face recognition frame in the pixel coordinate system and the face in the camera coordinate system are found. Then, the pixel coordinates of the upper left corner and the lower right corner of the face recognition frame are substituted into formula (7) to obtain the horizontal and vertical coordinates in the camera coordinate system, and Δx, Δy are obtained to obtain the current actual face size H cur *W cur :

[0158] H cur =Δy; (8)

[0159] W cur =Δx; (9)

[0160] At this point, the processor can calculate the actual face size corresponding to the face recognition frame as H cur *W cur .

[0161] In step 52, the facial type of the candidate facial region is determined according to the actual size and the preset size range, and the facial type includes a real face and a false face.

[0162] In this step, the processor can obtain the preset size range [H norm，Wnorm For example, the face range of a girl with a height of 165 cm is H norm =19.5-22cm, face width is W norm= 12-13.6 cm. In this way, the preset size range of the face of male and female with a preset height range can be determined in this step. It should be noted that the above preset size range can be determined according to the target object and with reference to the above female method, and the corresponding solution falls within the protection scope of the present disclosure.

[0163] In one example, the processor obtains a preset size range, including: the processor can perform object type recognition on the face in the candidate facial area, so as to determine the target object type corresponding to the face in the candidate facial area, such as whether it is a male or a female when the target object is a human body, or an animal type when the target object is an animal. For example, when the processor uses a preset classification model to perform facial type recognition, it can simultaneously identify the object type, such as male or female, which can be directly called here. It is understandable that when training the above-mentioned preset classification model, the annotated data of the training sample is further added with an object type label, such as a male or female label, and the other training processes are similar to the scheme of the above-mentioned embodiment. After determining the object type of the face in the candidate facial area, a preset size range corresponding to the object type can be selected, such as a preset size range for males or a preset size range for females.

[0164] In another example, a preset height recognition model is set in the electronic device, and the processor can call the preset debugging recognition model to recognize the original image to obtain the height of each target object, such as the height of a human body. It should be noted that the preset height recognition model can include two sub-models. The first sub-model first detects the feet or feet in the original image. When the feet are detected, the height of the target object where the feet are located is recognized. If the feet are not detected, there is no need to perform height recognition, thereby reducing the amount of data processing. The preset size range corresponding to the target object at each height is stored in the electronic device, and the processor queries the matching preset size range according to the height and type.

[0165] It should be noted that two schemes for obtaining a preset size range are described in this embodiment. Those skilled in the art can select a corresponding scheme according to specific scenarios. If a preset size scheme can be determined, the corresponding scheme falls within the protection scope of this disclosure.

[0166] In this step, the processor can compare the actual size with the preset size range to determine whether the actual size is within the preset size range to determine whether the face is a real face.

[0167]

[0168] In formula (10), isRealFace cur Indicates the face type, True indicates a real face, False indicates a fake face, H norm and W normRespectively represent the height range and width range, H cur and W cur Respectively represent the height and width of the actual size of the face.

[0169] In step 53, the candidate facial regions are screened according to the facial type to obtain a second facial region containing a real face.

[0170] In this step, the face size recognition can be used to exclude faces whose actual face size is too small or too large, and the candidate face area containing the false face can be eliminated to obtain the accurate second face area containing the real face.

[0171] In step 13, a corresponding operation is performed according to the second facial area.

[0172] In one embodiment, after determining the second facial area, the processor may also send the external frame of the second facial area as an identification frame as a display object to the display module. At this time, the display module may display the identification frame of the second facial area to improve the accuracy of facial recognition.

[0173] In one embodiment, the processor may also focus on the next frame of the original image according to the recognition frame of the second facial area to obtain the recognition frame corresponding to the next frame of the original image.

[0174] In this step, the processor can determine the target position of the lens according to the shooting parameters corresponding to the next frame of the original image, the depth information of each pixel in the second facial area, and the actual size of the face in the second facial area; then, the processor can adjust the lens to the target position; thereafter, generate an identification frame at the second facial area in the original image, and obtain an identification frame containing the real face in the original image. Fig. 9 , assuming that the initial position P1 of the lens can be moved to the target position P2, so as to obtain a more accurate facial area and recognition frame.

[0175] Thus, this embodiment can eliminate false faces in the shooting scene and obtain real faces through face size recognition processing, thereby improving the accuracy of face recognition during the shooting process and enhancing the shooting experience.

[0176] The following describes the process of focusing in a shooting scene using a focusing method provided by an embodiment of the present disclosure in conjunction with the accompanying drawings. Fig.10 ,include:

[0177] After the camera application is turned on, an original image is collected, and the processor can perform face recognition on the original image to obtain an external frame of at least one face area in the original image, namely, an initial recognition frame.

[0178] The processor may use the disparity recognition technology (or the phase detection technology PD) to acquire the first depth information.

[0179] The processor can control the TOF sensor to obtain depth information of the nearest point in the shooting scene when shooting the original image.

[0180] The processor may use the depth information collected by the TOF sensor to perform correction processing on the first depth information to obtain the second depth information.

[0181] The processor may perform pixel matching on the original image and the second depth information to obtain target depth information corresponding to the original image, and then obtain the target depth information within the identification frame corresponding to the original image to obtain the first facial region. Alternatively, the processor may crop the original image according to the initial identification frame, and perform correction processing on the cropped image portion using the second depth information to obtain the first facial region. It is understandable that the number of the first facial regions is the same as the number of the initial identification frames.

[0182] The processor can obtain the center point of each first facial region respectively, and obtain the depth data of multiple pixels in a preset region with the center point as a reference point. Then, the processor can input the depth data of the multiple pixels into a preset classification model to obtain the face type of the face in each first facial region, and obtain the candidate facial region containing the face type that is a real face. In this step, the false faces in the first facial region are filtered out, so that the accuracy of the face in the candidate facial region being a real face is greatly improved.

[0183] After obtaining the candidate facial region, the processor can obtain the shooting parameters of the camera when shooting the original image, and determine the actual size of the face according to the shooting parameters and the size of the initial recognition frame. Then, the processor can determine whether the actual size is within the preset size range. If the height (Hur) of the actual size is outside the preset size range (Hnorm) and the width (Wur) of the actual size is outside the preset size range (Wnorm), it is determined that the face in the candidate facial region is a false face, and the candidate facial region can be discarded; if the height (Hur) of the actual size is within the preset size range (Hnorm) and the width (Wur) of the actual size is within the preset size range (Wnorm), it is determined that the face in the candidate facial region is a real face, and a second facial region is obtained. In this step, false faces that are mistakenly identified as real faces in the candidate facial region can be eliminated, thereby improving the accuracy of facial recognition.

[0184] In the process of determining the second facial region, the first facial region is actually screened twice to obtain the second facial region and the external bounding box corresponding to the second facial region. The processor can reposition the original image according to the identification box corresponding to the second facial region and display the remaining identification box on the display screen. In an example, the processor can focus the next frame of the image according to the second facial region to obtain the identification box corresponding to the next frame of the original image.

[0185] It should be noted that, considering that there may be multiple initial recognition frames, the processor can use a loop iteration method to recognize and judge the faces of each initial recognition frame in turn, such as pixel depth data association, depth correction, face type recognition, shooting parameter acquisition, actual face judgment, etc. If any recognition judgment fails, it will jump to the face judgment of the next initial recognition face until all recognition frames are recognized, and the target depth image and its recognition frame are obtained.

[0186] In the present disclosure, a focusing method is provided. The present disclosure also provides a focusing device. Fig.11 , the device comprises:

[0187] A first region acquisition module 111, configured to obtain a first facial region according to depth information of at least one facial region in an original image;

[0188] A second region acquisition module 112, configured to perform facial size recognition on the first facial region to obtain a second facial region;

[0189] The operation execution module 113 is used to execute a corresponding operation according to the second facial area.

[0190] In one embodiment, the second region acquisition module includes:

[0191] A candidate region acquisition submodule, configured to perform facial classification and recognition processing on the first facial region to obtain a candidate facial region containing a real face;

[0192] The second region acquisition submodule is used to perform size recognition processing on the candidate facial region to obtain a second facial region containing a real face.

[0193] In one embodiment, the candidate region acquisition submodule includes:

[0194] A depth information acquisition unit, configured to acquire depth information of a plurality of pixels in the first facial region;

[0195] A depth information acquisition unit, used to input the depth information into a preset classification model to obtain a face type output by the preset classification model, wherein the face type includes a real face and a fake face;

[0196] The candidate region acquisition unit is used to determine the first facial region whose facial type is a real face as a candidate facial region containing the real face.

[0197] In one embodiment, the depth information acquisition unit includes:

[0198] A center point acquisition subunit, used to acquire the center point of the first facial area;

[0199] The depth information acquisition subunit is used to acquire the depth information of each pixel point in a preset area with the central point as a reference point as the depth information of multiple pixel points in the first facial area.

[0200] In one embodiment, the second region acquisition submodule includes:

[0201] An actual size acquisition unit, used to acquire the actual size of the face in the candidate face area;

[0202] A face type acquisition unit, configured to determine a face type of the candidate face region according to the actual size and a preset size range, wherein the face type includes a real face and a false face;

[0203] The second region acquisition unit is used to filter the candidate facial regions according to the facial type to obtain a second facial region containing a real face.

[0204] In one embodiment, the actual size acquisition unit includes:

[0205] A shooting parameter acquisition subunit, used to acquire shooting parameters corresponding to the original image;

[0206] The actual size acquisition subunit is used to determine the actual size of the face in the candidate face area according to the shooting parameters.

[0207] In one embodiment, the first region acquisition module includes:

[0208] The target information acquisition submodule is used to acquire the target depth information corresponding to the original image;

[0209] The first region acquisition submodule is used to acquire the facial region within each initial recognition frame in the target depth information, and obtain the first facial region corresponding to each initial recognition frame.

[0210] In one embodiment, the target information acquisition submodule includes:

[0211] A first information acquisition unit, configured to acquire first depth information corresponding to the original image;

[0212] A pixel information acquisition unit, used to acquire depth information of at least one pixel in the original image;

[0213] A second information acquisition unit, configured to perform correction processing on the first depth information based on the depth information of the at least one pixel point to obtain second depth information;

[0214] The target information acquisition unit is used to align the second depth information with the original image to obtain target depth information corresponding to the original image.

[0215] In one embodiment, the first information acquisition unit includes:

[0216] An image block division subunit, used for dividing the original image into a plurality of image blocks;

[0217] A phase difference acquisition subunit, used to acquire the phase difference corresponding to each image block according to the parallax ranging method;

[0218] The first information acquisition subunit is used to acquire the depth information of each pixel in the original image according to the phase difference, so as to obtain the first depth information corresponding to the original image.

[0219] In one embodiment, the pixel information acquisition unit includes:

[0220] A partition division subunit, used to divide the shooting area corresponding to the original image into at least one partition;

[0221] The depth information acquisition subunit is used to acquire the depth information of the nearest point of each partition in the at least one partition as the depth information of at least one pixel point in the original image.

[0222] In one embodiment, the operation execution module includes:

[0223] The recognition frame acquisition submodule is used to perform a focusing operation on the next frame of image according to the second facial area to obtain a recognition frame corresponding to the next frame of original image.

[0224] It should be noted that the device embodiment shown in this embodiment matches the content of the above method embodiment, and the content of the above method embodiment can be referred to, which will not be repeated here.

[0225] Fig.12 1 is a block diagram of an electronic device according to an exemplary embodiment. For example, the electronic device 1200 may be a smart phone, a computer, a digital broadcast terminal, a tablet device, a medical device, a fitness device, a personal digital assistant, etc.

[0226] Reference Fig.12The electronic device 1200 may include one or more of the following components: a processing component 1202 , a memory 1204 , a power component 1206 , a multimedia component 1208 , an audio component 1210 , an input / output (I / O) interface 1212 , a sensor component 1214 , a communication component 1216 , and an image acquisition component 1218 .

[0227] The processing component 1202 generally controls the overall operation of the electronic device 1200, such as operations associated with display, phone calls, data communications, camera operations, and recording operations. The processing component 1202 may include one or more processors 1220 to execute computer programs. In addition, the processing component 1202 may include one or more modules to facilitate interaction between the processing component 1202 and other components. For example, the processing component 1202 may include a multimedia module to facilitate interaction between the multimedia component 1208 and the processing component 1202. In an example, the processor 1220 may implement the scheme of the above-mentioned power supply control method.

[0228] The memory 1204 is configured to store various types of data to support operations on the electronic device 1200. Examples of such data include computer programs for any application or method operating on the electronic device 1200, contact data, phone book data, messages, pictures, videos, etc. The memory 1204 can be implemented by any type of volatile or non-volatile storage device or a combination thereof, such as static random access memory (SRAM), electrically erasable programmable read-only memory (EEPROM), erasable programmable read-only memory (EPROM), programmable read-only memory (PROM), read-only memory (ROM), magnetic memory, flash memory, magnetic disk or optical disk.

[0229] The power supply component 1206 provides power to various components of the electronic device 1200. The power supply component 1206 may include a power management system, one or more power supplies, and other components associated with generating, managing, and distributing power for the electronic device 1200. The power supply component 1206 may include a power supply chip, and the controller may communicate with the power supply chip to control the power supply chip to turn on or off the first switching device, so that the battery supplies power to the mainboard circuit or does not supply power.

[0230] The multimedia component 1208 includes a screen that provides an output interface between the electronic device 1200 and the target object. In some embodiments, the screen may include a liquid crystal display (LCD) and a touch panel (TP). If the screen includes a touch panel, the screen may be implemented as a touch screen to receive input information from the target object. The touch panel includes one or more touch sensors to sense touch, slide, and gestures on the touch panel. The touch sensor may not only sense the boundaries of a touch or slide action, but also detect the duration and pressure associated with the touch or slide operation.

[0231] The audio component 1210 is configured to output and / or input audio file information. For example, the audio component 1210 includes a microphone (MIC), and when the electronic device 1200 is in an operation mode, such as a call mode, a recording mode, and a speech recognition mode, the microphone is configured to receive external audio file information. The received audio file information can be further stored in the memory 1204 or sent via the communication component 1216. In some embodiments, the audio component 1210 also includes a speaker for outputting audio file information.

[0232] The I / O interface 1212 provides an interface between the processing component 1202 and a peripheral interface module, which may be a keyboard, a click wheel, a button, etc.

[0233] The sensor component 1214 includes one or more sensors for providing various aspects of status assessment for the electronic device 1200. For example, the sensor component 1214 can detect the open / closed state of the electronic device 1200, the relative positioning of components, such as the display screen and keypad of the electronic device 1200, and the sensor component 1214 can also detect the position change of the electronic device 1200 or a component, the presence or absence of contact between the target object and the electronic device 1200, the orientation or acceleration / deceleration of the electronic device 1200, and the temperature change of the electronic device 1200. In this example, the sensor component 1214 can include a magnetic sensor, a gyroscope, and a magnetic field sensor, and can also include an inertial sensor, an image sensor, etc., wherein the magnetic field sensor includes at least one of the following: a Hall sensor, a thin film magnetoresistive sensor, and a magnetic liquid acceleration sensor.

[0234] The communication component 1216 is configured to facilitate wired or wireless communication between the electronic device 1200 and other devices. The electronic device 1200 can access a wireless network based on a communication standard, such as WiFi, 2G, 3G, 4G, 5G, or a combination thereof. In an exemplary embodiment, the communication component 1216 receives broadcast information or broadcast-related information from an external broadcast management system via a broadcast channel. In an exemplary embodiment, the communication component 1216 also includes a near field communication (NFC) module to facilitate short-range communication. For example, the NFC module can be implemented based on radio frequency identification (RFID) technology, infrared data association (IrDA) technology, ultra-wideband (UWB) technology, Bluetooth (BT) technology and other technologies.

[0235] In an exemplary embodiment, the electronic device 1200 may be implemented by one or more application-specific integrated circuits (ASICs), digital signal processors (DSPs), digital signal processing devices (DSPDs), programmable logic devices (PLDs), field programmable gate arrays (FPGAs), controllers, microcontrollers, microprocessors, or other electronic components.

[0236] In an exemplary embodiment, the present disclosure also provides a non-transitory computer-readable storage medium, and when an executable computer program in the storage medium is executed by a processor, the above-mentioned method can be implemented.

[0237] Those skilled in the art will readily appreciate other embodiments of the present disclosure after considering the specification and practicing the disclosure disclosed herein. The present disclosure is intended to cover any variations, uses, or adaptations that follow the general principles of the present disclosure and include common knowledge or customary techniques in the art that are not disclosed in the present disclosure. The description and examples are to be considered exemplary only, and the true scope and spirit of the present disclosure are indicated by the following claims.

[0238] It should be understood that the present disclosure is not limited to the exact structures that have been described above and shown in the drawings, and that various modifications and changes may be made without departing from the scope thereof. The scope of the present disclosure is limited only by the appended claims.

Claims

1. A focusing method, characterized in that: include: Obtaining a first facial region according to depth information of at least one facial region in the original image; Performing facial size recognition on the first facial region to obtain a second facial region; A corresponding operation is performed according to the second facial region.

2. The focusing method according to claim 1, characterized in that: Performing facial size recognition on the first facial region to obtain a second facial region includes: Performing facial classification and recognition processing on the first facial region to obtain a candidate facial region containing a real face; The candidate facial region is processed for size recognition to obtain a second facial region containing a real face.

3. The focusing method according to claim 2, characterized in that: Performing facial classification and recognition processing on the first facial region to obtain a candidate facial region containing a real face includes: Acquire depth information of a plurality of pixels in the first facial region; Inputting the depth information into a preset classification model to obtain a face type output by the preset classification model, wherein the face type includes a real face and a fake face; A first facial region whose facial type is a real face is determined as a candidate facial region containing a real face.

4. The focusing method according to claim 3, characterized in that: Acquiring depth information of a plurality of pixels in the first facial region includes: Obtaining a center point of the first facial area; Depth information of each pixel in a preset area with the center point as a reference point is obtained as depth information of multiple pixels in the first facial area.

5. The focusing method according to claim 3, characterized in that: Performing size recognition processing on the candidate facial region to obtain a second facial region containing a real face, including: Obtaining the actual size of the face in the candidate face area; Determine a face type of the candidate face region according to the actual size and a preset size range, wherein the face type includes a real face and a false face; The candidate facial regions are screened according to the facial type to obtain a second facial region containing a real face.

6. The focusing method according to claim 5, characterized in that: Obtaining the actual size of the face in the candidate face region includes: Obtaining shooting parameters corresponding to the original image; The actual size of the face in the candidate face area is determined according to the shooting parameters.

7. The focusing method according to claim 1, characterized in that: Obtaining a first facial region according to depth information of at least one facial region in an original image includes: Obtaining target depth information corresponding to the original image; Acquire the facial regions within each initial recognition frame in the target depth information to obtain the first facial regions corresponding to each initial recognition frame.

8. The focusing method according to claim 7, characterized in that: Obtaining target depth information corresponding to the original image, including: Acquire first depth information corresponding to the original image; Acquire depth information of at least one pixel in the original image; Correcting the first depth information based on the depth information of the at least one pixel to obtain second depth information; The second depth information is aligned with the original image to obtain target depth information corresponding to the original image.

9. The focusing method according to claim 8, characterized in that: Acquiring first depth information corresponding to the original image, including: Dividing the original image into a plurality of image blocks; Obtaining the phase difference corresponding to each image block according to the parallax ranging method; Depth information of each pixel in the original image is acquired according to the phase difference to obtain first depth information corresponding to the original image.

10. The focusing method according to claim 8, characterized in that: Acquiring depth information of at least one pixel in the original image, including: Dividing the shooting area corresponding to the original image into at least one partition; Depth information of the nearest point of each partition in the at least one partition is obtained as depth information of at least one pixel point in the original image.

11. The focusing method according to claim 1, characterized in that: Performing a corresponding operation according to the second facial region includes: A focusing operation is performed on the next frame of image according to the second facial region to obtain an identification frame corresponding to the next frame of original image.

12. A focusing device, characterized in that: include: A first region acquisition module, configured to obtain a first facial region according to depth information of at least one facial region in an original image; A second region acquisition module, configured to perform facial size recognition on the first facial region to obtain a second facial region; An operation execution module is used to execute a corresponding operation according to the second facial area.

13. An electronic device, characterized in that: include: Processor and memory; The memory is used to store a computer program executable by the processor; The processor is configured to execute the computer program in the memory to implement the method according to any one of claims 1 to 11.

14. A non-transitory computer-readable storage medium, characterized in that: When the executable computer program in the storage medium is executed by a processor, the method according to any one of claims 1 to 11 can be implemented.

Images