Image processing method, device, equipment and storage medium

Abstract

The present application provides an image processing method, device, equipment and storage medium, which relate to the field of face recognition technology. The method includes determining whether a face area is contained in an image to be processed, and the image to be processed is an image containing an object to be recognized; if the image to be processed contains a face area, initial depth information of the face area is obtained according to the face area and an initial reference image, and a two-dimensional image of the face area is obtained; a first target reference image is determined according to the initial depth information of the face area; target depth information of the face area is calculated according to the first target reference image and the face area to obtain a three-dimensional image of the face area; the object to be recognized is recognized based on the three-dimensional image of the face area and the two-dimensional image of the face area. By applying the embodiments of the present application, the accuracy of recognizing the object to be recognized can be improved.

Description

Technical Field

[0001] The present application relates to the field of face recognition technology, and in particular to an image processing method, device, equipment and storage medium. Background Art

[0002] With the development of science and technology, especially in the field of face recognition, such as the rise of face unlocking, face payment and other functions, devices embedded with 3D structured light modules (such as mobile phones, tablets, smart door locks, etc.) have been widely used. Among them, 3D structured light modules are combined with processors to perform three-dimensional imaging of the object to be identified.

[0003] At present, the processor directly calculates the depth data of the entire area on the image of the object to be identified obtained by using the 3D structured light module, and determines which reference image the object to be identified is closer to based on the distance obtained from the depth data. The nearest reference image is used as the target reference image, and finally, the object to be identified can be three-dimensionally imaged according to the target reference image, that is, the object to be identified is identified.

[0004] However, when recognizing a face, the face may be tilted forward, which will cause the face and body in the image of the object to be recognized to be not in the same plane. If the target reference image is directly obtained using the entire area on the image of the object to be recognized, the target reference image will be selected inaccurately, thereby reducing the accuracy of recognizing the object to be recognized. Summary of the invention

[0005] The purpose of the present application is to provide an image processing method, device, equipment and storage medium to address the deficiencies in the above-mentioned prior art, which can accurately select a target reference image and thereby improve the accuracy of identifying the object to be identified.

[0006] In a first aspect, an embodiment of the present application provides an image processing method, which is applied to an electronic device, and the method includes:

[0007] Determining whether the image to be processed contains a face area, wherein the image to be processed is an image containing an object to be identified;

[0008] If the image to be processed includes a face region, initial depth information of the face region is obtained according to the face region and an initial reference image, and a two-dimensional image of the face region is acquired, wherein the initial reference image is one of the multiple reference images, the multiple reference images are pre-stored in the electronic device, each reference image corresponds to a different distance parameter, and the distance parameter is used to characterize the distance between the object included in the reference image and the electronic device;

[0009] Determine a first target reference image according to the initial depth information of the face area;

[0010] Calculating target depth information of the face region according to the first target reference image and the face region to obtain a three-dimensional image of the face region;

[0011] The object to be identified is identified based on the three-dimensional image of the face area and the two-dimensional image of the face area.

[0012] Optionally, obtaining initial depth information of the face region according to the face region and an initial reference image includes:

[0013] Determining whether the face region is credible according to a face frame corresponding to the face region and a preset threshold;

[0014] If the face region is credible, initial depth information of the face region is obtained according to the face region and an initial reference image.

[0015] Optionally, the method further comprises:

[0016] If the face area is not credible, initial depth information of the image to be processed is obtained according to an optional area on the image to be processed and the initial reference image, wherein the optional area includes the face area and an area outside the face area.

[0017] Optionally, determining the first target reference image according to the initial depth information of the face region includes:

[0018] Determine the distance between the face part of the object to be identified and the electronic device according to the initial depth information of the face area;

[0019] A first target reference image is determined according to the distance between the face part of the object to be identified and the electronic device and the distance parameters corresponding to the reference images.

[0020] Optionally, determining the distance between the face part of the object to be identified and the electronic device according to the initial depth information of the face area includes:

[0021] According to the initial depth information of the face area, acquiring the depth data corresponding to the face area and the number of the depth data;

[0022] The distance between the face part of the object to be identified and the electronic device is determined according to the depth data corresponding to the face area and the number of the depth data.

[0023] Optionally, the method further comprises:

[0024] If the image to be processed does not contain the face area, initial depth information of the image to be processed is obtained according to the area on the image to be processed and the initial reference image, and a two-dimensional image of the object to be identified is obtained;

[0025] Determining the distance between the object to be identified and the electronic device according to the initial depth information of the image to be processed;

[0026] Obtaining a second target reference image according to the distance between the object to be identified and the electronic device and the distance parameters corresponding to each reference image;

[0027] Calculating target depth information of the image to be processed according to the second target reference image and the image to be processed to obtain a three-dimensional image of the object to be identified;

[0028] The object to be identified is identified based on the three-dimensional image of the object to be identified and the two-dimensional image of the object to be identified.

[0029] Optionally, determining the distance between the object to be identified and the electronic device according to the initial depth information of the image to be processed includes:

[0030] Acquire initial depth information corresponding to a central area of ​​the image to be processed according to the initial depth information of the image to be processed;

[0031] The distance between the object to be identified and the device is determined according to initial depth information corresponding to the central area of ​​the image to be processed.

[0032] In a second aspect, an embodiment of the present application further provides an image processing device, which is applied to an electronic device, and includes:

[0033] A judgment module, used to judge whether the image to be processed contains a face area, wherein the image to be processed is an image containing an object to be identified;

[0034] A first determination module, configured to obtain initial depth information of the face region according to the face region and an initial reference image, and acquire a two-dimensional image of the face region if the image to be processed includes a face region, wherein the initial reference image is one of the multiple reference images, the multiple reference images are pre-stored in the electronic device, each reference image corresponds to a different distance parameter, and the distance parameter is used to characterize the distance between the object included in the reference image and the electronic device;

[0035] A second determination module, configured to determine a first target reference image according to the initial depth information of the face region;

[0036] A calculation module, configured to calculate target depth information of the face region according to the first target reference image and the face region to obtain a three-dimensional image of the face region;

[0037] The recognition module is used to recognize the object to be recognized based on the three-dimensional image of the face area and the two-dimensional image of the face area.

[0038] Optionally, the first determination module is specifically used to determine whether the face area is credible based on a face frame corresponding to the face area and a preset threshold; if the face area is credible, obtaining initial depth information of the face area based on the face area and an initial reference image.

[0039] Optionally, the first determination module is further specifically used to obtain initial depth information of the image to be processed based on an optional area on the image to be processed and the initial reference image if the face area is not credible, and the optional area includes the face area and an area outside the face area.

[0040] Optionally, the second determination module is specifically used to determine the distance between the facial part of the object to be identified and the electronic device based on the initial depth information of the facial area; and determine the first target reference image based on the distance between the facial part of the object to be identified and the electronic device and the distance parameters corresponding to each reference image.

[0041] Optionally, the second determination module is further specifically used to obtain depth data corresponding to the facial area and the number of depth data based on the initial depth information of the facial area; and determine the distance between the facial part of the object to be identified and the electronic device based on the depth data corresponding to the facial area and the number of depth data.

[0042] Optionally, the first determination module is further configured to obtain initial depth information of the image to be processed according to the area on the image to be processed and the initial reference image, and acquire a two-dimensional image of the object to be identified if the image to be processed does not contain the face area;

[0043] The second determination module is further used to determine the distance between the object to be identified and the electronic device according to the initial depth information of the image to be processed; and obtain a second target reference image according to the distance between the object to be identified and the electronic device and the distance parameters corresponding to each reference image;

[0044] The calculation module is further used to calculate the target depth information of the image to be processed according to the second target reference image and the image to be processed, so as to obtain a three-dimensional image of the object to be identified;

[0045] The recognition module is further used to recognize the object to be recognized based on the three-dimensional image of the object to be recognized and the two-dimensional image of the face area.

[0046] Optionally, the second determination module is further specifically used to obtain initial depth information corresponding to the central area of ​​the image to be processed based on the initial depth information of the image to be processed; and determine the distance between the object to be identified and the device based on the initial depth information corresponding to the central area of ​​the image to be processed.

[0047] In a third aspect, an embodiment of the present application provides an electronic device, comprising: a processor, a storage medium and a bus, wherein the storage medium stores machine-readable instructions executable by the processor, and when the electronic device is running, the processor and the storage medium communicate via the bus, and the processor executes the machine-readable instructions to perform the steps of the image processing method of the first aspect above.

[0048] In a fourth aspect, an embodiment of the present application provides a storage medium having a computer program stored thereon, and when the computer program is executed by a processor, the steps of the image processing method of the first aspect are executed.

[0049] The beneficial effects of this application are:

[0050] The embodiments of the present application provide an image processing method, apparatus, device and storage medium, which may include: determining whether an image to be processed contains a face area, where the image to be processed is an image containing an object to be identified; if the image to be processed contains a face area, obtaining initial depth information of the face area according to the face area and an initial reference image, and acquiring a two-dimensional image of the face area, wherein the initial reference image is one of a plurality of reference images, the plurality of reference images are pre-stored in an electronic device, each reference image corresponds to a different distance parameter, and the distance parameter is used to characterize the distance between the object contained in the reference image and the electronic device; determining a first target reference image according to the initial depth information of the face area; calculating target depth information of the face area according to the first target reference image and the face area to obtain a three-dimensional image of the face area; and identifying the object to be identified based on the three-dimensional image of the face area and the two-dimensional image of the face area.

[0051] Using the image processing method provided by the embodiment of the present application, after acquiring the image to be processed, the electronic device will first determine whether the image to be processed contains a face area. If the image to be processed contains a face area, the electronic device will determine a first target reference image from multiple reference images pre-stored thereon based on the face area and the initial reference image, and finally obtain a three-dimensional image of the face area based on the first target reference image and the face area, and then identify the object to be identified contained in the image to be processed. In other words, when the image to be processed contains a face area, the first target reference image is determined only based on the face area. In this way, the phenomenon of poor selection of the target reference image can be avoided when the face on the object to be identified appears to be leaning forward in the face recognition scene. That is, under the premise of accurately selecting the target reference image, the accuracy of identifying the object to be identified can be improved. BRIEF DESCRIPTION OF THE DRAWINGS

[0052] In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the drawings required for use in the embodiments are briefly introduced below. It should be understood that the following drawings only show certain embodiments of the present invention and therefore should not be regarded as limiting the scope. For ordinary technicians in this field, other related drawings can be obtained based on these drawings without creative work.

[0053] Figure 1 A schematic diagram of a structure of an electronic device supporting face recognition provided in an embodiment of the present application;

[0054] Figure 2 A flowchart of an image processing method provided in an embodiment of the present application;

[0055] Figure 3 A flowchart of another image processing method provided in an embodiment of the present application;

[0056] Figure 4 A flowchart of another image processing method provided in an embodiment of the present application;

[0057] Figure 5 A flowchart of another image processing method provided in an embodiment of the present application;

[0058] Figure 6 A schematic diagram of the structure of an image processing device provided in an embodiment of the present application;

[0059] Figure 7 Another structural schematic diagram of an electronic device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0060] In order to make the purpose, technical solutions and advantages of the embodiments of the present application clearer, the technical solutions in the embodiments of the present application will be clearly and completely described below in combination with the drawings in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, not all of the embodiments.

[0061] Therefore, the following detailed description of the embodiments of the present application provided in the accompanying drawings is not intended to limit the scope of the invention claimed for protection, but merely represents selected embodiments of the present application. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present application.

[0062] It should be noted that similar reference numerals and letters denote similar items in the following drawings, and therefore, once an item is defined in one drawing, further definition and explanation thereof is not required in subsequent drawings.

[0063] Before explaining the embodiments of the present application in detail, the application scenario of the present application is first introduced. The application scenario can specifically be a scenario of face recognition, such as face payment, face unlocking, etc., which is not explained in this application. The method of the present application can be applied to electronic devices that support face recognition. Figure 1 A schematic diagram of a structure of an electronic device supporting face recognition provided in an embodiment of the present application, such as Figure 1 As shown, the electronic device may include a structured light module 101 and a processor 102, wherein the structured light module 101 is connected to the processor 102, wherein the structured light module 101 may include a transmitter and a collector, wherein the transmitter may specifically be a laser projector, wherein the transmitter is used to project structured light onto the surface of the object to be identified, and the collector is used to photograph the object to be identified through a camera to obtain a structured light image, i.e., an image to be processed, wherein the camera may include an infrared camera and an RGB camera. The collector may send the collected image to be processed to the processor 102, and the processor 102 may identify the face area on the image to be processed. When the face area is identified, the target reference image may be determined based on the face area and a preset initial reference image, and then a three-dimensional image of the face area may be obtained based on the target reference image and the face area, thereby identifying the object to be identified.

[0064] The above-mentioned electronic devices may specifically be mobile phones, tablet computers, smart door locks, wearable devices, etc., which are not limited in this application.

[0065] Specifically, in a face unlocking scenario, the electronic device can capture a user image through its camera and identify the user image. When it is identified that the user image contains a face area, a target reference image can be obtained based on the face area, and then the user can be identified based on the face area and the target reference image. When the identification is successful, the electronic device is in an unlocked state.

[0066] The image processing method mentioned in this application is illustrated as follows with reference to the accompanying drawings. Figure 2 FIG. 1 is a flow chart of an image processing method provided in an embodiment of the present application, and the method can be executed by the processor 102 in the above-mentioned image processing system. Figure 2 As shown, the method may include:

[0067] S201, determining whether an image to be processed contains a face region, where the image to be processed is an image containing an object to be recognized.

[0068] Among them, the electronic device can obtain an image to be processed containing an object to be identified through a camera thereon, wherein the object to be identified can be a person or an object of other forms (such as an object, a scene, etc.). It should be noted that this application focuses on the case where the object to be identified is a person. The object to be identified can be in front of the electronic device, and the structured light module embedded in the electronic device can collect the image of the object to be identified as the image to be processed.

[0069] After the processor obtains the image to be processed, it can extract relevant feature data from the image to be processed, and based on the feature data and preset facial feature data, it can be determined whether the image to be processed includes a facial area.

[0070] S202: If the image to be processed includes a face region, initial depth information of the face region is obtained according to the face region and an initial reference image, and a two-dimensional image of the face region is acquired.

[0071] The initial reference image is one of a plurality of reference images, which are pre-stored in the electronic device. Each reference image corresponds to a different distance parameter, which is used to characterize the distance between the object contained in the reference image and the electronic device.

[0072] The structured light module in the electronic device has a certain working range, which can be (0.3m-1.2m). The working range can be determined by the characteristics of the structured light module. Different structured light modules have different working ranges. The reference image is a pre-collected structured light image on a certain plane of the electronic device. Each reference image corresponds to a reference position, and each reference position is represented by a distance parameter. For example, for a structured light module with a working range of (0.3m-1.2m), the corresponding reference images can be obtained from three reference positions respectively, and the distance parameters corresponding to each reference image can be 40cm, 60cm, and 80cm respectively. After the reference image is obtained, the reference image can be associated with its corresponding distance parameter and saved.

[0073] Generally, the initial image is a reference image whose distance parameter satisfies a preset condition among multiple reference images, and the preset condition includes: being in the middle position after sorting the distance parameters according to the numerical value. Continuing with the above example, for example, a reference image with a distance parameter of 60 cm can be taken as the initial reference image.

[0074] When the processor determines that the image to be processed collected by the infrared camera contains a face area, the face area can be brightened and framed with a frame according to the face boundary. Pixel information in the frame and pixel information in the initial image are extracted, the pixel deviation between the two is calculated, and the initial depth information corresponding to the face area can be calculated according to the camera parameters of the electronic device. At the same time, the processor can also extract a two-dimensional image of the face area in the image to be processed collected by the RGB camera in the same way as above.

[0075] S203: Determine a first target reference image according to the initial depth information of the face region.

[0076] The processor can remove invalid depth information from the initial depth information of the face area, calculate the distance between the face of the object to be identified and the electronic device based on the valid depth data on the initial depth image corresponding to the face area and the number of the valid depth data, and use the reference image closest to the face of the object to be identified as the first target reference image. The first target reference image may or may not be the initial reference image. The first target reference image determined by the face area on the image to be processed is more in line with the actual situation, thereby improving the accuracy of identifying the object to be identified.

[0077] S204: Calculate target depth information of the face region according to the first target reference image and the face region to obtain a three-dimensional image of the face region.

[0078] S205: Identify the object to be identified based on the three-dimensional image of the face region and the two-dimensional image of the face region.

[0079] Taking the first target reference image as a reference, pixel information on the first target reference image, pixel information on the face area and camera parameters are obtained, and the target depth information of the face area can be calculated. According to the target depth information, a target depth image of the face area can be constructed. Finally, the target depth image is transformed between the image coordinate system, the camera coordinate system and the world coordinate system to construct a three-dimensional image of the face area.

[0080] In face recognition scenarios, such as face unlocking, the processor can identify the face area based on the three-dimensional image of the face area and the two-dimensional image of the face area, and match the recognition result with the pre-stored unlock image. If the match is successful, it means that the electronic device has successfully identified the object to be identified, that is, the electronic device is in an unlocked state.

[0081] In summary, in the image processing method provided by the present application, after acquiring the image to be processed, the electronic device will first determine whether the image to be processed contains a face area. If the image to be processed contains a face area, the electronic device will determine a first target reference image from multiple reference images pre-stored thereon based on the face area and the initial reference image, and finally obtain a three-dimensional image of the face area based on the first target reference image and the face area, and then identify the object to be identified contained in the image to be processed. That is to say, when the image to be processed contains a face area, the first target reference image is determined only based on the face area. Compared with the method of determining the reference image based on the entire image to be processed in the prior art, this embodiment can avoid the phenomenon of poor selection of the target reference image when the face on the object to be identified appears to be leaning forward in the face recognition scene. That is, under the premise of accurately selecting the target reference image, the accuracy of identifying the object to be identified can be improved.

[0082] Figure 3 A flowchart of another image processing method provided in an embodiment of the present application is shown below. Figure 3 As shown, optionally, the above-mentioned obtaining the initial depth information of the face area according to the face area and the initial reference image includes:

[0083] S301: Determine whether the face region is credible based on a face frame corresponding to the face region and a preset threshold.

[0084] The structured light module on the electronic device has a corresponding working range, which can be (0.3m-1.2m). When the object to be identified is farther away from the electronic device (such as 1.2m), the face frame of the facial part of the object to be identified on the electronic device is smaller; when the object to be identified is farther away from the electronic device (such as 0.3m), the face frame of the facial part of the object to be identified on the electronic device is larger.

[0085] In general, the preset threshold is obtained in advance based on the corresponding face frame sizes when multiple objects to be identified are located at the farthest boundaries of the working range of the structured light module. Specifically, the face frame size corresponding to each object to be identified at the farthest boundary of the working range (such as 1.2m) can be obtained, and the sizes of each face frame can be averaged to obtain the average face frame size, which can be used as the preset threshold. The processor can compare the face frame size corresponding to the face area with the preset threshold, and judge whether the face area is credible based on the comparison result. Since in reality, the size of each person's face is different, the threshold can be obtained based on multiple objects to be identified, which can improve the accuracy of judging the credibility of the face area.

[0086] S302: If the face region is credible, obtain initial depth information of the face region according to the face region and the initial reference image.

[0087] When the face frame size corresponding to the face area is greater than the preset threshold, it indicates that the position of the object to be identified does not exceed the farthest boundary of the working range of the structured light module, that is, the face area is credible. On the premise that the face area is credible, the processor calculates the initial depth information corresponding to the face area based on the pixel information of the face area, the pixel information of the initial reference image, and the camera parameters.

[0088] S303: If the face region is not credible, obtaining initial depth information of the image to be processed according to the optional region on the image to be processed and the initial reference image.

[0089] Wherein, the optional area includes the face area and the area outside the face area. Exemplarily, the optional area may be an area including the entire image to be processed, or the optional area may also be a partial area of ​​the image to be processed that includes the above-mentioned face area and the area outside the face area. When the face frame size corresponding to the face area is smaller than the preset threshold, it indicates that the position of the object to be identified exceeds the farthest boundary of the working range of the structured light module, that is, the face area is unreliable. Under the premise that the face area is unreliable, the processor needs to calculate the initial depth information corresponding to the image to be processed based on the pixel information corresponding to the face area on the image to be processed and the pixel information corresponding to the area outside the face area, the pixel information of the initial reference image, and the camera parameters.

[0090] The basis for acquiring the target reference image is determined by judging whether the face area is credible. That is to say, when the face area is credible, the accuracy of acquiring the target reference image (first target reference image) can be improved based on the face area. On the premise that the accuracy of the target reference image is high, the accuracy of identifying the object to be identified can be further improved.

[0091] Figure 4 A flowchart of another image processing method provided in an embodiment of the present application is shown below. Figure 4 As shown, optionally, the determining of the first target reference image according to the initial depth information of the face region includes:

[0092] S401: Determine the distance between the face part of the object to be identified and the electronic device according to initial depth information of the face area.

[0093] Optionally, based on the initial depth information of the facial area, depth data corresponding to the facial area and the number of depth data are obtained; based on the depth data corresponding to the facial area and the number of depth data, the distance between the facial part of the object to be identified and the electronic device is determined.

[0094] The processor may calculate the distance between the face of the object to be identified and the electronic device according to the following formula:

[0095]

[0096] Among them, sum represents the sum of each depth data, and size represents the number of depth data.

[0097] After obtaining the initial depth information of the face area, the processor can extract the effective depth information therein, count the number of depth data contained in the effective depth information, and calculate the sum of each depth data, and then calculate the distance between the face part of the object to be identified and the electronic device based on the above formula.

[0098] S402: Determine a first target reference image according to the distance between the face part of the object to be identified and the electronic device and the distance parameters corresponding to each reference image.

[0099] Among them, the distance between each reference image and the electronic device is pre-stored in the electronic device. After obtaining the distance between the facial part of the object to be identified and the electronic device, it is possible to determine which reference image is closest to the facial part of the object to be identified based on the distance and the distance between each reference image and the electronic device (distance parameter), and use the reference image closest to the facial part of the object to be identified as the first target reference image.

[0100] Figure 5 A flowchart of another image processing method provided in an embodiment of the present application is shown below. Figure 5 As shown, the method may also include:

[0101] S501: If the image to be processed does not contain the face region, initial depth information of the image to be processed is obtained according to the region on the image to be processed and the initial reference image, and a two-dimensional image of the object to be identified is acquired.

[0102] Among them, after the processor obtains the image to be processed, it can extract the feature data in the image to be processed and determine whether the feature data contains facial features (such as facial features). If the image to be processed does not have features that match the facial features, it means that the image to be processed does not contain the facial area. Then the processor can obtain the initial depth information of the image to be processed based on the pixel information of each pixel in the image to be processed collected by the infrared camera, the pixel information of the initial reference image, and the camera parameters. At the same time, the processor can also extract a two-dimensional image of the object to be processed collected by the RGB camera from the image to be processed.

[0103] S502: Determine the distance between the object to be identified and the electronic device according to the initial depth information of the image to be processed.

[0104] The processor can obtain the depth data corresponding to the image to be processed and the number of the depth data according to the initial depth information of the image to be processed, and then calculate the distance between the object to be identified and the electronic device according to the depth data corresponding to the image to be processed and the number of the depth data in combination with the above formula (1).

[0105] S503: Obtain a second target reference image according to the distance between the object to be identified and the electronic device and the distance parameters corresponding to each reference image.

[0106] Among them, the distance between each reference image and the electronic device is pre-stored in the electronic device. After the distance between the object to be identified and the electronic device is obtained, it is possible to determine which reference image is closest to the object to be identified based on the distance and the distance between each reference image and the electronic device (distance parameter), and use the reference image closest to the object to be identified as the second target reference image.

[0107] S504: Calculate target depth information of the image to be processed according to the second target reference image and the image to be processed to obtain a three-dimensional image of the object to be identified.

[0108] S505: Identify the object to be identified based on the three-dimensional image of the object to be identified and the two-dimensional image of the object to be identified.

[0109] Taking the second target reference image as a reference, the pixel information on the second target reference image, the pixel information of the image to be processed and the camera parameters are obtained, and the target depth information of the image to be processed can be calculated. According to the target depth information, the target depth image of the image to be processed can be constructed. Finally, the target depth image is transformed between the image coordinate system, the camera coordinate system and the world coordinate system to construct a three-dimensional image of the object to be identified, and then the object to be identified is modeled according to the three-dimensional image of the object to be identified and the two-dimensional image of the object to be identified, and the model corresponding to the object to be identified is obtained.

[0110] Optionally, the above-mentioned determining the distance between the object to be identified and the electronic device based on the initial depth information of the image to be processed includes: obtaining initial depth information corresponding to the central area of ​​the image to be processed based on the initial depth information of the image to be processed; and determining the distance between the object to be identified and the device based on the initial depth information corresponding to the central area of ​​the image to be processed.

[0111] Among them, the initial depth information of the image to be processed includes the initial depth information of the central area and the initial depth information of the non-central area. The central area can be set according to actual needs and is not limited in this application. After determining the central area of ​​the image to be processed, the initial depth information corresponding to the central area of ​​the image to be processed can be extracted. Based on the initial depth information corresponding to the central area of ​​the image to be processed, the depth data corresponding to the central area and the number of the depth data can be obtained. According to the depth data corresponding to the face area and the number of the depth data and the above formula (1), the distance between the object to be identified and the electronic device is calculated.

[0112] Figure 6 Schematic diagram of the structure of an image processing device provided in an embodiment of the present application. Figure 6 As shown, the device may include:

[0113] A determination module 601 is used to determine whether the image to be processed contains a face region, where the image to be processed is an image containing an object to be identified;

[0114] A first determination module 602 is configured to obtain initial depth information of the face region according to the face region and the initial reference image, and acquire a two-dimensional image of the face region if the image to be processed includes a face region;

[0115] A second determination module 603 is used to determine a first target reference image according to the initial depth information of the face region;

[0116] A calculation module 604 is used to calculate target depth information of the face region according to the first target reference image and the face region to obtain a three-dimensional image of the face region;

[0117] The recognition module 605 is used to recognize the object to be recognized based on the three-dimensional image of the face area and the two-dimensional image of the face area.

[0118] Optionally, the first determination module 602 is specifically used to determine whether the face area is credible based on the face frame corresponding to the face area and a preset threshold; if the face area is credible, obtain initial depth information of the face area based on the face area and the initial reference image.

[0119] Optionally, the first determination module 602 is further specifically configured to obtain initial depth information of the image to be processed based on an optional area on the image to be processed and an initial reference image if the face area is not credible, the optional area including the face area and an area outside the face area.

[0120] Optionally, the second determination module 603 is specifically used to determine the distance between the facial part of the object to be identified and the electronic device based on the initial depth information of the facial area; determine the first target reference image based on the distance between the facial part of the object to be identified and the electronic device and the distance parameters corresponding to each reference image.

[0121] Optionally, the second determination module 603 is further specifically used to obtain depth data and the number of depth data corresponding to the face area based on the initial depth information of the face area; and determine the distance between the face part of the object to be identified and the electronic device based on the depth data and the number of depth data corresponding to the face area.

[0122] Optionally, the first determination module 602 is further configured to obtain initial depth information of the image to be processed according to the area on the image to be processed and the initial reference image if the image to be processed does not contain a face area, and obtain a two-dimensional image of the object to be identified;

[0123] The second determination module 603 is further used to determine the distance between the object to be identified and the electronic device according to the initial depth information of the image to be processed; and obtain a second target reference image according to the distance between the object to be identified and the electronic device and the distance parameters corresponding to each reference image;

[0124] The calculation module 604 is further used to calculate the target depth information of the image to be processed according to the second target reference image and the image to be processed, so as to obtain a three-dimensional image of the object to be identified;

[0125] The recognition module 605 is further configured to recognize the object to be recognized based on the three-dimensional image of the object to be recognized and the two-dimensional image of the object to be recognized.

[0126] Optionally, the second determination module 603 is further specifically used to obtain initial depth information corresponding to the central area of ​​the image to be processed based on the initial depth information of the image to be processed; and determine the distance between the object to be identified and the device based on the initial depth information corresponding to the central area of ​​the image to be processed.

[0127] The above-mentioned device is used to execute the method provided by the aforementioned embodiment, and its implementation principle and technical effect are similar, which will not be repeated here.

[0128] The above modules may be one or more integrated circuits configured to implement the above methods, such as one or more application specific integrated circuits (ASIC), or one or more microprocessors (DSP), or one or more field programmable gate arrays (FPGA). For another example, when a module is implemented in the form of a processing element scheduling program code, the processing element may be a general-purpose processor, such as a central processing unit (CPU) or other processor that can call program code. For another example, these modules may be integrated together and implemented in the form of a system-on-a-chip (SOC).

[0129] Figure 7 Another structural diagram of an electronic device provided in an embodiment of the present application is shown in FIG. Figure 7As shown, the specific form of the electronic device can be a mobile phone, a tablet computer, a wearable device, etc. The electronic device may include: a processor 701, a storage medium 702 and a bus 703, the storage medium 702 stores machine-readable instructions executable by the processor 701, when the electronic device is running, the processor 701 and the storage medium 702 communicate through the bus 703, and the processor 701 executes the machine-readable instructions to perform the steps of the above-mentioned image processing method. The specific implementation method and technical effect are similar, and will not be repeated here.

[0130] Optionally, the present application also provides a storage medium, on which a computer program is stored, and when the computer program is executed by a processor, the steps of the above-mentioned image processing method are executed.

[0131] In the several embodiments provided in this application, it should be understood that the disclosed devices and methods can be implemented in other ways. For example, the device embodiments described above are only schematic. For example, the division of the units is only a logical function division. There may be other division methods in actual implementation, such as multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the indirect coupling or communication connection of the device or unit can be electrical, mechanical or other forms.

[0132] The units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed on multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0133] In addition, each functional unit in each embodiment of the present application may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit. The above-mentioned integrated unit may be implemented in the form of hardware or in the form of hardware plus software functional units.

[0134] The above-mentioned integrated unit implemented in the form of a software functional unit can be stored in a computer-readable storage medium. The above-mentioned software functional unit is stored in a storage medium, including a number of instructions for a computer device (which can be a personal computer, a server, or a network device, etc.) or a processor (English: processor) to perform some steps of the method described in each embodiment of the present application. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (English: Read-Only Memory, abbreviated: ROM), random access memory (English: Random Access Memory, abbreviated: RAM), disk or optical disk and other media that can store program codes.

[0135] It should be noted that, in this article, relational terms such as "first" and "second" are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Moreover, the terms "include", "comprise" or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or device. In the absence of further restrictions, the elements defined by the sentence "comprise a ..." do not exclude the existence of other identical elements in the process, method, article or device including the elements.

Claims

1. An image processing method, It is characterized in that The method is applied to an electronic device, and the method comprises: Determining whether the image to be processed contains a face area, wherein the image to be processed is an image containing an object to be identified; If the image to be processed includes a face region, initial depth information of the face region is obtained according to the face region and an initial reference image, and a two-dimensional image of the face region is acquired, wherein the initial reference image is one of the multiple reference images, the multiple reference images are pre-stored in the electronic device, each reference image corresponds to a different distance parameter, and the distance parameter is used to characterize the distance between the object included in the reference image and the electronic device; Determine a first target reference image according to the initial depth information of the face area; Calculating target depth information of the face region according to the first target reference image and the face region to obtain a three-dimensional image of the face region; Based on the three-dimensional image of the face area and the two-dimensional image of the face area, identifying the object to be identified; The obtaining the initial depth information of the face region according to the face region and the initial reference image includes: Determining whether the face region is credible according to a face frame corresponding to the face region and a preset threshold; If the face region is credible, obtaining initial depth information of the face region according to the face region and the initial reference image; If the face area is not credible, initial depth information of the image to be processed is obtained according to an optional area on the image to be processed and the initial reference image, wherein the optional area includes the face area and an area outside the face area.

2. The method according to claim 1, It is characterized in that The determining of the first target reference image according to the initial depth information of the face region includes: Determine the distance between the face part of the object to be identified and the electronic device according to the initial depth information of the face area; A first target reference image is determined according to the distance between the face part of the object to be identified and the electronic device and the distance parameters corresponding to the reference images.

3. The method according to claim 2, It is characterized in that The determining the distance between the face part of the object to be identified and the electronic device according to the initial depth information of the face area includes: According to the initial depth information of the face area, acquiring the depth data corresponding to the face area and the number of the depth data; The distance between the face part of the object to be identified and the electronic device is determined according to the depth data corresponding to the face area and the number of the depth data.

4. The method according to claim 1, It is characterized in that The method further comprises: If the image to be processed does not contain the face area, initial depth information of the image to be processed is obtained according to the area on the image to be processed and the initial reference image, and a two-dimensional image of the object to be identified is obtained; Determining the distance between the object to be identified and the electronic device according to the initial depth information of the image to be processed; Obtaining a second target reference image according to the distance between the object to be identified and the electronic device and the distance parameters corresponding to each reference image; Calculating target depth information of the image to be processed according to the second target reference image and the image to be processed to obtain a three-dimensional image of the object to be identified; The object to be identified is identified based on the three-dimensional image of the object to be identified and the two-dimensional image of the object to be identified.

5. The method according to claim 4, It is characterized in that The determining the distance between the object to be identified and the electronic device according to the initial depth information of the image to be processed includes: Acquire initial depth information corresponding to a central area of ​​the image to be processed according to the initial depth information of the image to be processed; The distance between the object to be identified and the device is determined according to initial depth information corresponding to the central area of ​​the image to be processed.

6. An image processing device, It is characterized in that The device is applied to electronic equipment, and the device comprises: A judgment module, used to judge whether the image to be processed contains a face area, wherein the image to be processed is an image containing an object to be identified; A first determination module, configured to obtain initial depth information of the face region according to the face region and an initial reference image, and acquire a two-dimensional image of the face region if the image to be processed includes a face region, wherein the initial reference image is one of the multiple reference images, the multiple reference images are pre-stored in the electronic device, each reference image corresponds to a different distance parameter, and the distance parameter is used to characterize the distance between the object included in the reference image and the electronic device; A second determination module, configured to determine a first target reference image according to the initial depth information of the face region; A calculation module, configured to calculate target depth information of the face region according to the first target reference image and the face region to obtain a three-dimensional image of the face region; A recognition module, used for recognizing the object to be recognized based on the three-dimensional image of the face area and the two-dimensional image of the face area; The first determining module is specifically used for: Determining whether the face region is credible according to a face frame corresponding to the face region and a preset threshold; If the face region is credible, obtaining initial depth information of the face region according to the face region and the initial reference image; If the face area is not credible, initial depth information of the image to be processed is obtained according to an optional area on the image to be processed and the initial reference image, wherein the optional area includes the face area and an area outside the face area.

7. An electronic device, It is characterized in that include: A processor, a storage medium and a bus, wherein the storage medium stores machine-readable instructions executable by the processor, and when the electronic device is running, the processor communicates with the storage medium via the bus, and the processor executes the machine-readable instructions to perform the steps of the image processing method as described in any one of claims 1 to 5.

8. A storage medium, It is characterized in that The storage medium stores a computer program, and when the computer program is executed by a processor, the steps of the image processing method according to any one of claims 1 to 5 are executed.

Images