Image processing method and device

Abstract

The present invention discloses an image processing method and device. The method comprises: obtaining a first image and a second image, wherein the first image is a non-living image of a target object, and the second image is a living image of a current object acquired in real time; segmenting the first image to obtain a first region, wherein the first region is the region where the target object is located; rotating the first region based on a target circumscribed rectangle corresponding to the first region to obtain a target rotation angle; verifying the second image based on the first region, the first image, and the target rotation angle to obtain a verification result, wherein the verification result is used to determine whether the current object in the second image is the target object in the first image. The present invention solves the technical problem of low face recognition accuracy due to a large non-portrait image region in the related art.

Description

Technical Field

[0001] The present invention relates to the field of face recognition applications, and in particular to an image processing method and device. Background Art

[0002] In the field of face recognition applications, faces are mainly recognized based on ID photos, such as client login. However, there may be situations where users fail to upload ID photos as required, the ID photos are rotated, or the portrait area is small (that is, the unnecessary background area is large), which leads to low accuracy in face recognition.

[0003] To address the above-mentioned problems, no effective solution has been proposed yet. Summary of the invention

[0004] The embodiments of the present invention provide an image processing method and device to at least solve the technical problem in the related art that the accuracy of face recognition is low due to the large non-portrait image area.

[0005] According to one aspect of an embodiment of the present invention, there is provided an image processing method, characterized in that it includes: acquiring a first image and a second image, wherein the first image is a non-living image of a target object, and the second image is a living image of a current object acquired in real time; segmenting the first image to obtain a first area, wherein the first area is an area where the target object is located; rotating the first area based on a target circumscribed rectangle corresponding to the first area to obtain a target rotation angle; verifying the second image based on the first area, the first image, and the target rotation angle to obtain a verification result, wherein the verification result is used to determine whether the current object in the second image is the target object in the first image.

[0006] Optionally, the first area is rotated based on a target circumscribed rectangle corresponding to the first area to obtain a target rotation angle, including: determining a first side of the target circumscribed rectangle, wherein a length of the first side is greater than a length of a second side of the target circumscribed rectangle; dividing the target circumscribed rectangle based on the first side to obtain a first rectangle and a second rectangle; determining a target central axis segment of the target circumscribed rectangle based on the first rectangle and the second rectangle, wherein the target central axis segment perpendicularly bisects the second side and is parallel to the first side of the target circumscribed rectangle; rotating the first area based on the target central axis segment to obtain a target rotation angle.

[0007] Optionally, determining a target central axis segment of a target circumscribed rectangle based on the first rectangle and the second rectangle includes: determining a first circle fit corresponding to the first rectangle, and determining a second circle fit corresponding to the second rectangle; comparing the first circle fit and the second circle fit to obtain a comparison result, wherein the comparison result is used to determine whether the first circle fit is greater than the second circle fit; determining a target rectangle based on the comparison result, wherein the target rectangle is a rectangle with the largest circle fit between the first rectangle and the second rectangle; and determining a target central axis segment of the target circumscribed rectangle based on the target rectangle.

[0008] Optionally, the first area is rotated based on the target central axis segment to obtain a target rotation angle, including: determining a target coordinate axis corresponding to the first image; determining a first coordinate and a second coordinate corresponding to the target central axis segment based on the target coordinate axis, wherein the first coordinate and the second coordinate are coordinates corresponding to the starting point and the end point of the target central axis segment, respectively; determining a central axis angle of the target central axis based on the first coordinate, the second coordinate and the target coordinate axis, wherein the central axis angle is the angle between the target central axis and the axis of the target coordinate axis; determining the target rotation angle of the first area based on the central axis angle.

[0009] Optionally, the second image is verified based on the first area and the target rotation angle to obtain a verification result of the target object, including: cropping the first image based on the first area to obtain a third image; rotating the third image based on the target rotation angle to obtain a fourth image; determining the target similarity between the fourth image and the second image, and determining a target score corresponding to the target similarity; when the target score is greater than a preset score, determining that the verification result is that the current object is the target object in the first image; when the target score is less than or equal to the preset score, determining that the verification result is that the current object is not the target object in the first image.

[0010] Optionally, the first image is segmented to obtain the first region, including: using a target segmentation network to segment the first image to obtain the first region; wherein the target segmentation network includes: a convolution layer, an attention layer, and an upsampling layer; using the target segmentation network to segment the first image to obtain the first region, including: using the convolution layer to convolve the first image to obtain an initial feature map; using the attention layer to process the initial feature map to obtain a target feature map; using the upsampling layer to upsample the target feature map to obtain the first region.

[0011] Optionally, upsampling the target feature image using an upsampling layer to obtain the first region includes: upsampling the target feature image using an upsampling layer to obtain multiple sampling images; adding features in the multiple sampling images to obtain the first region.

[0012] Optionally, the method further includes: the target bounding rectangle is a minimum bounding rectangle.

[0013] According to another aspect of an embodiment of the present invention, there is also provided an image processing device, including: an acquisition module, used to acquire a first image and a second image, wherein the first image is a non-living image of a target object, and the second image is a living image of a current object acquired in real time; a segmentation processing module, used to perform segmentation processing on the first image to obtain a first area, wherein the first area is an area where the target object is located; a rotation processing module, used to perform rotation processing on the first area based on a target circumscribed rectangle corresponding to the first area to obtain a target rotation angle; a verification module, used to verify the second image based on the first area, the first image, and the target rotation angle to obtain a verification result, wherein the verification result is used to determine whether the current object in the second image is the target object in the first image.

[0014] According to another aspect of an embodiment of the present invention, a storage medium is further provided. The storage medium includes a stored program, wherein when the program is running, the device where the storage medium is located is controlled to execute any one of the image processing methods provided in the above embodiments.

[0015] According to another aspect of an embodiment of the present invention, a computer terminal is provided, including: a processor and a memory, wherein the processor is used to run a program stored in the memory, wherein when the program is run, any one of the image processing methods provided in the above embodiments is executed.

[0016] In the embodiment of the present invention, a first image and a second image are acquired, wherein the first image is a non-living image of a target object, and the second image is a living image of a current object collected in real time; the first image is segmented to obtain a first region, wherein the first region is the region where the target object is located; the first region is rotated based on a target circumscribed rectangle corresponding to the first region to obtain a target rotation angle; the second image is verified based on the first region, the first image, and the target rotation angle to obtain a verification result, wherein the verification result is used to determine whether the current object in the second image is the target object in the first image, thereby achieving the purpose of improving the accuracy of face recognition. It is easy to notice that the target region is obtained by segmenting the first image and rotating the first region based on the circumscribed rectangle, that is, the target region contains the image required for face recognition, thereby achieving the purpose of accurately recognizing a face, thereby achieving the technical effect of improving the accuracy of face recognition, and thus solving the technical problem of low face recognition accuracy caused by a large non-portrait image region in the related art. BRIEF DESCRIPTION OF THE DRAWINGS

[0017] The drawings described herein are used to provide a further understanding of the present invention and constitute a part of this application. The exemplary embodiments of the present invention and their descriptions are used to explain the present invention and do not constitute an improper limitation of the present invention. In the drawings:

[0018] Figure 1 is a flowchart of an image processing method according to an embodiment of the present invention;

[0019] Figure 2 is an optional main technical flow chart according to an embodiment of the present invention;

[0020] Figure 3 is a schematic diagram of an optional ID card portrait segmentation network according to this embodiment;

[0021] Figure 4 is an optional flow chart for calculating a rotation angle according to an embodiment of the present invention;

[0022] Figure 5 is a schematic diagram of an optional situation when start_point_x is equal to end_point_x according to an embodiment of the present invention;

[0023] Figure 6 is a schematic diagram of an optional situation when alpha is greater than 0 according to an embodiment of the present invention;

[0024] Figure 7 is a schematic diagram of an optional face recognition step processing according to an embodiment of the present invention;

[0025] Figure 8 is a structural schematic diagram of an image processing device according to an embodiment of the present invention. DETAILED DESCRIPTION

[0026] In order to enable those skilled in the art to better understand the scheme of the present invention, the technical scheme in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work should fall within the scope of protection of the present invention.

[0027] It should be noted that the terms "first", "second", etc. in the specification and claims of the present invention and the above-mentioned drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence. It should be understood that the data used in this way can be interchanged where appropriate, so that the embodiments of the present invention described herein can be implemented in an order other than those illustrated or described herein. In addition, the terms "including" and "having" and any variations thereof are intended to cover non-exclusive inclusions, for example, a process, method, system, product or device that includes a series of steps or units is not necessarily limited to those steps or units that are clearly listed, but may include other steps or units that are not clearly listed or inherent to these processes, methods, products or devices.

[0028] In the field of face recognition applications, ID photos are involved in many scenarios, such as financial credit identity recognition, bank client login, etc. ID photos here include ID cards, driver's licenses, graduation certificates, etc. Users sometimes upload ID photos not in accordance with the requirements, and there will be rotation or small target area (i.e. large unnecessary background area), which will cause two problems when performing face recognition: 1. The face cannot be detected and the application directly returns an error; 2. The face detection is wrong and the recognition pass rate is low. Therefore, it is of practical significance to reasonably and effectively correct the ID photos so that the corrected image is the portrait area of ​​interest.

[0029] Most of the existing methods for processing ID photos uploaded by users are to find the four corner points of the ID photo on the algorithm side, and then obtain the required image through perspective transformation. There are four corner point detection methods, such as corner point detection based on contour curves, corner point detection based on binary images or grayscale images, etc. These methods can produce the expected results under ideal conditions, but they will be difficult to work when the ID photo is incomplete or the boundaries are inseparable. In the field of face recognition, even if the image is transformed after perspective transformation, the problem of large background area and portrait rotation will still bring huge challenges to subsequent recognition.

[0030] Example 1

[0031] According to an embodiment of the present invention, an embodiment of an image processing method is provided. It should be noted that the steps shown in the flowchart of the accompanying drawings can be executed in a computer system such as a set of computer executable instructions, and although a logical order is shown in the flowchart, in some cases, the steps shown or described can be executed in an order different from that shown here.

[0032] Figure 1 is a flowchart of an image processing method according to an embodiment of the present invention. Figure 1 As shown, the method comprises the following steps:

[0033] Step S102 , acquiring a first image and a second image, wherein the first image is a non-living image of the target object, and the second image is a living image of the current object acquired in real time.

[0034] The target object is a user who needs to perform face recognition; the non-living image can be an ID photo of the user who needs face recognition, or a life photo, a selfie, etc., but is not limited to this. In this embodiment, the ID photo is used as an example for explanation; the first image can be an ID photo of the user who needs face recognition.

[0035] The above-mentioned current object is a user who is undergoing face recognition, and the living image is a portrait image of the user who is undergoing face recognition collected in real time; the second image can be a portrait image of the user who is undergoing face recognition collected in real time.

[0036] In an optional embodiment, when performing face recognition, firstly, an ID photo of the user who needs to perform face recognition is obtained, and secondly, a portrait image of the user who is performing face recognition is collected in real time.

[0037] In another optional embodiment, when performing face recognition, while obtaining the ID photo of the user who needs to perform face recognition, a portrait image of the user who is performing face recognition is collected in real time.

[0038] Step S104: segment the first image to obtain a first region, wherein the first region is a region where the target object is located.

[0039] The above-mentioned segmentation processing of the first image can be based on the deep residual network (RestNet) 34 as the backbone network to build a segmentation network and perform portrait segmentation processing on the user's ID photo. Among them, RestNet34 is used in fields such as target classification, and as a part of the backbone classical neural network of computer vision tasks, the RestNet34 network proves that the network can develop in a deeper direction (including more hidden layers), mainly to solve the problem of network degradation.

[0040] The above-mentioned first area can be the portrait contour area in the user's ID photo obtained after segmentation processing, wherein the segmentation processing refers to dividing the image into several non-overlapping sub-areas, identifying the image, so that the features in the same sub-area have a certain similarity, and the features of different sub-areas show more obvious differences.

[0041] In an optional embodiment, after obtaining the user's ID photo (ie, the first image), the ID photo is segmented based on RestNet34 as the backbone to obtain the portrait contour area (ie, the first area) in the ID photo.

[0042] Step S106: rotating the first area based on the target circumscribed rectangle corresponding to the first area to obtain a target rotation angle.

[0043] The target bounding rectangle mentioned above may be a minimum bounding rectangle that can contain the portrait contour area (ie, the first area).

[0044] In an optional embodiment, the first area is rotated by establishing a coordinate system based on a minimum circumscribed rectangle, dividing the minimum circumscribed rectangle in the direction of the long side, and performing circle fitting on the two small rectangles obtained by cutting, determining that the end with a larger circle fitting degree is the end where the head of the portrait is located, and obtaining a central axis pointing line segment, and rotating the first area based on the center point of the target circumscribed rectangle through the angle between the central axis pointing line segment and the horizontal direction, wherein the circle fitting is to obtain a curve of a circle through a function formula while ensuring a small error, the central axis segment is a line segment that bisects the short side of the target circumscribed rectangle and is perpendicular to the short side, and the central axis pointing line segment is a central axis segment pointing in the direction of the head of the portrait.

[0045] The target rotation angle may be an angle that enables the minimum circumscribed rectangle of the portrait contour area to rotate from the current position to a position where the central axis of the minimum circumscribed rectangle points to a line segment perpendicular to the X-axis and the Y-axis coordinate of the end point of the central axis pointing to the line segment is greater than the Y-axis coordinate of the starting point.

[0046] In an optional embodiment, a coordinate system can be established based on the target circumscribed rectangle of the first area, the target circumscribed rectangle is divided in the long side direction and circle fitting is performed on the two small rectangles respectively, and the end with a larger circle fitting degree is determined as the end where the portrait head is located, that is, a central axis pointing line segment is obtained, and by judging the angle between the central axis pointing line segment and the horizontal direction, the first area is rotated based on the center point of the target circumscribed rectangle, and the target angle is obtained by calculating the angle of the minimum circumscribed rectangle rotated from the current position to the position where the central axis pointing line segment is perpendicular to the X-axis and the Y-axis coordinate of the end point of the central axis pointing line segment is greater than the Y-axis coordinate of the starting point. In this embodiment, the target angle is illustrated using theta as an example.

[0047] Step S108, verifying the second image based on the first area, the first image, and the target rotation angle to obtain a verification result, wherein the verification result is used to determine whether the current object in the second image is the target object in the first image.

[0048] The above-mentioned verification of the second image based on the first area, the first image, and the target rotation angle can be to crop the first image based on the first area to obtain a portrait image with a portrait outline, and then rotate the portrait image with a portrait outline based on the target rotation angle to obtain the target image, and finally compare the target image with the portrait image (i.e., the second image) collected in real time. The final verification can be to set a preset distribution relationship between the similarity and the target score and the preset target score in advance. For example, the target score corresponding to the similarity of 10% is 1 point, and the preset target score can be set to 7 points. The target image and the second image are verified for similarity, and the corresponding target score is obtained based on the similarity. Then, the corresponding target score is compared with the preset target score. If the corresponding target score is greater than the preset target score, it can be determined whether the current object in the second image is the target object in the first image.

[0049] In an optional embodiment, the first image is first cropped based on the first area to obtain a portrait image with a portrait outline, and then the portrait image with the portrait outline is rotated based on the target rotation angle to obtain a target image. Finally, the target image and the second image are verified for similarity to obtain the similarity, and a corresponding target score is obtained based on the similarity. The corresponding target score is compared with a preset target score. If the corresponding target score is greater than the preset target score, it is confirmed whether the current object in the second image is the target object in the first image.

[0050] Optionally, the first area is rotated based on a target circumscribed rectangle corresponding to the first area to obtain a target rotation angle, including: determining a first side of the target circumscribed rectangle, wherein a length of the first side is greater than a length of a second side of the target circumscribed rectangle; dividing the target circumscribed rectangle based on the first side to obtain a first rectangle and a second rectangle; determining a target central axis segment of the target circumscribed rectangle based on the first rectangle and the second rectangle, wherein the target central axis segment perpendicularly bisects the second side and is parallel to the first side of the target circumscribed rectangle; rotating the first area based on the target central axis segment to obtain a target rotation angle.

[0051] The above-mentioned target central axis segment can be a directed line segment that is perpendicular to and bisects the second side of the target circumscribed rectangle, and the line segment is parallel to the first side of the target circumscribed rectangle, and the direction of the line segment is the direction of the head of the portrait in the first area; the first side can be the length of the target circumscribed rectangle, and the second side can be the width of the target circumscribed rectangle.

[0052] The first rectangle mentioned above may be a rectangle where the portrait head is located in the first area obtained by bisecting the long side of the target circumscribed rectangle; the second rectangle mentioned above may be a rectangle where the non-portrait head is located in the first area obtained by bisecting the long side of the target circumscribed rectangle.

[0053] In an optional embodiment, performing segmentation processing on the target circumscribed rectangle based on the first side to obtain the first rectangle and the second rectangle may be a segmentation processing in which the length of the target circumscribed rectangle is equally divided.

[0054] In another optional embodiment, first, the length of the target circumscribed rectangle (i.e., the first side) is determined, and secondly, based on a segmentation process of dividing the length of the target circumscribed rectangle in half, a first rectangle, a second rectangle, and a target central axis segment are obtained, and the target angle is obtained by calculating the angle of the minimum circumscribed rectangle rotated from the current position to the point where the central axis pointing line segment is perpendicular to the horizontal direction and points in the direction of the head of the portrait in the first area.

[0055] Optionally, determining a target central axis segment of a target circumscribed rectangle based on the first rectangle and the second rectangle includes: determining a first circle fit corresponding to the first rectangle, and determining a second circle fit corresponding to the second rectangle; comparing the first circle fit and the second circle fit to obtain a comparison result, wherein the comparison result is used to determine whether the first circle fit is greater than the second circle fit; determining a target rectangle based on the comparison result, wherein the target rectangle is a rectangle with the largest circle fit between the first rectangle and the second rectangle; and determining a target central axis segment of the target circumscribed rectangle based on the target rectangle.

[0056] The above-mentioned determination of the first circle fitting degree corresponding to the first rectangle and the determination of the second circle fitting degree corresponding to the second rectangle can be implemented according to the following formula:

[0057]

[0058] Among them, pixel val Represents the pixel value, count represents the number of pixels with a pixel value of 255, and area_circle represents the area of ​​the fitted circle.

[0059] The target rectangle mentioned above may be a rectangle where the head of the portrait is located in the first area.

[0060] In an optional embodiment, firstly, based on the circle fit formula, the first circle fit and the second circle fit of the first rectangle and the second rectangle are calculated respectively. Secondly, the first circle fit and the second circle fit are compared to determine that the rectangle with a larger circle fit is the rectangle where the head of the portrait is located in the first area. Therefore, the rectangle with a larger circle fit is used as the target rectangle. The direction of the target central axis segment can be determined according to the position of the target rectangle in the target circumscribed rectangle. Finally, based on the second side of the target circumscribed rectangle, a line segment that is perpendicular to and bisects the second side of the target circumscribed rectangle is made to obtain the target central axis segment.

[0061] Optionally, the first area is rotated based on the target central axis segment to obtain a target rotation angle, including: determining a target coordinate axis corresponding to the first image; determining a first coordinate and a second coordinate corresponding to the target central axis segment based on the target coordinate axis, wherein the first coordinate and the second coordinate are coordinates corresponding to the starting point and the end point of the target central axis segment, respectively; determining a central axis angle of the target central axis based on the first coordinate, the second coordinate and the target coordinate axis, wherein the central axis angle is the angle between the target central axis and the axis of the target coordinate axis; determining the target rotation angle of the first area based on the central axis angle.

[0062] In an optional embodiment, the target coordinate axis corresponding to the first image may be determined by taking a right angle of the first image as the origin, a side parallel to the horizontal direction as the X-axis, and a side perpendicular to the horizontal direction as the Y-axis.

[0063] The first coordinate mentioned above is the starting point coordinate of the target central axis segment, which can be expressed as (start_point_x, start_point_y), and the second coordinate is the end point coordinate of the target central axis segment, which can be expressed as (end_point_x, end_point_y), where start_point_x represents the X-axis coordinate of the starting point, start_point_y represents the Y-axis coordinate of the starting point, end_point_x represents the X-axis coordinate of the end point, and end_point_y represents the Y-axis coordinate of the end point.

[0064] The center axis angle of the target center axis mentioned above may be the angle between the extension line of the target center axis segment and the X-axis. In this embodiment, the center axis angle is described by taking alpha as an example.

[0065] In an optional embodiment, the target rotation angle of the first area is determined based on the central axis angle, and the value of theta can be obtained through the value of alpha.

[0066] In an optional embodiment, the target coordinate axis corresponding to the first image is first determined, and then the first coordinate and the second coordinate of the target central axis segment are determined. Then, the central axis angle alpha is determined based on the first coordinate, the second coordinate and the target coordinate axis. Finally, the target rotation angle theta of the first area is determined based on alpha. For example, when start_point_x is equal to end_point_x, when start_point_y is greater than end_point_y, alpha=90°, theta=0, and when start_point_y is less than end_point_y, theta=180°.

[0067] In another optional embodiment, the target coordinate axis corresponding to the first image is first determined, and then the first coordinate and the second coordinate of the target central axis segment are determined. Then, the central axis angle alpha is determined based on the first coordinate, the second coordinate and the target coordinate axis. Finally, the target rotation angle theta of the first area is determined based on alpha. For example, when start_point_y is equal to end_point_y, when start_point_x is greater than end_point_x, theta=-90°, and when start_point_x is less than end_point_x, theta=90°.

[0068] In another optional embodiment, the target coordinate axis corresponding to the first image is first determined, and then the first coordinate and the second coordinate of the target central axis segment are determined. Then, the central axis angle alpha is determined based on the first coordinate, the second coordinate and the target coordinate axis. Finally, the target rotation angle theta of the first area is determined based on alpha. For example, when start_point_x is not equal to end_point_x and start_point_y is not equal to end_point_y, when alpha is greater than 0 and end_point_y is less than start_point_y, theta=-(alpha+90) and when end_point_y is greater than start_point_y, theta=90-alpha.

[0069] In another optional embodiment, the target coordinate axis corresponding to the first image is first determined, and then the first coordinate and the second coordinate of the target central axis segment are determined. Then, the central axis angle alpha is determined based on the first coordinate, the second coordinate and the target coordinate axis. Finally, the target rotation angle theta of the first area is determined based on alpha. For example, when start_point_x is not equal to end_point_x and start_point_y is not equal to end_point_y, when alpha is less than 0, when end_point_y is less than start_point_y, theta=90-alpha, and when end_point_y is greater than start_point_y, theta=-90-alpha.

[0070] Optionally, the second image is verified based on the first area and the target rotation angle to obtain a verification result of the target object, including: cropping the first image based on the first area to obtain a third image; rotating the third image based on the target rotation angle to obtain a fourth image; determining the target similarity between the fourth image and the second image, and determining a target score corresponding to the target similarity; when the target score is greater than a preset score, determining that the verification result is that the current object is the target object in the first image; when the target score is less than or equal to the preset score, determining that the verification result is that the current object is not the target object in the first image.

[0071] The above-mentioned third image can be a portrait image obtained by cropping the user's ID photo based on the portrait contour area; the fourth image can be an image obtained by rotating the third image based on the target rotation angle, in which the central axis pointing line segment of the minimum circumscribed rectangle is perpendicular to the X-axis, and the Y-axis coordinate of the end point of the central axis pointing line segment is greater than the Y-axis coordinate of the starting point.

[0072] In an optional embodiment, the user may set a preset distribution relationship between similarity and target score and a preset target score in advance. For example, a target score corresponding to 10% similarity is 1 point, and the preset target score may be set to 7 points.

[0073] In another optional embodiment, the user's ID photo is first cropped based on the portrait contour area to obtain a portrait image (i.e., the third image); the portrait image is then rotated based on the target rotation angle to obtain a fourth image; the fourth image is then compared with a portrait image of the user that is undergoing face recognition and collected in real time (i.e., the second image) to obtain a target similarity; and based on a preset distribution relationship between the similarity and the target score that is set in advance, a target score can be obtained; the target score is compared with a preset target score; when the target score is greater than the preset score, it is determined that the current object is the target object in the first image; and when the target score is less than or equal to the preset score, it is determined that the current object is not the target object in the first image.

[0074] Optionally, the first image is segmented to obtain the first region, including: using a target segmentation network to segment the first image to obtain the first region; wherein the target segmentation network includes: a convolution layer, an attention layer, and an upsampling layer; using the target segmentation network to segment the first image to obtain the first region, including: using the convolution layer to convolve the first image to obtain an initial feature map; using the attention layer to process the initial feature map to obtain a target feature map; using the upsampling layer to upsample the target feature map to obtain the first region.

[0075] The above-mentioned convolution layer can have multiple values. In this embodiment, a convolution layer with a convolution kernel of 3*3 and a stride of 2 is used as an example for explanation, but it is not limited to this. Among them, the stride is the sampling interval of the convolution kernel passing through the input feature map; the attention layer contains compression and excitation structures, which can improve the interactive ability of channel information; the main purpose of the upsampling layer is to amplify the original image so that the target feature map can be displayed on a display device with higher resolution.

[0076] The above-mentioned using a convolution layer to perform a convolution operation on the first image to obtain the initial feature map may be performing feature extraction on the first image through convolution to obtain the initial feature map of the first image.

[0077] The above-mentioned use of the attention layer to process the initial feature image to obtain the target feature map can be to extract the portrait contour area in the initial feature image through the attention layer to obtain the portrait contour area feature map (ie, the target feature map).

[0078] The above-mentioned upsampling process of the target feature map by using the upsampling layer to obtain the first region may be performed by upsampling the portrait contour feature region so that the portrait contour feature region is enlarged, that is, the first region is obtained.

[0079] In an optional embodiment, firstly, features of the first image can be extracted through convolution to obtain an initial feature map, and then the portrait contour area in the initial feature map is extracted to obtain a portrait contour area feature map. Finally, based on upsampling processing, a larger resolution portrait contour area feature map, i.e., the first area, can be obtained.

[0080] Optionally, upsampling the target feature image using an upsampling layer to obtain the first region includes: upsampling the target feature image using an upsampling layer to obtain multiple sampling images; adding features in the multiple sampling images to obtain the first region.

[0081] In an optional embodiment, the target feature map is first upsampled using an upsampling layer to obtain multiple portrait contour area feature maps with larger resolutions (i.e., multiple sampling images). Next, each portrait contour area feature map with larger resolution is added to the initial feature map in the convolution layer to perform feature fusion, and finally the first area is obtained. It is easy to notice that when performing feature fusion during upsampling, addition is used instead of multiplication in order to reduce the amount of calculation so that the network can meet embedded requirements.

[0082] Optionally, the method further includes: the target bounding rectangle is a minimum bounding rectangle.

[0083] The aforementioned minimum circumscribed rectangle is a rectangle that is tangent to the outline of the first region.

[0084] In view of the special requirements of face recognition for ID photos and the shortcomings of existing correction technology, the present invention provides a method and system for ID photo correction in face recognition, which combines deep learning with traditional pattern recognition classifier design, performs correction processing on ID photos uploaded by users, and uses the final acquired portrait area of ​​interest as a comparison base map for subsequent recognition, effectively reducing storage and improving the pass rate and accuracy of recognition.

[0085] Figure 2 is an optional main technical flow chart according to an embodiment of the present invention, such as Figure 2 As shown, in order to achieve the above-mentioned invention purpose, the main technical process steps include the following:

[0086] Step S201: Use a self-built lightweight segmentation network based on resNet34 to perform portrait segmentation on the ID photo uploaded by the user to obtain the face contour area;

[0087] Step S202: Calculate the minimum circumscribed rectangle of the portrait contour area to obtain the positions of four vertices;

[0088] Step S203: Calculate the circle fitting degree of the upper and lower ends of the circumscribed rectangle to determine the head and tail;

[0089] Step S204: Calculate the target rotation angle;

[0090] Step S205: cropping the face area and rotating it;

[0091] Step S206: Using the cropped and rotated portrait image to replace the ID photo to participate in the subsequent face recognition process;

[0092] Step S207: collecting the user's current live body photo and uploading the live body photo;

[0093] Step S208: comparing the living body photo with the portrait image in the obtained ID photo to obtain a target similarity, determining a target score according to the target similarity, and comparing the target score with a preset target score;

[0094] Step S209: Determine whether the target score is greater than the preset target score; if so, proceed to step S210; if not, proceed to step S211;

[0095] Step S210: The user and the person in the ID photo are the same person;

[0096] Step S211: The user and the person in the ID photo are not the same person.

[0097] The beneficial effects of the present invention are:

[0098] 1. The present invention combines a lightweight segmentation network with a traditional discriminator design to quickly realize the correction of ID photos with a high accuracy. After the neural network is fully trained, it can effectively cover adverse conditions such as highlights and blurs.

[0099] 2. The final cropped image of the portrait area of ​​interest is more conducive to improving the subsequent face recognition accuracy and pass rate, making the scene involving ID photo face recognition more robust.

[0100] In order to better understand the technical solution of the present invention, the present invention will be further explained in detail below in conjunction with the accompanying drawings and embodiments.

[0101] The present invention provides an effective ID photo correction method in face recognition. By training a segmentation network, the ID photo uploaded by the user is segmented to obtain the portrait area, a pattern recognition angle discriminator is designed to calculate the target rotation angle theta, the face area is cropped and rotated, and finally the corrected portrait image is used in the subsequent face recognition process.

[0102] The present invention provides an effective ID photo correction method for face recognition, which specifically comprises the following steps:

[0103] Step S1: Build a network for ID card image segmentation. Figure 3 is a schematic diagram of an optional ID card portrait segmentation network according to this embodiment, such as Figure 3 As shown, this implementation example uses the improved resNet34 as the backbone, which is reflected in the input image of (289, 289, 3) (odd size is more conducive to position positioning during convolution calculation), and uses a convolution operation with a convolution kernel of 3x3 and a stride of 2 instead of a pooling operation. The attention layer structure is added to the last layer of the backbone to improve the channel information interaction ability. When upsampling for feature fusion, addition is used instead of multiplication to reduce the amount of calculation, so that the network meets the embedded requirements. During training, the training data is enhanced in multiple dimensions, such as brightness adjustment, random noise addition, blur adjustment, and random subtraction of part of the picture. The ID photo is segmented through the trained model to obtain a 0-255 binary grayscale image.

[0104] Step S2: Fit a minimum bounding rectangle to the portrait area so that all pixel points with a value of 255 just fall within the minimum bounding rectangle.

[0105] Step S3: The minimum circumscribed rectangle is equally divided into two small rectangles occupying both ends along the long side direction, and the minimum circumscribed circle is fitted to the portrait position points (pixel points with a pixel value of 255) in the two small rectangles, and then the circle fitting degree of the two rectangles is calculated respectively. The circle fitting degree formula is as follows:

[0106]

[0107] Among them, pixel val Represents the pixel value, count represents the number of pixels with a pixel value of 255, and area_circle represents the area of ​​the fitted circle.

[0108] Step S4: The rectangle with a large circle fitting degree is the target rectangle where the portrait head is located. The width of the minimum circumscribed rectangle is vertically divided into two halves to obtain a central axis pointing line segment. The pointing direction of the line segment is from the bottom of the portrait outline to the head direction of the portrait outline. A coordinate system is established based on the minimum circumscribed rectangle, and the angle alpha between the central axis pointing line segment and the horizontal direction is calculated. The specific method is as follows, wherein the starting point coordinates of the central axis pointing line segment can be expressed as (start_point_x, start_point_y), and the end point coordinates of the central axis pointing line segment can be expressed as (end_point_x, end_point_y), wherein start_point_x represents the X-axis coordinate of the starting point, start_point_y represents the Y-axis coordinate of the starting point, end_point_x represents the X-axis coordinate of the end point, and end_point_y represents the Y-axis coordinate of the end point.

[0109] Figure 4 is an optional flow chart for calculating the rotation angle according to an embodiment of the present invention. Figure 4 As shown, the method comprises the following steps:

[0110] Step S401: Determine whether start_point_x is equal to end_point_x, if so, proceed to step S402, if not, proceed to step S405;

[0111] Step S402, determine whether start_point_y is greater than end_point_y, if so, proceed to step S403, if not, proceed to step S404;

[0112] Step S403, start_point_x is equal to end_point_x, start_point_y is greater than end_point_y, theta is equal to 0, and the process ends;

[0113] Step S404, start_point_x is equal to end_point_x, start_point_y is less than end_point_y, theta is equal to 180 degrees, and the process ends;

[0114] Step S405, determine whether start_point_y is equal to end_point_y, if so, proceed to step S406, if not, proceed to step S409;

[0115] Step S406: Is start_point_x greater than end_point_x? If yes, proceed to step S407; if no, proceed to step S408;

[0116] Step S407, start_point_y is equal to end_point_y, start_point_x is greater than end_point_x, theta is equal to -90 degrees, and the process ends;

[0117] Step S408, start_point_y is equal to end_point_y, start_point_x is less than end_point_x, theta is equal to 90 degrees, and the process ends;

[0118] Step S409, whether the angle alpha between the middle axis pointing line segment and the horizontal direction is greater than 0, if so, proceed to step S410, if not, proceed to step S413;

[0119] Step S410, determine whether start_point_y is greater than end_point_y, if so, proceed to step S411, if not, proceed to step S412;

[0120] Step S411, alpha is greater than 0, start_point_y is greater than end_point_y, theta is equal to -alpha-90 degrees, and the process ends;

[0121] Step S412, alpha is greater than 0, start_point_y is less than end_point_y, theta is equal to 90 degrees - alpha, and the process ends;

[0122] Step S413, determine whether start_point_y is greater than end_point_y, if so, proceed to step S414, if not, proceed to step S415;

[0123] Step S414, alpha is less than 0, start_point_y is greater than end_point_y, theta is equal to 90 degrees - alpha, and the process ends;

[0124] Step S415, alpha is less than 0, start_point_y is less than end_point_y, theta is equal to -alpha-90 degrees, and the process ends.

[0125] Figure 5 is a schematic diagram of an optional situation when start_point_x is equal to end_point_x according to an embodiment of the present invention, such as Figure 5 As shown, the middle axis pointing line segment is perpendicular to the X-axis, but there are two situations for the head position in the portrait contour area.

[0126] Figure 6 is a schematic diagram of an optional situation when alpha is greater than 0 according to an embodiment of the present invention, such as Figure 6 As shown, start_point_x is not equal to end_point_x, start_point_y is not equal to end_point_y, and the target rotation angle theta needs to be obtained based on the angle alpha between the middle axis pointing line segment and the horizontal direction.

[0127] Step S5: cropping the minimum circumscribed rectangular area of ​​the portrait contour area from the original ID photo, and rotating it by a target angle theta as the portrait region of interest image.

[0128] Step S6: The corrected portrait ROI image is used for subsequent face comparison.

[0129] Figure 7 It is a schematic diagram of an optional face recognition step processing according to an embodiment of the present invention, which can reflect the contents of the above steps S1 to S6.

[0130] The present invention provides a method and system for correcting ID photos in face recognition, which can perform rotation correction on ID photos uploaded by users, and use the cropped ID portrait area as a backup image for subsequent face comparison process. Specifically: first, a segmentation network constructed using resNet34 as the backbone is used to perform ID portrait image segmentation on the ID photos uploaded by users to obtain the portrait contour area, and then the angle is calculated using the designed angle detector, and the ID photos are rotated and the portrait area image is cropped, and the database is updated. When performing face comparison, the cropped portrait area image is used to replace the original ID photo image, which greatly reduces storage, and improves the recognition accuracy and robustness of the ID photo face recognition system, and has a wider range of applications.

[0131] Example 2

[0132] According to another aspect of an embodiment of the present invention, an image processing device is also provided, which can execute the image processing method provided in the above embodiment 1. The specific implementation method and preferred application scenario are the same as those of the above embodiment 1 and will not be repeated here.

[0133] Figure 8 is a schematic diagram of the structure of an image processing device according to an embodiment of the present invention. Figure 8As shown, the device includes: an acquisition module 80, used to acquire a first image and a second image, wherein the first image is a non-living image of a target object, and the second image is a living image of a current object acquired in real time; a segmentation processing module 82, used to segment the first image to obtain a first area, wherein the first area is an area where the target object is located; a rotation processing module 84, used to rotate the first area based on a target circumscribed rectangle corresponding to the first area to obtain a target rotation angle; a verification module 86, used to verify the second image based on the first area, the first image, and the target rotation angle to obtain a verification result, wherein the verification result is used to determine whether the current object in the second image is the target object in the first image.

[0134] Optionally, the rotation processing module includes: a first determination unit, used to determine the first side of the target circumscribed rectangle, wherein the length of the first side is greater than the length of the second side of the target circumscribed rectangle; a processing unit, used to segment the target circumscribed rectangle based on the first side to obtain a first rectangle and a second rectangle; a second determination unit, used to determine a target central axis segment of the target circumscribed rectangle based on the first rectangle and the second rectangle, wherein the target central axis segment perpendicularly bisects the second side and is parallel to the first side of the target circumscribed rectangle; a rotation processing unit, which rotates the first area based on the target central axis segment to obtain a target rotation angle.

[0135] Optionally, the second determination unit includes: a first determination subunit, used to determine the first circle fit corresponding to the first rectangle, and determine the second circle fit corresponding to the second rectangle; a comparison unit, used to compare the first circle fit and the second circle fit to obtain a comparison result, wherein the comparison result is used to determine whether the first circle fit is greater than the second circle fit; a second determination subunit, used to determine a target rectangle based on the comparison result, wherein the target rectangle is the rectangle with the largest circle fit between the first rectangle and the second rectangle; and a third determination subunit, used to determine a target central axis segment of a target circumscribed rectangle based on the target rectangle.

[0136] Optionally, the rotation processing unit includes: a first determination subunit, used to determine the target coordinate axis corresponding to the first image; a second determination subunit, used to determine the first coordinate and the second coordinate corresponding to the target central axis segment based on the target coordinate axis, wherein the first coordinate and the second coordinate are the coordinates corresponding to the starting point and the end point of the target central axis segment, respectively; a third determination subunit, used to determine the central axis angle of the target central axis based on the first coordinate, the second coordinate and the target coordinate axis, wherein the central axis angle is the angle between the target central axis and the axis of the target coordinate axis; a fourth determination subunit, used to determine the target rotation angle of the first area based on the central axis angle.

[0137] Optionally, the verification module includes: a cropping unit, used to crop the first image based on the first area to obtain a third image; a rotation processing unit, used to rotate the third image based on a target rotation angle to obtain a fourth image; a first determination unit, used to determine the target similarity between the fourth image and the second image, and determine a target score corresponding to the target similarity; a second determination unit, used to determine that the verification result is that the current object is the target object in the first image when the target score is greater than a preset score; and a third determination unit, used to determine that the verification result is that the current object is not the target object in the first image when the target score is less than or equal to the preset score.

[0138] Optionally, the segmentation processing module includes: a first segmentation processing unit, used to use the target segmentation network to perform segmentation processing on the first image to obtain a first area; wherein the target segmentation network includes: a convolution layer, an attention layer, and an upsampling layer, and the first image is segmented using the target segmentation network to obtain the first area, including: a convolution operation unit, used to perform a convolution operation on the first image using the convolution layer to obtain an initial feature map; a processing unit, used to process the initial feature map using the attention layer to obtain a target feature map; an upsampling processing unit, used to perform upsampling processing on the target feature map using the upsampling layer to obtain the first area.

[0139] Optionally, the upsampling processing unit includes: an upsampling processing subunit, used to use the upsampling layer to upsample the target feature map to obtain multiple sampling images; an adding subunit, used to add the features in the multiple sampling images to obtain the first area.

[0140] Optionally, the first determining unit is used to determine a first side of a target circumscribed rectangle, wherein the length of the first side is greater than the length of a second side of the target circumscribed rectangle, and the target circumscribed rectangle is a minimum circumscribed rectangle.

[0141] Example 3

[0142] According to another aspect of an embodiment of the present invention, a storage medium is further provided. The storage medium includes a stored program, wherein when the program is running, the device where the storage medium is located is controlled to execute any one of the image processing methods provided in the above embodiments.

[0143] Example 4

[0144] According to another aspect of an embodiment of the present invention, a computer terminal is provided, including: a processor and a memory, wherein the processor is used to run a program stored in the memory, wherein when the program is run, any one of the image processing methods provided in the above embodiments is executed.

[0145] The serial numbers of the above embodiments of the present invention are only for description and do not represent the advantages or disadvantages of the embodiments.

[0146] In the above embodiments of the present invention, the description of each embodiment has its own emphasis. For parts that are not described in detail in a certain embodiment, reference can be made to the relevant descriptions of other embodiments.

[0147] In the several embodiments provided in this application, it should be understood that the disclosed technical content can be implemented in other ways. Among them, the device embodiments described above are only schematic. For example, the division of the units can be a logical function division. There may be other division methods in actual implementation. For example, multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, indirect coupling or communication connection of units or modules, which can be electrical or other forms.

[0148] The units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed on multiple units. Some or all of the units may be selected according to actual needs to achieve the purpose of the present embodiment.

[0149] In addition, each functional unit in each embodiment of the present invention may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit. The above-mentioned integrated unit may be implemented in the form of hardware or in the form of software functional units.

[0150] If the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention, in essence, or the part that contributes to the prior art, or all or part of the technical solution can be embodied in the form of a software product, and the computer software product is stored in a storage medium, including a number of instructions for a computer device (which can be a personal computer, a server or a network device, etc.) to perform all or part of the steps of the method described in each embodiment of the present invention. The aforementioned storage medium includes: U disk, read-only memory (ROM, Read-Only Memory), random access memory (RAM, Random Access Memory), mobile hard disk, magnetic disk or optical disk and other media that can store program codes.

[0151] The above is only a preferred embodiment of the present invention. It should be pointed out that for ordinary technicians in this technical field, several improvements and modifications can be made without departing from the principle of the present invention. These improvements and modifications should also be regarded as the scope of protection of the present invention.

Claims

1. An image processing method, characterized in that: include: Acquire a first image and a second image, wherein the first image is a non-living image of the target object, and the second image is a living image of the current object acquired in real time; Segmenting the first image to obtain a first region, wherein the first region is a region where the target object is located; Rotate the first area based on a target circumscribed rectangle corresponding to the first area to obtain a target rotation angle; Verifying the second image based on the first area, the first image, and the target rotation angle to obtain a verification result, wherein the verification result is used to determine whether the current object in the second image is the target object in the first image; Rotating the first area based on a target circumscribed rectangle corresponding to the first area to obtain a target rotation angle includes: Determine a first side of the target circumscribed rectangle, wherein the length of the first side is greater than the length of the second side of the target circumscribed rectangle; Segmenting the target circumscribed rectangle based on the first side to obtain a first rectangle and a second rectangle; Determine a target central axis segment of the target circumscribed rectangle based on the first rectangle and the second rectangle, wherein the target central axis segment perpendicularly bisects the second side and is parallel to the first side of the target circumscribed rectangle; The first region is rotated based on the target central axis segment to obtain the target rotation angle.

2. The method according to claim 1, characterized in that Determining a target central axis segment of the target circumscribed rectangle based on the first rectangle and the second rectangle includes: Determine a first circle fitting degree corresponding to the first rectangle, and determine a second circle fitting degree corresponding to the second rectangle; Comparing the first circle fit degree and the second circle fit degree to obtain a comparison result, wherein the comparison result is used to determine whether the first circle fit degree is greater than the second circle fit degree; Determine a target rectangle based on the comparison result, wherein the target rectangle is a rectangle with the largest circle fitting degree between the first rectangle and the second rectangle; The target central axis segment of the target circumscribed rectangle is determined based on the target rectangle.

3. The method according to claim 2, characterized in that The first region is rotated based on the target central axis segment to obtain the target rotation angle, including: determining a target coordinate axis corresponding to the first image; Determine a first coordinate and a second coordinate corresponding to the target central axis segment based on the target coordinate axis, wherein the first coordinate and the second coordinate are coordinates corresponding to the starting point and the end point of the target central axis segment, respectively; Determine a center axis angle of the target center axis based on the first coordinate, the second coordinate and the target coordinate axis, wherein the center axis angle is an angle between the target center axis and an axis of the target coordinate axis; A target rotation angle of the first area is determined based on the central axis angle.

4. The method according to claim 1, characterized in that: Verifying the second image based on the first area and the target rotation angle to obtain a verification result of the target object includes: Cropping the first image based on the first area to obtain a third image; Rotate the third image based on the target rotation angle to obtain a fourth image; Determining a target similarity between the fourth image and the second image, and determining a target score corresponding to the target similarity; When the target score is greater than a preset score, determining that the verification result is that the current object is the target object in the first image; When the target score is less than or equal to the preset score, it is determined that the verification result is that the current object is not the target object in the first image.

5. The method according to claim 1, characterized in that Segmenting the first image to obtain a first region includes: Using a target segmentation network to segment the first image to obtain a first region; The target segmentation network includes: a convolution layer, an attention layer, and an upsampling layer. The target segmentation network is used to segment the first image to obtain a first region, including: Performing a convolution operation on the first image using the convolution layer to obtain an initial feature map; Processing the initial feature map using the attention layer to obtain a target feature map; The target feature map is upsampled by using the upsampling layer to obtain the first region.

6. The method according to claim 5, characterized in that The target feature image is upsampled by using the upsampling layer to obtain the first region, including: Performing upsampling processing on the target feature map by using the upsampling layer to obtain a plurality of sampled images; The features in the multiple sample images are added to obtain the first region.

7. The method according to claim 1, characterized in that The target bounding rectangle is a minimum bounding rectangle.

8. An image processing device, characterized in that: include: An acquisition module, configured to acquire a first image and a second image, wherein the first image is a non-living image of a target object, and the second image is a living image of a current object acquired in real time; A segmentation processing module, used for performing segmentation processing on the first image to obtain a first area, wherein the first area is an area where the target object is located; A rotation processing module, used for performing a rotation processing on the first area based on a target circumscribed rectangle corresponding to the first area to obtain a target rotation angle; a verification module, configured to verify the second image based on the first area, the first image, and the target rotation angle to obtain a verification result, wherein the verification result is used to determine whether the current object in the second image is the target object in the first image; The rotation processing module also includes: determining the first side of the target circumscribed rectangle, wherein the length of the first side is greater than the length of the second side of the target circumscribed rectangle; segmenting the target circumscribed rectangle based on the first side to obtain a first rectangle and a second rectangle; determining a target central axis segment of the target circumscribed rectangle based on the first rectangle and the second rectangle, wherein the target central axis segment perpendicularly bisects the second side and is parallel to the first side of the target circumscribed rectangle; rotating the first area based on the target central axis segment to obtain the target rotation angle.

9. A storage medium, characterized in that: The storage medium includes a stored program, wherein when the program is executed, the device where the storage medium is located is controlled to execute any one of the image processing methods in claims 1 to 7.

10. A computer terminal, characterized in that: include: A processor and a memory, wherein the processor is used to run a program stored in the memory, wherein the program executes any one of the image processing methods of claims 1 to 7 when running.

Images