Image processing method, device, medium and electronic device

Abstract

The present disclosure relates to an image processing method, device, medium and electronic equipment. The method comprises: obtaining a to-be-cropped image; determining a perspective correction angle of the to-be-cropped image; detecting a face image region in the to-be-cropped image as a first reference region; adjusting the first reference region by the perspective correction angle to obtain a second reference region, and adjusting the to-be-cropped image according to the perspective correction angle to obtain a first sub-image; adjusting the first sub-image according to the position of the second reference region in the first sub-image, and determining a target cropped image according to the adjusted first sub-image. In this way, all faces or most of the main faces in the image can be used to adjust the composition, better processing the image of a group photo, and further determining the perspective correction angle required by the image to adjust the composition of the to-be-cropped image, better processing the shooting angle of the image, so as to achieve a better intelligent cropping effect.

Description

Technical Field

[0001] This disclosure relates to the field of image processing, and more specifically, to an image processing method, apparatus, medium, and electronic device. Background Technology

[0002] In real-life scenarios, due to users' lack of knowledge about image composition, it's common to capture unattractive or crooked images when taking pictures of people. Intelligent image cropping technology can help users improve the presentation of less-than-ideal images through cropping. Current technologies, when processing facial images, typically crop and adjust the image directly based on the location of the largest face in the image, which often yields poor results in group photos. Summary of the Invention

[0003] The purpose of this disclosure is to provide an image processing method, apparatus, medium, and electronic device that can not only crop images based on the largest face in the image, but also use the face image area containing all or most of the main faces in the image for cropping composition, thereby better processing group photos. Furthermore, it can determine the required viewing angle correction angle of the image and use rotation to crop the image to be cropped, thereby better handling the shooting angle of the image and achieving better intelligent cropping effects in group photos.

[0004] To achieve the above objectives, this disclosure provides an image processing method, the method comprising:

[0005] Obtain the image to be cropped;

[0006] Determine the viewpoint correction angle of the image to be cropped;

[0007] The face image region in the image to be cropped is used as the first reference region;

[0008] The first reference region is adjusted by the viewpoint correction angle to obtain the second reference region, and the image to be cropped is adjusted according to the viewpoint correction angle to obtain the first sub-image.

[0009] The first sub-image is adjusted according to the position of the second reference region in the first sub-image, and the target cropped image is determined based on the adjusted first sub-image.

[0010] Optionally, determining the viewpoint correction angle of the image to be cropped includes:

[0011] Identify at least one first straight line included in the image to be cropped;

[0012] Clustering is performed on the at least one first straight line to obtain at least one cluster;

[0013] The second straight line corresponding to each cluster is determined based on the first straight line included in each cluster.

[0014] Based on the length of the second line, the angle between it and the preset coordinate axis, and the positional relationship between it and the vanishing point in the image to be cropped, a third line from the second line that can be used to calculate the viewpoint correction angle is selected.

[0015] The angle between the longest line in the third straight line and the preset coordinate axis is used as the viewpoint correction angle.

[0016] Optionally, clustering the at least one first straight line to obtain at least one cluster includes:

[0017] Traverse the positional relationships between each first straight line and determine whether the positional relationships meet preset conditions;

[0018] The two first straight lines whose positional relationship meets the preset conditions are divided into the same cluster;

[0019] The preset conditions include: the distances from both endpoints of one first straight line to the other first straight line are both less than a first threshold; the difference between the angles between the two first straight lines and the preset coordinate axes is less than a second threshold; and the closest distance from the endpoint of one first straight line to the endpoint of the other first straight line is less than a third threshold.

[0020] Optionally, the step of selecting a third line from the second line that can be used to calculate the viewpoint correction angle based on the length of the second line, the angle between the second line and a preset coordinate axis, and the positional relationship between the third line and the vanishing point in the image to be cropped includes:

[0021] The length of the target line is determined based on the length of the longest line in the second line.

[0022] The line in the second line whose length is not less than the length of the target line, and the line whose angle with the preset coordinate axis is not greater than the fourth threshold, are determined as the fourth line;

[0023] Vanishing point detection is performed on the image to be detected based on the fourth straight line to obtain target vanishing points, wherein the target vanishing points are one or more.

[0024] The third straight line is defined as the straight line that does not pass through the target vanishing point, and the straight line that passes through any of the target vanishing points and whose angle with the plane of the image to be detected is not greater than the fifth threshold.

[0025] Optionally, detecting the face image region in the image to be cropped as the first reference region includes:

[0026] Detect all face regions in the image to be cropped;

[0027] The largest facial region is assigned to a preset set;

[0028] Iterate through the distance between each face region in the preset set and the face regions not in the preset set;

[0029] When the distance meets the preset condition, the corresponding face region that is not in the preset set is added to the preset set;

[0030] Repeat the step of traversing the distance between each face region in the preset set and the face regions not in the preset set until the distance between the face regions not in the preset set and any face region in the preset set no longer meets the preset condition;

[0031] The smallest bounding rectangle of all face regions in the preset set is taken as the face image region, and the face image region is taken as the first reference region.

[0032] Optionally, adjusting the first sub-image based on the position of the second reference region within the first sub-image, and determining the target cropped image based on the adjusted first sub-image, includes:

[0033] Obtain pre-set reference position points and / or reference lines in the first sub-image;

[0034] Determine the target reference point or target reference line that is closest to the second reference area;

[0035] The first sub-image is cropped to obtain a second sub-image, so that the second reference region partially coincides with the target reference position point or the target reference line, and the target cropped image is determined based on the second sub-image.

[0036] Optionally, the method further includes:

[0037] The salient regions in the image to be cropped are detected as the third reference regions;

[0038] Determining the target cropped image based on the second sub-image includes:

[0039] Determine the position of the third reference region in the second sub-image;

[0040] The second sub-image is cropped to obtain a third sub-image, such that the third reference region is located in the center region of the third sub-image;

[0041] The third sub-image is used as the target cropped image.

[0042] This disclosure also provides an image processing apparatus, the apparatus comprising:

[0043] The acquisition module is used to acquire the image to be cropped.

[0044] The first processing module is used to determine the viewing angle correction angle of the image to be cropped;

[0045] The second processing module is used to detect the face image region in the image to be cropped as the first reference region;

[0046] The third processing module is used to adjust the first reference region by the viewpoint correction angle to obtain the second reference region, and to adjust the image to be cropped according to the viewpoint correction angle to obtain the first sub-image.

[0047] The cropping module is used to adjust the first sub-image according to the position of the second reference region in the first sub-image, and to determine the target cropped image according to the adjusted first sub-image.

[0048] This disclosure also provides a non-transitory computer-readable storage medium having a computer program stored thereon that, when executed by a processor, implements the steps of the method described above.

[0049] This disclosure also provides an electronic device, including:

[0050] A memory on which computer programs are stored;

[0051] A processor is configured to execute the computer program in the memory to implement the steps of the method described above.

[0052] Through the above technical solution, when performing intelligent cropping on images including faces, it can not only adjust the image based on the face with the largest area in the image, but also adjust the composition using the face image area where all or most of the main faces are located in the image, thus better handling group photos. Furthermore, it can further determine the required viewing angle correction angle of the image and use rotation to adjust the composition of the image to be cropped, better handling the shooting angle of the image, thereby achieving better intelligent cropping results in group photos.

[0053] Other features and advantages of this disclosure will be described in detail in the following detailed description section. Attached Figure Description

[0054] The accompanying drawings are provided to further illustrate the present disclosure and form part of the specification. They are used together with the following detailed description to explain the present disclosure, but do not constitute a limitation thereof. In the drawings:

[0055] Figure 1 This is a flowchart illustrating an image processing method according to an exemplary embodiment of the present disclosure.

[0056] Figure 2 This is a schematic diagram illustrating an image processing method according to an exemplary embodiment of the present disclosure, in which an image to be cropped is cropped based on a viewing angle correction.

[0057] Figure 3 This is a flowchart illustrating an image processing method according to yet another exemplary embodiment of the present disclosure.

[0058] Figure 4 This is a flowchart illustrating an image processing method according to yet another exemplary embodiment of the present disclosure.

[0059] Figure 5 This is a flowchart illustrating an image processing method according to yet another exemplary embodiment of the present disclosure.

[0060] Figure 6 This is a schematic diagram illustrating the distribution of reference position points and reference lines in an image processing method according to yet another exemplary embodiment of this disclosure.

[0061] Figure 7 This is a flowchart illustrating an image processing method according to yet another exemplary embodiment of the present disclosure.

[0062] Figure 8 This is a structural block diagram of an image processing apparatus according to an exemplary embodiment of the present disclosure.

[0063] Figure 9 This is a block diagram illustrating an electronic device according to an exemplary embodiment.

[0064] Figure 10 This is a block diagram illustrating an electronic device according to an exemplary embodiment. Detailed Implementation

[0065] The specific embodiments of this disclosure will be described in detail below with reference to the accompanying drawings. It should be understood that the specific embodiments described herein are for illustration and explanation only and are not intended to limit this disclosure.

[0066] Figure 1 This is a flowchart illustrating an image processing method according to an exemplary embodiment of the present disclosure. Figure 1As shown, the method includes steps 101 to 105.

[0067] In step 101, the image to be cropped is obtained.

[0068] In step 102, the viewpoint correction angle of the image to be cropped is determined. This viewpoint correction angle can be achieved using various image angle correction methods. For example, it can be determined by vanishing point detection of the image, or it can be determined using any rotation correction method for the image.

[0069] In step 103, the face image region in the image to be cropped is detected as the first reference region.

[0070] The face image region may include one or more faces. When there are a large number of faces in the image to be cropped, instead of referring to only one face image as the reference for image cropping, all faces in the image to be cropped will be detected, and a face image region that can represent the location of all or most of the faces in the image to be cropped will be determined as the first reference region.

[0071] In step 104, the first reference region is adjusted by the viewpoint correction angle to obtain the second reference region, and the image to be cropped is adjusted according to the viewpoint correction angle to obtain the first sub-image.

[0072] In step 105, the first sub-image is adjusted according to the position of the second reference region in the first sub-image, and the target cropped image is determined based on the adjusted first sub-image.

[0073] When adjusting the image to be cropped using the aforementioned perspective correction angle, it can be done as follows: Figure 2 The method shown is used to perform this. For example... Figure 2 As shown, the first image frame 1 represents the image to be cropped, and the second image frame 2 represents the image after the first image frame 1 has been rotated according to the viewing angle correction. The largest inscribed rectangle is found in the second image frame 2, and the image within the second image frame 2 is adjusted according to this inscribed rectangle to become the first sub-image. Figure 2 The third image frame 3 shown.

[0074] When adjusting the first reference area using the aforementioned perspective correction angle, in order to avoid the following... Figure 2 The rotated black frame shown, also known as the fourth image frame 4, can be directly determined according to... Figure 2 The clipping parameters shown are used to clip the first reference region. Figure 2As shown, after rotation and searching the inscribed rectangle, the first sub-image and the black frame generated after one rotation of the image to be cropped, that is, the fourth image frame 4, are finally determined to have a certain distance on the top, bottom, left and right sides. This distance can be used as the cropping parameter. Then, the first reference area is cropped directly according to the cropping parameter corresponding to the first sub-image to obtain the second reference area.

[0075] The position of the second reference region in the first sub-image can be characterized by the position of the center point of the second reference region in the first sub-image. Similarly, the position of the first reference region in the image to be cropped can also be characterized by the position of the center point of the second reference region in the image to be cropped.

[0076] When cropping the first sub-image based on the position of the second reference region within the first sub-image, the specific cropping strategy can be determined according to actual needs. For example, if it is desired that the second reference region is positioned closer to the center of the first sub-image, the first sub-image can be adjusted accordingly to center the second reference region within the adjusted first sub-image. Finally, when determining the target cropped image based on the adjusted first sub-image, the first sub-image can be directly selected as the target cropped image, or the adjusted first sub-image can be further optimized before determining the target cropped image.

[0077] Through the above technical solution, when performing intelligent cropping on images including faces, it can not only adjust the image based on the face with the largest area in the image, but also adjust the composition using the face image area where all or most of the main faces are located in the image, thus better handling group photos. Furthermore, it can further determine the required viewing angle correction angle of the image and use rotation to adjust the composition of the image to be cropped, better handling the shooting angle of the image, thereby achieving better intelligent cropping results in group photos.

[0078] Figure 3 This is a flowchart illustrating an image processing method according to yet another exemplary embodiment of the present disclosure. For example... Figure 3 As shown, the method further includes steps 301 to 305.

[0079] In step 301, at least one first straight line included in the image to be cropped is determined.

[0080] The first straight line can be any straight line detected in the image to be cropped, or it can be obtained after preprocessing the detected straight lines in the image. For example, existing straight line detection algorithms, such as EDLines and LSD, can be used to detect straight lines in the image. Then, based on the angles between all detected straight lines and the distances between them, the lines can be initially merged to avoid a complete straight line in the image to be cropped being detected as multiple line segments due to detection errors. The straight line obtained after this initial merging preprocessing can then be used as the first straight line for the next step.

[0081] In step 302, the at least one first straight line is clustered to obtain at least one cluster.

[0082] The method for clustering the first straight line can be as follows: traverse the positional relationships between each first straight line and determine whether the positional relationships meet preset conditions; group the two first straight lines whose positional relationships meet the preset conditions into the same cluster; wherein the preset conditions include: the distances from both endpoints of one first straight line to the other first straight line are both less than a first threshold; the difference in the angles between the two first straight lines and the preset coordinate axes is less than a second threshold; and the closest distance from the endpoint of one first straight line to the endpoint of the other first straight line is less than a third threshold. The method for determining the closest distance can be as follows: for example, first straight line A has two endpoints (a first endpoint and a second endpoint), and first straight line B has two endpoints (a third endpoint and a fourth endpoint). The distance between the first endpoint of first straight line A and the third endpoint of first straight line B is 5, and the distance between the first endpoint of first straight line A and the fourth endpoint of first straight line B is 14. The distance between the second endpoint of first straight line A and the third endpoint of first straight line B is 6, and the distance between the second endpoint of first straight line A and the fourth endpoint of first straight line B is 11. Therefore, the closest distance from the endpoint of first straight line A to the endpoint of first straight line B is 5.

[0083] Furthermore, the method of clustering is not limited in this disclosure. In addition to the clustering method exemplified above, other methods can also be used to cluster the first line.

[0084] In step 303, a second straight line corresponding to each cluster is determined based on the first straight line included in each cluster.

[0085] The method for determining the second line corresponding to each cluster can be as follows: For all the first lines in a cluster, find the endpoints of the two first lines that are farthest apart as the two endpoints of the first line corresponding to this cluster. The two farthest endpoints can be the two endpoints of the same first line or the endpoints of two different first lines.

[0086] In step 304, based on the length of the second straight line, the angle between it and the preset coordinate axis, and the positional relationship between it and the vanishing point in the image to be cropped, a third straight line that can be used to calculate the viewpoint correction angle is selected from the second straight lines.

[0087] The above-described step of filtering the second straight line into a third straight line that can be used to calculate the viewpoint correction angle based on the length of the second straight line, the angle between the second straight line and the preset coordinate axis, and the positional relationship between the second straight line and the vanishing point in the image to be cropped, can filter the second straight line from three angles: length, angle, and positional relationship with the vanishing point. The third straight line obtained by the final filtering is the third straight line that can be used to calculate the viewpoint correction angle.

[0088] Specifically, the selection process for the third straight line can be as follows: determine the target straight line length based on the length of the longest straight line among the second straight lines; determine the fourth straight line as the straight line whose length is not less than the target straight line length and the straight line whose angle with the preset coordinate axis is not greater than a fourth threshold; perform vanishing point detection on the image to be detected based on the fourth straight line to obtain target vanishing points, wherein the target vanishing points are one or more; determine the third straight line as the straight line among the fourth straight lines that does not pass through the target vanishing points and the straight line that passes through any of the target vanishing points and whose angle with the plane of the image to be detected is not greater than a fifth threshold.

[0089] The method for vanishing point detection in the above screening process can be any existing vanishing point detection method, and this disclosure does not limit the method. The length of the target line can be half the length of the longest line. For example, if the length of the longest line in the second line is length, then the length of the target line can be 0.5*length.

[0090] In step 305, the angle between the longest line in the third straight line and the preset coordinate axis is taken as the viewpoint correction angle.

[0091] In one possible embodiment, the third straight line may not exist, in which case the viewpoint correction angle will also be zero, indicating that the image to be cropped does not require viewpoint correction.

[0092] The above technical solution discloses a method for determining the viewpoint correction angle corresponding to an image to be cropped. It can determine the angle that the image to be cropped needs to be adjusted by detecting straight lines in the image to be cropped, and then crop and compose the image to be cropped according to the viewpoint correction angle, so as to better handle the shooting viewpoint of the image.

[0093] Figure 4 This is a flowchart illustrating an image processing method according to yet another exemplary embodiment of the present disclosure. For example... Figure 4 As shown, the method further includes steps 401 to 406.

[0094] In step 401, all face regions in the image to be cropped are detected.

[0095] This disclosure does not limit the specific face detection algorithm; any detection algorithm that can detect faces in the image to be cropped is acceptable.

[0096] After obtaining the detection result of the face region, preprocessing can be performed to reduce the possibility of inaccurate face image regions due to the accuracy issues of the face detection algorithm. This preprocessing could, for example, involve identifying the largest face region among all face regions, determining an area threshold based on this largest area (e.g., 0.25 of the largest area), and finally removing detected face regions whose area does not reach the area threshold from the detection results.

[0097] In step 402, the face region with the largest area is assigned to a preset set.

[0098] In step 403, the distance between each face region in the preset set and the face regions not in the preset set is traversed.

[0099] In step 404, when the distance meets the preset condition, the corresponding face region that is not in the preset set is added to the preset set.

[0100] This preset set serves as the set of face image regions. After face region detection, the preset set is initially empty. The largest face region in the image to be cropped is then assigned to this preset set, thus including that largest face region. Next, each face region outside the preset set is iterated over. If the distance between a face region outside the preset set and any face region within the preset set meets a preset condition, then that face region outside the preset set is added to the preset set.

[0101] The preset condition can be determined separately based on the size of each face region in the preset set. That is, each time it is determined whether the distance between a face region outside the preset set and a face region in the preset set meets the preset condition, it needs to be determined in real time based on the size of the corresponding face region in the preset set. Specifically, the preset condition may include: the distance between a face region outside the preset set and a face region in the preset set is less than three times the longer side of the corresponding face region in the preset set.

[0102] In step 405, it is determined whether the distance between a face region not in the preset set and any face region in the preset set does not meet the preset condition. If yes, proceed to step 406; otherwise, proceed to step 403.

[0103] That is, the traversal operation in step 403 will be repeated until the distance between a face region not in the preset set and any face region in the preset set does not meet the preset condition.

[0104] In step 406, the smallest bounding rectangle of all face regions in the preset set is taken as the face image region, and the face image region is taken as the first reference region.

[0105] Figure 5 This is a flowchart illustrating an image processing method according to yet another exemplary embodiment of the present disclosure. For example... Figure 5 As shown, the method further includes steps 501 to 503.

[0106] In step 501, a pre-set reference position point and / or reference line in the first sub-image is obtained.

[0107] In step 502, the target reference location point or target reference line that is closest to the second reference area is determined.

[0108] In step 503, the first sub-image is cropped to obtain a second sub-image, so that the second reference region partially coincides with the target reference position point or the target reference line, and the target cropped image is determined based on the second sub-image.

[0109] The pre-set reference points and / or reference lines represent the most suitable locations or reference lines in the image to be cropped for displaying the main image. These are independent of the specific content of the image and can be uniformly set in advance based on the size of the image to be cropped or the first sub-image. The image quality is better when the main image is displayed on these reference points and / or reference lines.

[0110] Figure 6This illustrates the distribution of reference point 5 and / or reference line 6 within the first sub-image. The fifth image frame 7 can be the second reference region within the first sub-image, and its center point 8 is the midline point of that second reference region. Figure 6 As shown, the closest reference line to the second reference region is the reference line 6 to the right of the center point 8, therefore, this reference line 6 can be determined as the target reference line. An adjustment method to make the second reference region partially coincide with the target reference point or the target reference line could, for example, be to crop a portion of the right side of the first sub-image so that the corresponding reference line in the adjusted image size passes exactly through the center point 8. The specific adjustment method can be determined in real-time based on the target reference line or the target reference point.

[0111] Figure 7 This is a flowchart illustrating an image processing method according to yet another exemplary embodiment of the present disclosure. For example... Figure 7 As shown, the method further includes steps 701 to 704.

[0112] In step 701, a salient region in the image to be cropped is detected as a third reference region.

[0113] In step 702, the position of the third reference region in the second sub-image is determined.

[0114] In step 703, the second sub-image is cropped to obtain a third sub-image, such that the third reference region is located in the center region of the third sub-image.

[0115] In step 704, the third sub-image is used as the target cropped image.

[0116] The method for determining the salient region can be any salientity monitoring algorithm. After determining the third reference region and obtaining the second sub-image, the second sub-image can be further cropped based on the third reference region where the salient region is located.

[0117] For example, after determining the position of the third reference region within the second sub-image, the distances of the four boundaries of the third reference region from the four boundaries of the second sub-image can be known, denoted as left, right, top, and bottom. Then, for left and right, if left is smaller, the boundary of the second sub-image corresponding to right is cropped to be equal to left; for top and bottom, if top is smaller, the boundary of the second sub-image corresponding to bottom is cropped to be equal to top. This achieves the effect of the third reference region being the center region of the third sub-image obtained after cropping.

[0118] The above technical solution can further optimize the cropping effect by cropping the image based on its salient features.

[0119] Figure 8 This is a structural block diagram of an image processing apparatus according to an exemplary embodiment of the present disclosure. Figure 8 As shown, the device includes: an acquisition module 10 for acquiring an image to be cropped; a first processing module 20 for determining a viewing angle correction angle of the image to be cropped; a second processing module 30 for detecting a face image region in the image to be cropped as a first reference region; a third processing module 40 for adjusting the first reference region using the viewing angle correction angle to obtain a second reference region, and adjusting the image to be cropped according to the viewing angle correction angle to obtain a first sub-image; and a cropping module 50 for adjusting the first sub-image according to the position of the second reference region in the first sub-image, and determining a target cropped image based on the adjusted first sub-image.

[0120] Through the above technical solution, when performing intelligent cropping on images including faces, it can not only adjust the image based on the face with the largest area in the image, but also adjust the composition using the face image area where all or most of the main faces are located in the image, thus better handling group photos. Furthermore, it can further determine the required viewing angle correction angle of the image and use rotation to adjust the composition of the image to be cropped, better handling the shooting angle of the image, thereby achieving better intelligent cropping results in group photos.

[0121] In one possible implementation, the first processing module 20 is further configured to: determine at least one first straight line included in the image to be cropped; cluster the at least one first straight line to obtain at least one cluster; determine a second straight line corresponding to each cluster based on the first straight line included in each cluster; filter a third straight line among the second straight lines that can be used to calculate the viewpoint correction angle based on the length of the second straight line, the angle between the second straight line and a preset coordinate axis, and the positional relationship between the second straight line and the vanishing point in the image to be cropped; and take the angle between the longest third straight line and the preset coordinate axis as the viewpoint correction angle.

[0122] In one possible implementation, the first processing module 20 is further configured to: traverse the positional relationships between each first straight line and determine whether the positional relationships meet preset conditions; divide two first straight lines whose positional relationships meet the preset conditions into the same cluster; wherein the preset conditions include: the distances from both endpoints of one first straight line to the other first straight line are both less than a first threshold; the difference between the angles between the two first straight lines and the preset coordinate axis is less than a second threshold; and the closest distance from the endpoint of one first straight line to the endpoint of the other first straight line is less than a third threshold.

[0123] In one possible implementation, the first processing module 20 is further configured to: determine the length of a target line based on the length of the longest line among the second lines; determine the lines among the second lines whose length is not less than the target line length, and the lines whose angle with the preset coordinate axis is not greater than a fourth threshold, as the fourth line; perform vanishing point detection on the image to be detected based on the fourth line to obtain target vanishing points, wherein the target vanishing points are one or more; and determine the lines among the fourth lines that do not pass through the target vanishing points, and the lines among the lines that pass through any of the target vanishing points and whose angle with the plane where the image to be detected is not greater than a fifth threshold, as the third line.

[0124] In one possible implementation, the second processing module 30 is further configured to: detect all face regions in the image to be cropped; divide the face region with the largest area into a preset set; traverse the distance between each face region in the preset set and a face region not in the preset set; when the distance meets a preset condition, add the corresponding face region not in the preset set to the preset set; repeat the step of traversing the distance between each face region in the preset set and a face region not in the preset set until the distance between a face region not in the preset set and any face region in the preset set no longer meets the preset condition; use the smallest bounding rectangle of all face regions in the preset set as the face image region, and use the face image region as the first reference region.

[0125] In one possible implementation, the cropping module 50 is further configured to: acquire a pre-set reference position point and / or reference line in the first sub-image; determine a target reference position point or target reference line that is closest to the second reference region; crop the first sub-image to obtain a second sub-image, so that the second reference region partially coincides with the target reference position point or the target reference line; and determine the target cropped image based on the second sub-image.

[0126] In one possible implementation, the apparatus further includes: a fourth processing module (not shown) for detecting a salient region in the image to be cropped as a third reference region; the cropping module 50 is further configured to: determine the position of the third reference region in the second sub-image; crop the second sub-image to obtain a third sub-image such that the third reference region is located in the center region of the third sub-image; and use the third sub-image as the target cropped image.

[0127] Regarding the apparatus in the above embodiments, the specific manner in which each module performs its operation has been described in detail in the embodiments related to the method, and will not be elaborated upon here.

[0128] Figure 9 This is a block diagram illustrating an electronic device 900 according to an exemplary embodiment. For example... Figure 9 As shown, the electronic device 900 may include a processor 901 and a memory 902. The electronic device 900 may also include one or more of a multimedia component 903, an input / output (I / O) interface 904, and a communication component 905.

[0129] The processor 901 controls the overall operation of the electronic device 900 to complete all or part of the steps in the image processing method described above. The memory 902 stores various types of data to support the operation of the electronic device 900. This data may include, for example, instructions for any application or method operating on the electronic device 900, and application-related data such as contact data, sent and received messages, pictures, audio, video, etc. The memory 902 can be implemented by any type of volatile or non-volatile storage device or a combination thereof, such as Static Random Access Memory (SRAM), Electrically Erasable Programmable Read-Only Memory (EEPROM), Erasable Programmable Read-Only Memory (EPROM), Programmable Read-Only Memory (PROM), Read-Only Memory (ROM), magnetic storage, flash memory, magnetic disk, or optical disk. The multimedia component 903 may include a screen and audio components. The screen may be, for example, a touchscreen, and the audio component is used to output and / or input audio signals. For example, the audio component may include a microphone for receiving external audio signals. The received audio signals may be further stored in memory 902 or transmitted via communication component 905. The audio component also includes at least one speaker for outputting audio signals. I / O interface 904 provides an interface between processor 901 and other interface modules, such as a keyboard, mouse, buttons, etc. These buttons may be virtual or physical buttons. Communication component 905 is used for wired or wireless communication between the electronic device 900 and other devices. Wireless communication, such as Wi-Fi, Bluetooth, Near Field Communication (NFC), 2G, 3G, 4G, NB-IoT, eMTC, or other 5G technologies, or combinations thereof, is not limited here. Therefore, the corresponding communication component 905 may include: a Wi-Fi module, a Bluetooth module, an NFC module, etc.

[0130] In an exemplary embodiment, the electronic device 900 may be implemented by one or more application-specific integrated circuits (ASICs), digital signal processors (DSPs), digital signal processing devices (DSPDs), programmable logic devices (PLDs), field-programmable gate arrays (FPGAs), controllers, microcontrollers, microprocessors, or other electronic components to perform the image processing method described above.

[0131] In another exemplary embodiment, a computer-readable storage medium including program instructions is also provided, which, when executed by a processor, implement the steps of the image processing method described above. For example, the computer-readable storage medium may be the memory 902 including program instructions described above, which may be executed by the processor 901 of the electronic device 900 to complete the image processing method described above.

[0132] Figure 10 This is a block diagram illustrating an electronic device 1000 according to an exemplary embodiment. For example, the electronic device 1000 may be provided as a server. (Refer to...) Figure 10 The electronic device 1000 includes a processor 1022, which may be one or more, and a memory 1032 for storing computer programs executable by the processor 1022. The computer program stored in the memory 1032 may include one or more modules, each corresponding to a set of instructions. Furthermore, the processor 1022 may be configured to execute the computer program to perform the image processing method described above.

[0133] Additionally, the electronic device 1000 may also include a power supply component 1026 and a communication component 1050. The power supply component 1026 can be configured to perform power management of the electronic device 1000, and the communication component 1050 can be configured to enable communication of the electronic device 1000, such as wired or wireless communication. Furthermore, the electronic device 1000 may also include an input / output (I / O) interface 1058. The electronic device 1000 can operate on an operating system, such as Windows Server, stored in the memory 1032. TM Mac OS X TM Unix TM Linux TM etc.

[0134] In another exemplary embodiment, a computer-readable storage medium including program instructions is also provided, which, when executed by a processor, implement the steps of the image processing method described above. For example, the non-transitory computer-readable storage medium may be the memory 1032 including the program instructions described above, which may be executed by the processor 1022 of the electronic device 1000 to complete the image processing method described above.

[0135] In another exemplary embodiment, a computer program product is also provided, which includes a computer program executable by a programmable device, the computer program having a code portion for performing the image processing method described above when executed by the programmable device.

[0136] The preferred embodiments of this disclosure have been described in detail above with reference to the accompanying drawings. However, this disclosure is not limited to the specific details of the above embodiments. Within the scope of the technical concept of this disclosure, various simple modifications can be made to the technical solutions of this disclosure, and these simple modifications all fall within the protection scope of this disclosure.

[0137] It should also be noted that the various specific technical features described in the above embodiments can be combined in any suitable manner without contradiction. To avoid unnecessary repetition, this disclosure will not describe the various possible combinations separately.

[0138] Furthermore, various different embodiments of this disclosure can be combined in any way, as long as they do not violate the spirit of this disclosure, they should also be regarded as the content disclosed in this disclosure.

Claims

1. An image processing method, characterized in that, The method includes: Obtain the image to be cropped; Determine the viewpoint correction angle of the image to be cropped; The face image region in the image to be cropped is used as the first reference region; The first reference region is adjusted by the viewpoint correction angle to obtain the second reference region, and the image to be cropped is adjusted according to the viewpoint correction angle to obtain the first sub-image. Based on the position of the second reference region in the first sub-image, the first sub-image is adjusted, and the target cropped image is determined based on the adjusted first sub-image. The determination of the viewpoint correction angle of the image to be cropped includes: Identify at least one first straight line included in the image to be cropped; Clustering is performed on the at least one first straight line to obtain at least one cluster; The second straight line corresponding to each cluster is determined based on the first straight line included in each cluster. Based on the length of the second line, the angle between it and the preset coordinate axis, and the positional relationship between it and the vanishing point in the image to be cropped, a third line from the second line that can be used to calculate the viewpoint correction angle is selected. The angle between the longest line in the third straight line and the preset coordinate axis is used as the viewpoint correction angle.

2. The method according to claim 1, characterized in that, The step of clustering the at least one first straight line to obtain at least one cluster includes: Traverse the positional relationships between each first straight line and determine whether the positional relationships meet preset conditions; The two first straight lines whose positional relationship meets the preset conditions are divided into the same cluster; The preset conditions include: the distances from both endpoints of one first straight line to the other first straight line are both less than a first threshold; the difference between the angles between the two first straight lines and the preset coordinate axes is less than a second threshold; and the closest distance from the endpoint of one first straight line to the endpoint of the other first straight line is less than a third threshold.

3. The method according to claim 1, characterized in that, The step of selecting a third line from the second line that can be used to calculate the viewpoint correction angle based on the length of the second line, the angle between the second line and the preset coordinate axis, and the positional relationship between the third line and the vanishing point in the image to be cropped includes: The length of the target line is determined based on the length of the longest line in the second line. The line in the second line whose length is not less than the length of the target line, and the line whose angle with the preset coordinate axis is not greater than the fourth threshold, are determined as the fourth line; Vanishing point detection is performed on the image to be cropped based on the fourth straight line to obtain target vanishing points, wherein there are one or more target vanishing points; The third straight line is defined as the straight line that does not pass through the target vanishing point, and the straight line that passes through any of the target vanishing points and whose angle with the plane containing the image to be cropped is not greater than the fifth threshold.

4. The method according to claim 1, characterized in that, The step of detecting the face image region in the image to be cropped as the first reference region includes: Detect all face regions in the image to be cropped; The largest facial region is assigned to a preset set; Iterate through the distance between each face region in the preset set and the face regions not in the preset set; When the distance meets the preset condition, the corresponding face region that is not in the preset set is added to the preset set; Repeat the step of traversing the distance between each face region in the preset set and the face regions not in the preset set until the distance between the face regions not in the preset set and any face region in the preset set no longer meets the preset condition; The smallest bounding rectangle of all face regions in the preset set is taken as the face image region, and the face image region is taken as the first reference region.

5. The method according to claim 1, characterized in that, The step of adjusting the first sub-image based on the position of the second reference region in the first sub-image, and determining the target cropped image based on the adjusted first sub-image, includes: Obtain pre-set reference position points and / or reference lines in the first sub-image; Determine the target reference point or target reference line that is closest to the second reference area; The first sub-image is cropped to obtain a second sub-image, so that the second reference region partially coincides with the target reference position point or the target reference line, and the target cropped image is determined based on the second sub-image.

6. The method according to claim 5, characterized in that, The method further includes: The salient regions in the image to be cropped are detected as the third reference regions; Determining the target cropped image based on the second sub-image includes: Determine the position of the third reference region in the second sub-image; The second sub-image is cropped to obtain a third sub-image, such that the third reference region is located in the center region of the third sub-image; The third sub-image is used as the target cropped image.

7. An image processing apparatus, characterized in that, The device includes: The acquisition module is used to acquire the image to be cropped. The first processing module is used to determine the viewing angle correction angle of the image to be cropped; The second processing module is used to detect the face image region in the image to be cropped as the first reference region; The third processing module is used to adjust the first reference region by the viewpoint correction angle to obtain the second reference region, and to adjust the image to be cropped according to the viewpoint correction angle to obtain the first sub-image. The cropping module is used to adjust the first sub-image according to the position of the second reference region in the first sub-image, and to determine the target cropped image according to the adjusted first sub-image. The first processing module is further configured to: determine at least one first straight line included in the image to be cropped; cluster the at least one first straight line to obtain at least one cluster; determine a second straight line corresponding to each cluster based on the first straight line included in each cluster; filter a third straight line among the second straight lines that can be used to calculate the viewpoint correction angle based on the length of the second straight line, the angle between the second straight line and a preset coordinate axis, and the positional relationship between the second straight line and the vanishing point in the image to be cropped; and take the angle between the longest third straight line and the preset coordinate axis as the viewpoint correction angle.

8. A non-transitory computer-readable storage medium having a computer program stored thereon, characterized in that, When executed by a processor, the program implements the steps of the method described in any one of claims 1-6.

9. An electronic device, characterized in that, include: A memory on which computer programs are stored; A processor for executing the computer program in the memory to implement the steps of the method according to any one of claims 1-6.

Images