A method, apparatus, device and storage medium for image processing

Abstract

The present disclosure provides a kind of image processing method, device, equipment and storage medium, it is related to image processing field, especially it is related to target detection and image intelligent cutting field.The specific implementation scheme is: target detection is carried out to target object in input image, and the frame of target object is obtained;According to input image and the frame of target object, determine multiple candidate image regions;From multiple candidate image regions, determine a best candidate region;Then based on the frame of target object in best candidate region, input image is cut, so that the image effect obtained by cutting is better.

Description

Technical Field

[0001] This disclosure relates to the field of image processing technology, and more particularly to the field of object detection and intelligent image cropping technology. Background Technology

[0002] With the continuous development of internet media, images are everywhere. However, product pages often have requirements for the proportion of images displayed. The same image is often cropped into different proportions for display on different pages and on different clients (PC, mobile, etc.). Summary of the Invention

[0003] This disclosure provides a method, apparatus, device, and storage medium for image processing.

[0004] According to one aspect of this disclosure, an image processing method is provided, comprising: performing object detection on an input image to obtain at least one object detection result, the object detection result including the bounding box of a target object; determining at least one image region containing the target object based on the input image and the at least one object detection result; determining an image region from the at least one image region as a first target region; and cropping the input image based on the first target region according to the bounding box of the target object and cropping parameters to obtain a second target region, such that the bounding box of each target object in the second target region is complete.

[0005] According to another aspect of this disclosure, an image processing apparatus is provided, comprising: a target detection module for performing target detection on an input image to obtain at least one target detection result, the target detection result including a bounding box of a target object; a candidate image region determination module for determining at least one image region containing the target object based on the input image and the at least one target detection result; an optimal image region determination module for determining an image region from the at least one image region as a first target region; and a cropping region determination module for cropping the input image based on the first target region according to the bounding box of the target object and cropping parameters to obtain a second target region, such that the bounding box of each target object within the second target region is complete.

[0006] According to another aspect of this disclosure, an electronic device is provided, comprising: at least one processor; and a memory communicatively connected to the at least one processor; wherein the memory stores instructions executable by the at least one processor, the instructions being executed by the at least one processor to enable the at least one processor to perform the image processing method described above.

[0007] According to another aspect of this disclosure, a non-transitory computer-readable storage medium is provided storing computer instructions, wherein the computer instructions are used to cause the computer to perform the above-described image processing method.

[0008] According to another aspect of this disclosure, a computer program product is provided, comprising a computer program that, when executed by a processor, implements the above-described image processing method.

[0009] This disclosure provides a method, apparatus, device, and storage medium for image processing. The method performs target detection on an input image and crops the image based on the detected target object's bounding box, resulting in a better cropped image.

[0010] It should be understood that the description in this section is not intended to identify key or essential features of the embodiments of this disclosure, nor is it intended to limit the scope of this disclosure. Other features of this disclosure will become readily apparent from the following description. Attached Figure Description

[0011] The accompanying drawings are provided to better understand this solution and do not constitute a limitation of this disclosure. Wherein:

[0012] Figure 1 This is a schematic flowchart illustrating the image processing method implemented according to the first embodiment of this disclosure;

[0013] Figure 2 This is a schematic flowchart illustrating the image processing method according to the second embodiment of this disclosure;

[0014] Figure 3 This is a schematic diagram of a face frame and a modification thereof according to the second embodiment of this disclosure;

[0015] Figure 4 This is a schematic diagram of at least one image region determined based on a face according to the second embodiment of this disclosure;

[0016] Figure 5 This is a schematic diagram of the image region of the second embodiment of this disclosure;

[0017] Figure 6 This is a schematic diagram of another image region according to the second embodiment of this disclosure;

[0018] Figure 7 This is a schematic diagram of the cutting effect of the second embodiment of this disclosure;

[0019] Figure 8 This is a flowchart illustrating the method for image processing implemented according to the third embodiment of this disclosure;

[0020] Figure 9 This is a schematic diagram of the structure of an image processing apparatus according to an embodiment of the present disclosure;

[0021] Figure 10 This is a schematic diagram of the composition structure of an electronic device used to implement the image processing method according to an embodiment of the present disclosure. Detailed Implementation

[0022] The exemplary embodiments of this disclosure are described below with reference to the accompanying drawings, including various details of the embodiments to aid understanding, and should be considered merely exemplary. Therefore, those skilled in the art will recognize that various changes and modifications can be made to the embodiments described herein without departing from the scope and spirit of this disclosure. Similarly, for clarity and brevity, descriptions of well-known functions and structures are omitted in the following description.

[0023] In this disclosure, an image mainly refers to a digital image that can be processed by a computer, obtained by shooting, scanning, copying, or cropping, including attributes such as pixels, resolution, size, color, bit depth, hue, saturation, brightness, color channels, and image layers.

[0024] Image cropping refers to the process of determining an image region (imagesubset) that meets the specified cropping parameters from an image and copying that image region as a new image.

[0025] The cropping parameters include the size of the cropping area, the aspect ratio, and the defined cropping range.

[0026] Image cropping can be further divided into regular cropping and irregular cropping.

[0027] Regular cropping refers to cropping an image within a rectangular bounding area. The cropping position can be determined by the coordinates of the top-left and bottom-right corners, and then the pixels within that rectangle can be copied.

[0028] Irregular cropping refers to cropping an image whose boundary is an arbitrary polygon, making it impossible to determine the cropping position using the coordinates of the top left and bottom right corners. Instead, a complete closed polygonal region of arbitrary shape must be generated beforehand, and then the pixels in this closed polygonal region can be copied.

[0029] Figure 1 The main flow of an image processing method according to an embodiment of the present disclosure is shown. Referring to FIG1, the method includes:

[0030] Operation S110: Perform object detection on the input image to obtain at least one object detection result, the object detection result including the bounding box of the target object;

[0031] The input image is the original image to be cropped, which can be an image file of any format or a two-dimensional matrix representing pixels, etc.

[0032] Object detection, also known as object extraction, is an image segmentation technique based on the geometric and statistical features of objects. Specifically, it involves identifying target objects from an input image and determining their classification and location. The location of the target object is represented by its bounding box. Typically, the bounding box is a rectangle, which can be represented by the coordinates of a vertex / center point, its length, and its width.

[0033] In the embodiments of this disclosure, the target object can be flexibly specified according to implementation needs and application scenarios, such as: a face, a vehicle, a product, etc. If no target object is specified, any human body or object identified from the image can be determined as the target object.

[0034] Operation S120: Based on the input image and at least one target detection result, determine at least one image region containing the target object;

[0035] If there is only one target object in the input image, then there is only one image region containing the target object; if there are multiple target objects in the input image, then the image region containing the target objects can be multiple image regions containing each target object, or multiple image regions containing any number of target objects, etc.

[0036] The aforementioned image regions can serve as candidate image regions for cropping the input image and as the basis for image cropping, ensuring that the cropped image at least contains the target object.

[0037] Operation S130: Determine an image region as a first target region from at least one image region;

[0038] The first target region is an image region selected from at least one image region, which contains the target object to be displayed within the cropping region.

[0039] If there is only one image region containing the target object determined in operation S120, then that image region is the first target region; if there are two or more image regions containing the target object determined in operation S120, then one of them with better display effect can be selected as the first target region.

[0040] Thus, once the first target area is determined, it becomes possible to determine which target objects(s) will be displayed in the clipping area and to define the main extent of the clipping area.

[0041] Operation S140: Based on the bounding box of the target object and the cropping parameters, the input image is cropped based on the first target region to obtain the second target region, so that the bounding box of each target object in the second target region is complete.

[0042] Cropping the input image based on the first target region refers to: fine-tuning the first target region to obtain the region to be cropped, which includes further cropping the first target region or adding part of the image region from the input image to meet the cropping parameters; and then cropping (pixel copying) the region to be cropped to obtain the second target region.

[0043] The second target region is the output image obtained by cropping the input image.

[0044] In this embodiment, during the process of fine-tuning the first target area to obtain the area to be cropped, it is also considered whether the border of the target object is complete in the second target area: if it is complete, the area obtained by fine-tuning can be output as the second target area; if it is not complete, it can continue to fine-tune or enlarge or shorten to the left, right, up, or down until the border of each target object in the second target area is complete.

[0045] Of course, in actual implementation, if it is difficult to ensure that the border of each target object in the second target area is complete, then it is acceptable to sacrifice a few target objects to make most target objects complete.

[0046] In this embodiment of the disclosure, operation 110 performs target detection on the input image to obtain the bounding box of at least one target object, thus enabling targeted cropping based on the position of the target object. Then, operation 120 determines at least one image region based on the bounding boxes of these target objects as a candidate image region for the cropping region, ensuring that the cropping region contains at least one target object. Next, operation 130 determines which target object(s) to display in the cropping region and sets the image region containing these target objects as the first target object. Then, operation 140 performs image cropping based on these target objects and fine-tunes the cropping parameters based on the bounding boxes of the target objects to determine a second target region, ensuring that the bounding box of each target object within the second target region is complete.

[0047] This ensures that key items or key people are complete within the cropped image, avoiding or reducing miscropping where key information is obviously missing.

[0048] Figure 2 The flowchart illustrating another embodiment of the image processing method disclosed herein is shown. This embodiment is applied to a scenario involving intelligent cropping of portrait photos, particularly group photos.

[0049] In this application scenario, the input image is a single person photo or a group photo, and the target object for object detection is a face.

[0050] Specifically, such as Figure 2 As shown, the process of intelligently cropping photos mainly includes:

[0051] Operate S210 to perform face detection on the input photo;

[0052] In this embodiment of the disclosure, multiple faces in an input photo can be detected to obtain the detection results of multiple faces.

[0053] Since the face area is smaller when the number of people in a group photo exceeds 10, the probability of cropping half of a face is greatly reduced. Therefore, when the number of faces in the input photo exceeds 10, the following method can be used for image acquisition: retain the middle area for horizontal images and retain the top area for vertical images, in order to save computing power.

[0054] Input the photo and face detection results into the image cropping module to perform the following operations:

[0055] Operate S220 to correct the border of the face according to the preset correction rules;

[0056] Typically, the face detection result includes a bounding box containing the face, such as Figure 3 The face bounding box 301, with the midpoint dashed, only includes the facial features of the person and does not include other head information such as the forehead.

[0057] To further prevent the forehead and other head information of the subject from being mistakenly cropped out, in this embodiment, the face frame is expanded upwards, to the left, or to the right according to a certain proportion to obtain an expanded face frame, such as... Figure 3 The face bounding box 302 is shown by the dashed line. The above correction rules can be preset via code or configuration file.

[0058] However, due to the movement and posture of the human head, the edges of the rectangles 301 and 302 may not be vertical or horizontal, but rather have a certain angle. The image area enclosed by such a border is difficult to process. Therefore, in this embodiment, an external rectangle with vertical or horizontal edges is calculated outside the expanded face frame 302, such as... Figure 3 The face bounding box with a solid line is shown in figure 303.

[0059] In this embodiment, the modified face bounding box 303 is used as the edge bounding box of the target object. On the one hand, it can include the head information of the person as much as possible and reduce the loss of information; on the other hand, it can simplify subsequent processing and save computing resources by using the vertical or horizontal circumscribed rectangle.

[0060] Operation S230: Based on at least one face result from the input photo, determine the image region containing each N faces, and obtain at least one image region, where N is a natural number;

[0061] Here, the image region containing a face is the image region enclosed by the face bounding box, for example... Figure 3 The image area enclosed by the face frame 303. Depending on the specific implementation needs, the implementer can further expand the image area enclosed by the face frame 303 proportionally to ensure that the entire head information of the person can be included in the image area.

[0062] An image region containing every N faces can be at least one image region obtained by performing a full permutation and combination of the image regions containing each face. For example, an image region containing every 2 faces; an image region containing every 3 faces; and so on, up to an image region containing every M faces (M being the total number of faces).

[0063] by Figure 4 Taking the photograph shown as an example, in this embodiment of the disclosure, a full permutation of the image regions containing each face is calculated, resulting in: image regions 401 and 402 containing each face; image region 403 containing every two faces; a total of three image regions: 401, 402 and 403.

[0064] This allows for the acquisition of as many image regions as possible, providing more options and resulting in a better cropped image.

[0065] In other embodiments of this disclosure, the image region containing each N faces may also be a subset formed by selecting some permutations from the above full permutations according to a preset strategy, in order to further simplify the calculation and save computing resources.

[0066] Operation S240: According to the preset scoring rules, at least one image region is scored to obtain a score for each image region;

[0067] The scoring rules for evaluating image regions can be pre-defined based on specific image cropping requirements or expert experience. For example, if the image cropping requirements focus on image quality, scoring rules can be formulated based on quality-related indicators such as sharpness and color saturation, resulting in higher scores for higher quality images. If the image cropping requirements focus on composition, scoring can be formulated based on the distance and relative position between the target object and the center point of the input image, resulting in higher scores for objects that are more centered or closer to the golden ratio.

[0068] Scoring rules can be a tiered scoring table based on a certain indicator, or a function, formula, or model based on one or more indicators.

[0069] Specifically, in the embodiments of this disclosure, such as Figure 5 As shown, the image region 50 obtained by operation 240 includes a first target object 501 with a complete border and a second target object 502 with an incomplete border.

[0070] Among them, the first target object 501 is a first face with a complete face bounding box, and the second target object 502 is a second face with an incomplete face bounding box, which is an incomplete face that may be mistakenly cropped.

[0071] Considering that when cropping group photos, image regions containing more people or those containing the central person are usually selected, in this embodiment of the disclosure, two main factors are considered when scoring each image region: one is the effective area of ​​each image region, where a larger effective area usually contains more people; the other is the distance between each image region and the center point of the input photo, where a shorter distance indicates that the face is more centered and the corresponding person is more important.

[0072] The effective area is the area of ​​the image region minus the area of ​​the second target object.

[0073] Figure 5 The rectangle 503 enclosed by the dashed line is the sliding window to be cropped. Its size, especially its width, can be determined according to the cropping parameters. The content enclosed by this cropping window represents the content that may be displayed within the cropping area.

[0074] The aforementioned cropping window will slide from left to right within the input photo to determine the effective area of ​​each image region.

[0075] Specifically, it can be calculated using the following formula:

[0076] Sv = Sc - min(Se)

[0077] in,

[0078] Sv is the effective area of ​​the image region;

[0079] Sc represents the area of ​​the image region;

[0080] When Se is the area of ​​the sliding window to be cropped, it includes the area of ​​the second target object 502, which is the area of ​​the erroneous cropping that may have cropped other faces.

[0081] by Figure 5For example, Sc is the area of ​​image region 50; Se is the area of ​​the left image region located on the left side of the left side of the cropping window 503 when the cropping window 503 moves out of the second target object 502; Sv is the area of ​​the right image region located on the left side of the left side of the cropping window 503 when the cropping window 503 moves out of the second target object 502.

[0082] The distance between the image region and the center point of the input photo can be calculated using the following formula:

[0083]

[0084] in:

[0085] D is the distance between the center point of the image region and the center point of the input photo.

[0086] Xp represents the x-coordinate of the center point of the input photo;

[0087] Yp represents the coordinates y of the center point of the input photo;

[0088] Xc is the coordinate x of the center point of the image region;

[0089] Yc represents the coordinates of the center point of the image region.

[0090] After obtaining the effective area of ​​each image region and the distance of each image region from the center point of the input photo, the F-Score algorithm is used to calculate the final score of each image region:

[0091]

[0092] in,

[0093] F represents the final score;

[0094] β is a constant coefficient, which defaults to 1;

[0095] Sv is the effective area of ​​the image region;

[0096] D is the distance between the center point of the image region and the center point of the image.

[0097] By using the F-Score algorithm, the final score of each image region is calculated based on the effective area of ​​each image region and the distance of each image region from the center point of the input photo. This can find an optimal balance between the effective area of ​​the image and the distance from the center point, thus taking into account both the number of people and the selection of important people.

[0098] Operation S250: Based on the score of each image region, determine an image region from at least one image region as the first target region;

[0099] Typically, when determining an image region as the primary target region from at least one image region, the region with the highest score is selected; however, it is not excluded that in some cases, an image region with a higher score may be chosen. For example, if the highest-scoring region has a flaw that is difficult to conceal, that image region can be discarded, and the image region with the next highest score can be selected instead.

[0100] In this embodiment of the disclosure, by scoring each image region and selecting the image region with the highest score as the first target region, the results are more objective and controllable, and the effect is better.

[0101] Operation S260: Crop the input photo based on the first target region to obtain the second target region;

[0102] In this embodiment of the disclosure, an image convergence process is performed based on the first target region to further exclude information related to potentially miscropped faces.

[0103] Assuming the first target region includes a third target object with complete borders (e.g., faces 601 and 602) and a fourth target object with incomplete borders (e.g., faces 603 and 604), then based on the borders of the target objects and cropping parameters, the input image is cropped according to the first target region to obtain a second target region, including:

[0104] 1) Determine the first clipping area containing the third target object based on the border of the third target object;

[0105] by Figure 6 Taking the input photo 60 as an example, the main focus is on determining the face within the first target region Rc and expanding the range of the face: Extend the complete faces 601 and 602 within the first target region Rc to the left and right respectively by the width of the face, and construct the circumscribed rectangles to obtain... Figure 6 The rectangle shown by the dashed line is designated as the first clipping range, which is also the minimum clipping range Rmin.

[0106] 2) Determine the second clipping range that does not include the fourth target object based on the border of the fourth target object;

[0107] by Figure 6 Taking the input photo 60 as an example, this mainly refers to identifying faces outside the first target region Rc or incomplete faces within the region, and expanding the range of faces: Extend the faces 603 and 604 outside the first target region Rc to the left and right respectively by the width of the face, and construct an inscribed rectangle to obtain... Figure 6 The rectangle indicated by the dashed line is denoted as the maximum clipping range Rmax.

[0108] Extending the face to the left and right by the width of the face itself can include information about the outer part of the face, such as the shoulders, making the information about the person more complete.

[0109] 3) Based on the first cropping range, the second cropping range, and the cropping parameters, the input image is cropped to obtain the second target region.

[0110] The second target region is the region to be cropped. The region to be cropped can be determined by combining the first target region Rc, the maximum cropping range Rmax, and the minimum cropping range Rmin, expressed by the following formula:

[0111]

[0112] in,

[0113] X left This represents the X-coordinate value of the theoretically ideal starting point of the trimming area;

[0114] X Rc This represents the X-coordinate value of Rc;

[0115] WRc represents the width of Rc;

[0116] WRmin represents the width of Rmin;

[0117] The specific meaning of the above rules is as follows:

[0118] When W Rc >W Rmin At that time, X left Take the value of f1;

[0119] When W Rc =W Rmin At that time, X left Take X Rc The value;

[0120] When W Rc <W Rmin At that time, X left Take the value of f2.

[0121] The formula for f1 is as follows:

[0122]

[0123] in,

[0124] X Rmin Represents the X-coordinate value of Rmin;

[0125] The specific meaning of the above rules is as follows:

[0126] When X Rc >X RminWhen f1 takes X Rmin The value;

[0127] When X Rmin +W Rmin >W Rc +X Rc When f1 takes X Rmin +W Rmin -W Rc The value of .

[0128] The formula for f2 is as follows:

[0129] SP left =X Rmin +Len1-X Rc

[0130] SP right =X Rmin +W Rc -((X Rmin +W Rmin -Lnn2)

[0131]

[0132] in,

[0133] Len1 is the width of the leftmost face in the first target region, with... Figure 6 For example, the width of a face is 601;

[0134] Len2 is the width of the rightmost face in the first target region, with... Figure 6 For example, the width of a face is 602;

[0135] SP left This represents the remaining distance between the leftmost face and the leftmost side of the first target area;

[0136] SP right This indicates the remaining distance between the rightmost face and the rightmost side of the first target area.

[0137] The starting position (top left corner) of the area to be cropped, which is the second target area, is:

[0138] X Fleft =min(max(X) left X Rmax ,0),W image -W Rc )

[0139] in,

[0140] W image Enter the width of the photo.

[0141] Specifically, the second target area is:

[0142] Starting point coordinate system (X) Fleft ,0), width W Rc High H Rc The area.

[0143] Through the above image convergence process, the layout of the cropped image can be more reasonable, the face and face-related information can be more complete, and the display effect can be further improved.

[0144] Operate S270 to output the second target region as the cropping result.

[0145] In this embodiment of the disclosure, faces outside the first target region or incomplete faces within the first target region are excluded from the second target region as much as possible. This ensures that the faces within the second target region are as complete as possible.

[0146] Figure 7 The diagram shows a comparison of the image cropping effects achieved using the image processing method described above in the embodiments of this disclosure, and using a conventional centering method for image cropping.

[0147] Photo 701 is the input photo; Photo 702 is the cropping result obtained by cropping the image using the traditional centering method; Photo 703 is the cropping result obtained by cropping the image using the image processing method of this disclosure.

[0148] As can be seen from the cropping results, the cropping results obtained by using the image processing method of this disclosure can reduce the situation of accidentally cropping out parts of the face, making the cropping results more reasonable, easier to accept, and providing a better user experience.

[0149] Figure 8 The process of implementing an image processing method according to another embodiment of this disclosure is illustrated. Compared to Figure 2 In the embodiment shown, the input photo is further subject-detected, and the image regions of the input photo are prioritized to obtain image regions with high priority. Accordingly, at least one image region is scored to obtain a score for each image region, including: scoring at least one image region according to the order of priority from low to high to obtain a score for each image region; if the score of the current priority image region is greater than the scoring threshold, then the image region of the next priority is no longer scored.

[0150] Specifically, in Figure 8 The image cropping process for the input photo shown in the embodiment of this disclosure mainly includes:

[0151] Operate S810 to perform subject detection on the input photo and obtain the subject detection results;

[0152] Subject detection refers to identifying various objects contained in the input photo, including faces, human bodies, and other objects, and merging the identification results of various objects to obtain a single subject with only one bounding box. This single subject is the subject detection result obtained by subject detection in this embodiment of the disclosure.

[0153] The detection results of this subject can include a variety of important information, rather than just focusing on the face.

[0154] Operate the S820 to perform face detection on the input photo and obtain the face detection results;

[0155] Input the photo, subject detection results, and face detection results into the image cropping module and perform the following operations:

[0156] The S830 is operated to correct the border of the face according to the preset correction rules;

[0157] Operate the S840 to perform region classification on the input photo;

[0158] In this embodiment of the disclosure, the following will be:

[0159] The input photo will be used as the low-optimization area;

[0160] The main body detection results will be used as the medium priority area;

[0161] The facial regions are merged to form a high-priority region.

[0162] Operation S850: Determine whether the low-priority area meets the clipping parameters. If yes, continue to step S890; otherwise, continue to operation S860.

[0163] Operation S860: Determine whether the medium priority region meets the clipping parameters. If yes, continue to step S890; otherwise, continue to operation S870.

[0164] Operation S870: Determine whether the high-priority area meets the clipping parameters. If yes, continue to step S890; otherwise, continue to operation S880.

[0165] Operate S880 to continue subsequent processing and obtain the second target area;

[0166] Operate S890 to output the corresponding image region as the cropping result.

[0167] In this embodiment of the disclosure, the image regions of the input photo are classified and processed in order of priority from low to high. When the low priority region can meet the cropping parameters, no further processing is performed. This makes the entire image processing method more efficient, eliminating the need for subsequent complex calculations, thereby greatly saving computing resources and making the processing speed faster.

[0168] The acquisition, storage, and application of user personal information involved in the technical solution disclosed herein comply with the provisions of relevant laws and regulations and do not violate public order and good morals.

[0169] According to embodiments of this disclosure, this disclosure also provides an image processing apparatus, as shown in FIG9. The apparatus 90 includes: a target detection module 901, configured to perform target detection on an input image to obtain at least one target detection result, the target detection result including the bounding box of a target object; a candidate image region determination module 902, configured to determine at least one image region containing the target object based on the input image and at least one target detection result; an optimal image region determination module 903, configured to score the at least one image region according to a preset scoring rule to obtain a score for each image region, and determine an image region as a first target region from the at least one image region based on the score of each image region; and a cropping region determination module 904, configured to crop the input image based on the bounding box of the target object and cropping parameters, based on the first target region, to obtain a second target region, such that the bounding box of each target object within the second target region is complete.

[0170] According to one embodiment of this disclosure, the device 90 further includes: a border correction module, used to correct the border of the target object according to a preset correction rule.

[0171] According to one embodiment of the present disclosure, the candidate image region determination module 902 is specifically used to determine an image region containing every N target objects based on the input image and at least one target detection result, thereby obtaining at least one image region, where N is a natural number.

[0172] According to an embodiment of this disclosure, the optimal image region determination module 903 includes: a scoring submodule, used to score at least one image region according to a preset scoring rule to obtain a score for each image region; and an optimal image region determination module submodule, used to determine an image region as a first target region from at least one image region based on the score of each image region.

[0173] According to an embodiment of this disclosure, the first target region includes a third target object with a complete border and a fourth target object with an incomplete border. Accordingly, the cropping region determination module 904 includes: a first cropping range determination submodule, used to determine a first cropping range containing the third target object based on the border of the third target object; a second cropping range determination submodule, used to determine a second cropping range not containing the fourth target object based on the border of the fourth target object; and a cropping region determination submodule, used to crop the input image based on the first cropping range, the second cropping range, and cropping parameters to obtain the second target region.

[0174] According to one embodiment of this disclosure, the device 90 further includes: a region classification module, used to perform region classification processing on the input image to obtain at least one image region with priority from low to high; a cropping region determination module, further used to process each image region in the at least one image region according to the priority from low to high; if the current priority image region has met the cropping parameters, then the subsequent steps are skipped and the current priority image region is used as the second target region.

[0175] According to embodiments of this disclosure, this disclosure also provides an electronic device, a readable storage medium, and a computer program product.

[0176] Figure 10 A schematic block diagram of an example electronic device 1000 that can be used to implement embodiments of the present disclosure is shown. The electronic device is intended to represent various forms of digital computers, such as laptop computers, desktop computers, workstations, personal digital assistants, servers, blade servers, mainframe computers, and other suitable computers. The electronic device may also represent various forms of mobile devices, such as personal digital processors, cellular phones, smartphones, wearable devices, and other similar computing devices. The components shown herein, their connections and relationships, and their functions are merely illustrative and are not intended to limit the implementation of the present disclosure described and / or claimed herein.

[0177] like Figure 10 As shown, device 1000 includes a computing unit 1001, which can perform various appropriate actions and processes according to a computer program stored in read-only memory (ROM) 1002 or a computer program loaded into random access memory (RAM) 1003 from storage unit 1008. RAM 1003 may also store various programs and data required for the operation of device 1000. The computing unit 1001, ROM 1002, and RAM 1003 are interconnected via bus 1004. Input / output (I / O) interface 1005 is also connected to bus 1004.

[0178] Multiple components in device 1000 are connected to I / O interface 1005, including: input unit 1006, such as keyboard, mouse, etc.; output unit 1007, such as various types of monitors, speakers, etc.; storage unit 1008, such as disk, optical disk, etc.; and communication unit 1009, such as network card, modem, wireless transceiver, etc. Communication unit 1009 allows device 1000 to exchange information / data with other devices through computer networks such as the Internet and / or various telecommunications networks.

[0179] The computing unit 1001 can be various general-purpose and / or special-purpose processing components with processing and computing capabilities. Some examples of the computing unit 1001 include, but are not limited to, a central processing unit (CPU), a graphics processing unit (GPU), various special-purpose artificial intelligence (AI) computing chips, various computing units running machine learning model algorithms, a digital signal processor (DSP), and any suitable processor, controller, microcontroller, etc. The computing unit 1001 performs the various methods and processes described above, such as the image processing method of the embodiments of this disclosure. For example, in some embodiments, the image processing method of this disclosure can be implemented as a computer software program tangibly contained in a machine-readable medium, such as storage unit 1008. In some embodiments, part or all of the computer program can be loaded and / or installed on device 1000 via ROM 1002 and / or communication unit 1009. When the computer program is loaded into RAM 1003 and executed by the computing unit 1001, one or more steps of the image processing method of the embodiments of this disclosure described above can be performed. Alternatively, in other embodiments, the computing unit 1001 may be configured to perform the image processing method of the present disclosure embodiments by any other suitable means (e.g., by means of firmware).

[0180] Various embodiments of the systems and techniques described above herein can be implemented in digital electronic circuit systems, integrated circuit systems, field-programmable gate arrays (FPGAs), application-specific integrated circuits (ASICs), application-specific standard products (ASSPs), systems-on-a-chip (SoCs), payload-programmable logic devices (CPLDs), computer hardware, firmware, software, and / or combinations thereof. These various embodiments may include implementations in one or more computer programs that can be executed and / or interpreted on a programmable system including at least one programmable processor, which may be a dedicated or general-purpose programmable processor, capable of receiving data and instructions from a storage system, at least one input device, and at least one output device, and transferring data and instructions to the storage system, the at least one input device, and the at least one output device.

[0181] The program code used to implement the methods of this disclosure may be written in any combination of one or more programming languages. This program code may be provided to a processor or controller of a general-purpose computer, special-purpose computer, or other programmable data processing apparatus, such that when executed by the processor or controller, the program code causes the functions / operations specified in the flowcharts and / or block diagrams to be implemented. The program code may be executed entirely on a machine, partially on a machine, as a standalone software package partially on a machine and partially on a remote machine, or entirely on a remote machine or server.

[0182] In the context of this disclosure, a machine-readable medium can be a tangible medium that may contain or store a program for use by or in conjunction with an instruction execution system, apparatus, or device. A machine-readable medium can be a machine-readable signal medium or a machine-readable storage medium. Machine-readable media can be, but is not limited to, electronic, magnetic, optical, electromagnetic, infrared, or semiconductor systems, apparatus, or devices, or any suitable combination of the foregoing. More specific examples of machine-readable storage media include electrical connections based on one or more wires, portable computer disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fiber, portable compact disk read-only memory (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination of the foregoing.

[0183] To provide interaction with a user, the systems and techniques described herein can be implemented on a computer having: a display device for displaying information to the user (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor); and a keyboard and pointing device (e.g., a mouse or trackball) through which the user provides input to the computer. Other types of devices can also be used to provide interaction with the user; for example, feedback provided to the user can be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user can be received in any form (including sound input, voice input, or tactile input).

[0184] The systems and technologies described herein can be implemented in computing systems that include backend components (e.g., as a data server), or computing systems that include middleware components (e.g., an application server), or computing systems that include frontend components (e.g., a user computer with a graphical user interface or web browser through which a user can interact with embodiments of the systems and technologies described herein), or any combination of such backend, middleware, or frontend components. The components of the system can be interconnected via digital data communication of any form or medium (e.g., a communication network). Examples of communication networks include local area networks (LANs), wide area networks (WANs), and the Internet.

[0185] Computer systems can include clients and servers. Clients and servers are generally located far apart and typically interact via communication networks. Client-server relationships are created by computer programs running on the respective computers and having a client-server relationship with each other. Servers can be cloud servers, servers in distributed systems, or servers incorporating blockchain technology.

[0186] It should be understood that the various forms of processes shown above can be used to rearrange, add, or delete steps. For example, the steps described in this disclosure can be executed in parallel, sequentially, or in different orders, as long as the desired result of the technical solution disclosed in this disclosure can be achieved, and this is not limited herein.

[0187] The specific embodiments described above do not constitute a limitation on the scope of protection of this disclosure. Those skilled in the art should understand that various modifications, combinations, sub-combinations, and substitutions can be made according to design requirements and other factors. Any modifications, equivalent substitutions, and improvements made within the spirit and principles of this disclosure should be included within the scope of protection of this disclosure.

Claims

1. An image processing method, comprising: The input image is subjected to object detection to obtain at least one object detection result, the object detection result including the bounding box of the target object; Based on the input image and the at least one target detection result, at least one image region containing the target object is determined, the image region including a first target object with a complete border and a second target object with an incomplete border; According to a preset scoring rule, the at least one image region is scored to obtain a score for each image region. The preset scoring rule is based on the effective area of ​​each image region and / or the distance between each image region and the center point of the input image, wherein the effective area is the area of ​​the image region minus the area of ​​the second target object. Based on the score of each image region, an image region is determined from the at least one image region as a first target region; Based on the border of the target object and the cropping parameters, the input image is cropped based on the first target region to obtain a second target region, so that the border of each target object in the second target region is complete.

2. The method according to claim 1, further comprising: The border of the target object is corrected according to the preset correction rules.

3. The method according to claim 1, wherein, Determining at least one image region containing the target object based on the input image and the at least one target detection result includes: Based on the input image and the at least one target detection result, an image region containing every N target objects is determined to obtain at least one image region, where N is a natural number.

4. The method according to claim 1, wherein, The preset scoring rule is based on a combined score, F-Score, which is the effective area of ​​each image region and the distance between each image region and the center point of the input image.

5. The method according to claim 1, wherein, The first target region includes a third target object with complete borders and a fourth target object with incomplete borders. Accordingly, the step of cropping the input image based on the first target region according to the bounding box and cropping parameters of the target object to obtain the second target region includes: Based on the border of the third target object, determine a first clipping range containing the third target object; Based on the border of the fourth target object, determine a second clipping range that does not include the fourth target object; The input image is cropped according to the first cropping range, the second cropping range, and the cropping parameters to obtain the second target region.

6. The method according to claim 1, further comprising: The input image is subjected to region classification processing to obtain at least one image region with priority from low to high. Each image region in the at least one image region is processed in order of priority from low to high; If the current priority image region already meets the cropping parameters, skip the subsequent steps and use the current priority image region as the second target region.

7. An image processing apparatus, comprising: The target detection module is used to perform target detection on the input image to obtain at least one target detection result, wherein the target detection result includes the bounding box of the target object; A candidate image region determination module is used to determine at least one image region containing the target object based on the input image and the at least one target detection result, wherein the image region includes a first target object with a complete border and a second target object with an incomplete border; The scoring submodule is used to score the at least one image region according to a preset scoring rule to obtain a score for each image region. The preset scoring rule is based on the effective area of ​​each image region and / or the distance between each image region and the center point of the input image, wherein the effective area is the area of ​​the image region minus the area of ​​the second target object. The optimal image region determination module submodule is used to determine an image region as a first target region from the at least one image region based on the score of each image region; The cropping region determination module is used to crop the input image based on the first target region according to the border of the target object and the cropping parameters to obtain a second target region, so that the border of each target object in the second target region is complete.

8. The apparatus according to claim 7, further comprising: The border correction module is used to correct the border of the target object according to preset correction rules.

9. The apparatus according to claim 7, wherein, The candidate image region determination module is specifically used to determine an image region containing every N target objects based on the input image and the at least one target detection result, thereby obtaining at least one image region, where N is a natural number.

10. The apparatus according to claim 7, wherein, The first target region includes a third target object with complete borders and a fourth target object with incomplete borders. Accordingly, the cropping area determination module includes: The first cropping range determination submodule is used to determine the first cropping range containing the third target object based on the border of the third target object; The second cropping range determination submodule is used to determine a second cropping range that does not include the fourth target object based on the border of the fourth target object. The cropping region determination submodule is used to crop the input image based on the first cropping range, the second cropping range, and cropping parameters to obtain a second target region.

11. The apparatus according to claim 7, further comprising: The region classification module is used to perform region classification processing on the input image to obtain at least one image region with priority from low to high. The cropping region determination module is also used to process each image region in the at least one image region according to the order of priority from low to high; if the current priority image region has met the cropping parameters, the subsequent steps are skipped and the current priority image region is used as the second target region.

12. An electronic device, comprising: At least one processor; as well as A memory communicatively connected to the at least one processor; wherein, The memory stores instructions that can be executed by the at least one processor to enable the at least one processor to perform the method of any one of claims 1-6.

13. A non-transitory computer-readable storage medium storing computer instructions, wherein, The computer instructions are used to cause the computer to perform the method according to any one of claims 1-6.

14. A computer program product comprising a computer program that, when executed by a processor, implements the method according to any one of claims 1-6.

Images