Image cropping method, device, equipment and storage medium

Abstract

The present disclosure relates to an image cropping method, apparatus, device, and storage medium. The method comprises the following steps: obtaining an image to be cropped; determining a core prediction map, a distance prediction map, and an angle prediction map for each sub-image based on a pre-trained image detection model; determining a core pixel set for each sub-image based on the probability values ​​of the pixels in the core prediction map; determining at least one distance value corresponding to the distance prediction map and at least one angle value corresponding to the angle prediction map for each core pixel in the core pixel set of each sub-image; determining a predicted page frame for each sub-image based on the coordinates of each core pixel in each sub-image, at least one distance value corresponding to each core pixel, and at least one angle value corresponding to each core pixel; and cropping the image to be cropped based on the predicted page frame to obtain a target cropped image. This improves the cropping effect of the image.

Description

Technical Field

[0001] The present disclosure relates to the field of image processing technology, and in particular to an image cropping method, apparatus, device, and storage medium. Background Art

[0002] With the rapid development of the multimedia era and the convenience and diversification of taking photos, users have more and more demands for image cropping.

[0003] However, when the complexity of an image is high, it is difficult to crop at least one sub-image from an image, and the cropping effect is poor. Summary of the Invention

[0004] In order to solve the above technical problems, the present disclosure provides an image cropping method, apparatus, device and storage medium.

[0005] In a first aspect, the present disclosure provides an image cropping method, the method comprising:

[0006] Acquire an image to be cropped, wherein the image to be cropped includes at least one sub-image;

[0007] Based on the pre-trained image detection model, determine the core prediction map of each sub-image, the distance prediction map of each sub-image, and the angle prediction map of each sub-image;

[0008] For each sub-image, determining a core pixel set of each sub-image based on probability values ​​of pixels in the core prediction image;

[0009] For each core pixel point in the core pixel set of each sub-image, determining at least one distance value corresponding to the core pixel point in the distance prediction map and at least one angle value corresponding to the core pixel point in the angle prediction map;

[0010] determining a predicted page frame of each sub-image based on the coordinates of each core pixel point in each sub-image, at least one distance value corresponding to each core pixel point, and at least one angle value corresponding to each core pixel point;

[0011] The image to be cropped is cropped based on the predicted page frame to obtain a target cropped image.

[0012] In a second aspect, the present disclosure provides an image cropping device, the device comprising:

[0013] An image to be cropped module, configured to obtain an image to be cropped, wherein the image to be cropped includes at least one sub-image;

[0014] An image detection module is used to determine a core prediction map, a distance prediction map, and an angle prediction map for each sub-image based on a pre-trained image detection model.

[0015] A first determining module is configured to determine, for each sub-image, a core pixel set of each sub-image based on probability values ​​of pixels in the core prediction image;

[0016] a second determining module, configured to determine, for each core pixel point in the core pixel set of each sub-image, at least one distance value corresponding to the core pixel point in the distance prediction map and at least one angle value corresponding to the core pixel point in the angle prediction map;

[0017] a third determining module, configured to determine a predicted page frame of each sub-image based on the coordinates of each core pixel point in each sub-image, at least one distance value corresponding to each core pixel point, and at least one angle value corresponding to each core pixel point;

[0018] The image cropping module is used to crop the image to be cropped based on the predicted page frame to obtain a target cropped image.

[0019] In a third aspect, an embodiment of the present disclosure further provides an image cropping device, the device comprising:

[0020] one or more processors;

[0021] a storage device for storing one or more programs,

[0022] When the one or more programs are executed by the one or more processors, the one or more processors implement the image cropping method provided in the first aspect.

[0023] In a fourth aspect, an embodiment of the present disclosure further provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the image cropping method provided in the first aspect.

[0024] The technical solution provided by the embodiments of the present disclosure has the following advantages over the prior art:

[0025] The present invention discloses an image cropping method, apparatus, device, and storage medium. The method comprises obtaining an image to be cropped, wherein the image to be cropped includes at least one sub-image; determining a core prediction map, a distance prediction map, and an angle prediction map for each sub-image based on a pre-trained image detection model; determining a core pixel set for each sub-image based on the probability values ​​of the pixels in the core prediction map; determining, for each core pixel in the core pixel set of each sub-image, at least one distance value corresponding to the distance prediction map and at least one angle value corresponding to the angle prediction map; determining a predicted page frame for each sub-image based on the coordinates of each core pixel in each sub-image, at least one distance value corresponding to each core pixel, and at least one angle value corresponding to each core pixel; and cropping the image to be cropped based on the predicted page frame to obtain a target cropped image. Thus, the method can accurately determine the predicted page frame by identifying the coordinates of the core pixels of the image to be detected, the distance values ​​corresponding to the core pixels, and the angle values ​​corresponding to the core pixels, and perform image cropping based on the predicted page frame. This method can accurately crop images even with high complexity, ensuring an effective cropping effect. BRIEF DESCRIPTION OF THE DRAWINGS

[0026] The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments consistent with the present disclosure and, together with the description, serve to explain the principles of the present disclosure.

[0027] In order to more clearly illustrate the embodiments of the present disclosure or the technical solutions in the prior art, the following briefly introduces the drawings required for use in the embodiments or the description of the prior art. Obviously, for ordinary technicians in this field, other drawings can be obtained based on these drawings without any creative work.

[0028] Figure 1 This is an architecture diagram of an image cropping system provided by an embodiment of the present disclosure;

[0029] Figure 2 A flowchart of an image cropping method provided by an embodiment of the present disclosure;

[0030] Figure 3 A core prediction map of a sub-image provided by an embodiment of the present disclosure;

[0031] Figure 4 A distance prediction map of a sub-image provided by an embodiment of the present disclosure;

[0032] Figure 5a An angle prediction map of a sub-image provided by an embodiment of the present disclosure;

[0033] Figure 5bAn angle prediction map of another sub-image provided by an embodiment of the present disclosure;

[0034] Figure 5c An angle prediction map of another sub-image provided by an embodiment of the present disclosure;

[0035] Figure 5d An angle prediction map of another sub-image provided by an embodiment of the present disclosure;

[0036] Figure 6 A schematic structural diagram of an image detection model provided by an embodiment of the present disclosure is shown;

[0037] Figure 7 An overall logic diagram of an image cropping method provided by an embodiment of the present disclosure is shown;

[0038] Figure 8 A schematic diagram showing the flow of the post-processing process in the image cropping process provided by an embodiment of the present disclosure is shown;

[0039] Figure 9 A schematic structural diagram of an image cropping device provided in an embodiment of the present disclosure;

[0040] Figure 10 A schematic structural diagram of an image cropping device provided in an embodiment of the present disclosure. DETAILED DESCRIPTION

[0041] In order to more clearly understand the above-mentioned objectives, features and advantages of the present disclosure, the scheme of the present disclosure will be further described below. It should be noted that, in the absence of conflict, the embodiments of the present disclosure and the features therein can be combined with each other.

[0042] In the following description, many specific details are set forth to facilitate a full understanding of the present disclosure, but the present disclosure may also be implemented in other ways different from those described herein; it is obvious that the embodiments in the specification are only part of the embodiments of the present disclosure, rather than all of the embodiments.

[0043] In existing educational and teaching scenarios, students will take pictures of multiple homework papers at the same time, so that the same image includes at least one sub-image corresponding to the homework paper. This will greatly increase the difficulty of image cropping, and thus cannot guarantee the accuracy of subsequent homework correction.

[0044] In order to solve the above problems, the embodiments of the present disclosure provide an image cropping method, apparatus, device and storage medium.

[0045] Figure 1 The figure shows an architecture diagram of an image cropping system provided by an embodiment of the present disclosure.

[0046] like Figure 1As shown, the architecture diagram may include an electronic device 101 and a server 102. The electronic device 101 may establish a connection with the server 102 and exchange information via a network protocol such as Hypertext Transfer Protocol over Secure Socket Layer (HTTPS). The electronic device 101 may include a mobile phone, tablet computer, desktop computer, laptop computer, or other device with communication capabilities. The server 102 may be a cloud server or server cluster, or other device with storage and computing capabilities.

[0047] Based on the above architecture, in some embodiments, when image cropping is required, the electronic device 101 may first obtain the image to be cropped and then crop the image to be cropped.

[0048] Based on the above architecture, in other embodiments, when image cropping is required, the electronic device 101 can first obtain the image to be cropped, and then send the image to be cropped to the server 102, and use the server 102 to crop the image to be cropped, and the electronic device 101 obtains the target cropped image sent by the server 102.

[0049] According to the above architecture, the following Figures 2 to 7 The image cropping method provided in the embodiments of the present disclosure is described. In the embodiments of the present disclosure, the image cropping method can be executed by an electronic device or a server. The electronic device may include a mobile phone, tablet computer, desktop computer, laptop computer, or other device with communication capabilities. The server may be a cloud server or server cluster, or other device with storage and computing capabilities. It should be noted that the following embodiments are explained exemplarily using an electronic device as the execution subject.

[0050] Figure 2 A flow chart of an image cropping method provided by an embodiment of the present disclosure is shown.

[0051] like Figure 2 As shown, the image cropping method may include the following steps.

[0052] S210: Acquire an image to be cropped, where the image to be cropped includes at least one sub-image.

[0053] In the embodiment of the present disclosure, the image to be cropped is any image that needs to be cropped. For example, the image to be cropped includes sub-images, each sub-image is a piece of work paper, and each piece of work paper needs to be cropped.

[0054] Optionally, the image to be cropped may be acquired in real time by an image acquisition device, or may be a pre-stored image.

[0055] S220 : Based on a pre-trained image detection model, determine a core prediction map for each sub-image, a distance prediction map for each sub-image, and an angle prediction map for each sub-image.

[0056] In the disclosed embodiment, the core prediction map of each sub-image refers to a prediction map including the core region of each sub-image. Specifically, the core region of each sub-image is an area defined by the center point of each sub-image and pixels whose distance from the center point is less than a preset distance, wherein the preset distance can be a distance pre-set as needed.

[0057] Alternatively, the core area may be half of the sub-image.

[0058] To facilitate understanding of the core prediction graph, such as Figure 3 As shown, the image to be cropped includes two sub-images, namely sub-image 31 and sub-image 32. Sub-image 31 includes a core area 33, and sub-image 32 includes a core area 34. The area of ​​the core area in each sub-image is half the area of ​​its corresponding sub-image.

[0059] In the disclosed embodiments, the distance prediction map for each sub-image refers to the distance value from each pixel in the core area to the corner point of its corresponding sub-image. Optionally, the corner points of the sub-image include the upper left corner, the upper right corner, the lower right corner, and the lower left corner. Optionally, the distance value can be Euclidean distance or other distance, which is not limited here.

[0060] In order to facilitate the understanding of the distance prediction graph, such as Figure 4 As shown, the image to be cropped includes a sub-image, in which the core area of ​​the sub-image is pixel K, the upper left corner point A, the upper right corner point B, the lower right corner point C, and the lower left corner point D. The distance between pixel point K and the upper left corner point A is L1, the distance between pixel point K and the upper right corner point B is L2, the distance between pixel point K and the lower right corner point C is L3, and the distance between pixel point K and the lower left corner point D is L4.

[0061] In the embodiment of the present disclosure, the angle prediction map of each sub-image refers to the angle value from each pixel point in the core area to the corner point on its corresponding sub-image.

[0062] Specifically, when calculating the angle, for each pixel point in the core area, we can first calculate the angle between the pixel point and the four vertices, and then calculate the average value of the angles between all pixels in the core area and the four vertices to obtain the distance between the pixel point and the vertex in the same core area.

[0063] Optionally, the angle between the pixel point in the core area and the upper left corner point ranges from π / 2 to π, the angle between the pixel point and the upper right corner point ranges from 0 to π / 2, the angle between the pixel point and the lower right corner point ranges from -π / 2 to 0, and the angle between the pixel point and the lower left corner point ranges from -π to -π / 2.

[0064] In order to facilitate the understanding of the angle prediction diagram, such as Figure 5a-5d As shown, the angle between the pixel point K in the core area and the upper left corner point A is TA, the angle between the pixel point K and the upper right corner point B is TB, the angle between the pixel point K and the lower right corner point C is TC, and the angle between the pixel point K and the lower left corner point D is TD.

[0065] Therefore, the image detection model can be used to perform image detection on the image to be cropped to obtain the core prediction map, distance prediction map and angle prediction map of each sub-image.

[0066] S230 . For each sub-image, determine a core pixel set of each sub-image based on the probability values ​​of the pixels in the core prediction map.

[0067] In the embodiment of the present disclosure, the probability value refers to the probability that a pixel point belongs to the core area.

[0068] In the embodiment of the present disclosure, optionally, S230 may include the following steps:

[0069] Pixels with probability values ​​greater than or equal to a preset probability threshold are selected from the core prediction map to form a core pixel set for each sub-image.

[0070] The preset probability threshold refers to the probability value of a pixel belonging to the core area. Optionally, the preset probability threshold can be a value such as 0.8, which is not limited here.

[0071] S240 . For each core pixel point in the core pixel set of each sub-image, determine at least one distance value corresponding to the core pixel point in the distance prediction map and at least one angle value corresponding to the core pixel point in the angle prediction map.

[0072] It can be understood that for each core pixel point in the core prediction map, its corresponding distance prediction map and angle prediction map are obtained, and at least one distance value corresponding to the distance prediction map and at least one angle value corresponding to the angle prediction map are determined.

[0073] S250 : Determine a predicted page frame of each sub-image based on the coordinates of each core pixel point in each sub-image, at least one distance value corresponding to each core pixel point, and at least one angle value corresponding to each core pixel point.

[0074] In the embodiment of the present disclosure, optionally, S250 specifically includes the following steps:

[0075] Determine the coordinates of each corner point of each sub-image based on the coordinates of each core pixel point in each sub-image, at least one distance value corresponding to each core pixel point, and at least one angle value corresponding to each core pixel point;

[0076] According to the coordinates of each corner point of each sub-image, a predicted page frame of each sub-image is generated.

[0077] The predicted page frame of each sub-image can be understood as the cropping area of ​​each sub-image.

[0078] Specifically, after determining the coordinates of each corner point of each sub-image, the specific position of each corner point of each sub-image can be determined, and all corner points of each sub-image are connected to form a predicted page frame of each sub-image.

[0079] S260 : Cropping the image to be cropped based on the predicted page frame to obtain a target cropped image.

[0080] Specifically, the image to be cropped may be cropped based on the predicted position of each pixel point on the page frame to obtain a target cropped image, which includes the cropped image of each sub-image.

[0081] An image cropping method according to an embodiment of the present disclosure comprises: obtaining an image to be cropped, wherein the image to be cropped includes at least one sub-image; determining a core prediction map, a distance prediction map, and an angle prediction map of each sub-image based on a pre-trained image detection model; determining a core pixel set of each sub-image based on the probability values ​​of the pixels in the core prediction map; determining, for each core pixel in the core pixel set of each sub-image, at least one distance value corresponding to the distance prediction map and at least one angle value corresponding to the angle prediction map; determining a predicted page frame of each sub-image based on the coordinates of each core pixel in each sub-image, at least one distance value corresponding to each core pixel, and at least one angle value corresponding to each core pixel; and cropping the image to be cropped based on the predicted page frame to obtain a target cropped image. Thus, the predicted page frame can be accurately determined by identifying the coordinates of the core pixels of the image to be detected, the distance values ​​corresponding to the core pixels, and the angle values ​​corresponding to the core pixels, and the image cropping can be performed based on the predicted page frame. Accurate image cropping can also be performed on images with high complexity, thereby ensuring the image cropping effect.

[0082] In another embodiment of the present disclosure, the image detection model includes a feature extraction network, a core area prediction network, a distance prediction network, and an angle prediction network, and image detection is performed based on the above networks.

[0083] In the embodiment of the present disclosure, optionally, S220 may specifically include the following steps:

[0084] Using the feature extraction network of the image detection model, feature extraction is performed on the image to be cropped to obtain a feature map of the image to be cropped;

[0085] Using the core region prediction network of the image detection model, the feature map is normalized and a convolution operation is performed on the normalized feature map to obtain the core prediction map of each sub-image.

[0086] Using the distance prediction network of the image detection model, perform at least one convolution on the feature map to obtain a distance prediction map for each sub-image.

[0087] The angle prediction network of the image detection model is used to perform at least one convolution on the feature map to obtain an angle prediction map for each sub-image.

[0088] Among them, the feature extraction network can be the backbone network of the image detection model.

[0089] Among them, the core area prediction network, the distance prediction network and the angle prediction network can be convolutional networks, and each convolutional network can correspond to a different convolution kernel.

[0090] Optionally, the core area prediction network may include a 3*3*80 convolution kernel, so that a convolution process is performed on the feature map of the cropped image based on the 3*3*80 convolution kernel to obtain a core prediction map (Kernelmap) of each image.

[0091] Optionally, the distance prediction network may include four 3*3*80 convolution kernels, so that the feature map of the cropped image is convolved four times based on the four 3*3*80 convolution kernels to obtain a distance prediction map (Dist map) of each image.

[0092] Optionally, the distance prediction network may include four 3*3*80 convolution kernels, so that the feature map of the cropped image is convolved four times based on the four 3*3*80 convolution kernels to obtain an angle prediction map (Theta map) of each image.

[0093] To explain S220 in detail, Figure 6 A structural diagram of an image detection model provided by an embodiment of the present disclosure is shown.

[0094] like Figure 6 As shown in the figure, the image detection model includes a feature extraction network, a core area prediction network, a distance prediction network, and an angle prediction network.

[0095] Step 1: Input the image to be cropped into the feature extraction network of the image detection model to obtain the feature map corresponding to at least one sub-image;

[0096] Step 2: Input the feature map into the core area prediction network, distance prediction network and angle prediction network respectively to obtain the core prediction map corresponding to each sub-image, the distance prediction map of each sub-image and the angle prediction map of each sub-image.

[0097] Optionally, the image size of the image to be cropped can be 320*320*3, the backbone network can be a lightweight network (MobileNet-v2), the image size of the feature map can be 80*80*128, the core area detection network can use a 3*3*80 convolution kernel for convolution, the distance calculation network can use four 3*3*80 convolution kernels for convolution, and the angle calculation network can use four 3*3*80 convolution kernels for convolution.

[0098] In an embodiment of the present disclosure, optionally, a method for training an image detection model includes:

[0099] S1. Obtain a sample image to be cropped and an annotated image corresponding to the sample image to be cropped, wherein the sample image to be cropped includes at least one sub-image, and the annotated image includes a sample core prediction map corresponding to each sub-image, a sample distance prediction map corresponding to each sub-image, and a sample angle prediction map corresponding to each sub-image;

[0100] S2. Use the sample image to be cropped and the labeled image corresponding to the sample image to be cropped to train a preset network to obtain a trained image detection model.

[0101] The sample image to be cropped may be an original image that needs to be cropped in the training data.

[0102] Among them, the label value corresponding to the core area of ​​the sample core prediction map is 1, and the label value corresponding to the remaining areas is 0.

[0103] Specifically, S2 may include the following steps:

[0104] S21, inputting the sample image to be cropped into a preset network to obtain a predicted core map corresponding to at least one sub-image in the sample image to be cropped, a predicted distance map corresponding to each sub-image, and a predicted angle map corresponding to each sub-image;

[0105] S22, calculating a first loss function based on the sample core prediction map and the predicted core map, calculating a second loss function based on the sample distance prediction map and the predicted distance map, and calculating a third loss function based on the sample angle prediction map and the predicted angle map;

[0106] S23, calculating a target loss function of a preset network based on the first loss function, the second loss function, and the third loss function;

[0107] S24. Iteratively adjust the preset network using the target loss function until the target loss function is less than the preset loss function threshold, thereby obtaining a trained image detection model.

[0108] Optionally, the first loss function may be a geometric similarity measurement function, specifically a Dice coefficient.

[0109] Optionally, both the second loss function and the third loss function may be norm loss functions, specifically L1_Loss.

[0110] The preset loss function threshold may be a predetermined loss value used to determine whether to stop adjusting the preset network.

[0111] Therefore, the sample image to be cropped and its corresponding annotated image can be used to train an image detection model, so that image detection can be accurately performed using the image detection model.

[0112] In another embodiment of the present disclosure, if each sub-image corresponds to multiple predicted page frames, the multiple predicted page frames corresponding to each sub-image can be filtered, and then the image to be cropped can be cropped based on the filtered predicted page frames. Optionally, the predicted page frames can be filtered using a non-maximum suppression algorithm (NMS algorithm) or other methods, which are not limited here.

[0113] In the embodiment of the present disclosure, when each sub-image corresponds to multiple predicted page frames, S260 may specifically include the following steps:

[0114] S2601: Obtain a predicted probability value for each predicted page frame, and form a predicted page frame set based on the predicted page frames, wherein the predicted probability value is a probability value of the corresponding core pixel point;

[0115] S2602: Select a target page frame from the set of predicted page frames based on the predicted probability value of each predicted page frame and the overlap between two adjacent predicted page frames in the set of predicted page frames, wherein the overlap is the ratio of the intersection pixel area to the union pixel area of ​​two adjacent predicted page frames;

[0116] S2603: Crop the image to be cropped based on the target page frame to obtain a target cropped image.

[0117] With respect to S2601 , each predicted page frame may be sorted in descending order of predicted probability value to obtain a predicted page frame set.

[0118] S2602 may specifically include the following steps: starting with the first predicted page frame in the predicted page frame set, taking the first predicted page frame as the current page frame, and performing the following operations on the current page frame: calculating the degree of overlap between the current page frame and the next page frame of the current page frame; if the degree of overlap is greater than or equal to a preset overlap threshold, retaining the page frame with the largest probability value between the current page frame and the next page frame, and repeating the above operations until the current page frame is the last predicted page frame, thereby obtaining a target page frame.

[0119] Optionally, S2602 may further include the following steps: if the degree of overlap is less than a preset degree of overlap threshold, retaining the current page frame and the next page frame; continuing to calculate the degree of overlap between the current page frame and the page frame after the next page frame, and repeating the above operation until the current page frame is the last predicted page frame and the degrees of overlap between two adjacent predicted page frames are both less than the preset degree of overlap threshold, thereby obtaining the target page frame.

[0120] Therefore, in the case where each sub-image corresponds to multiple predicted page frames, the multiple predicted page frames corresponding to each sub-image can be filtered based on the overlap between the predicted page frames, and the filtered predicted page frames can be used to crop the image to be cropped, so as to avoid cropping results with a high degree of repetition.

[0121] In yet another embodiment of the present disclosure, an overall logic for image cropping is provided.

[0122] Figure 7 The figure shows an overall logic diagram of an image cropping method provided by an embodiment of the present disclosure.

[0123] like Figure 7 As shown, the image cropping process may include: a model prediction process and a post-processing process.

[0124] Among them, the model prediction process specifically detects the information image of the image set to be cropped, and obtains the core prediction map, distance prediction map and angle prediction map of each sub-image in the image to be cropped.

[0125] Among them, the post-processing process is specifically to determine the predicted page frame of each sub-image based on the core prediction map, distance prediction map and angle prediction map of each sub-image, and then crop the image to be cropped based on the predicted page frame to obtain the target cropped image.

[0126] In order to understand the post-processing process in detail, Figure 8 A flowchart of the post-processing process in the image cropping process provided by an embodiment of the present disclosure is shown.

[0127] like Figure 8 As shown, the post-processing process includes the following steps:

[0128] S810 . For each sub-image, determine a core pixel set of each sub-image based on probability values ​​of pixels in the core prediction map.

[0129] S820: Determine whether the pixel point in the core pixel set is 0. If so, end; otherwise, execute S830.

[0130] S830. For each core pixel point in the core pixel set of each sub-image, determine at least one distance value corresponding to it in the distance prediction map and at least one angle value corresponding to it in the angle prediction map.

[0131] S840 : Obtain a predicted probability value of each predicted page frame, and form a predicted page frame set based on the predicted page frames, wherein the predicted probability value is a probability value of the corresponding core pixel point.

[0132] S850: Select a target page frame from the prediction page frame set based on the prediction probability value of each prediction page frame and the overlap between two adjacent prediction page frames in the prediction page frame set, wherein the overlap is the ratio of the intersection pixel area to the union pixel area of ​​two adjacent prediction page frames.

[0133] S860: Crop the image to be cropped based on the target page frame to obtain a target cropped image.

[0134] The specific implementation of S830 to S860 can be found in the description of the above embodiment and will not be elaborated here.

[0135] The present disclosure also provides an image cropping device for implementing the above-mentioned image cropping method. Figure 9 In the embodiments of the present disclosure, the image cropping device may be an electronic device or a server. The electronic device may include a mobile phone, tablet computer, desktop computer, laptop computer, or other device with communication capabilities. The server may be a cloud server or server cluster, or other device with storage and computing capabilities.

[0136] Figure 9 A structural schematic diagram of an image cropping device provided by an embodiment of the present disclosure is shown.

[0137] like Figure 9 As shown, the image cropping device 900 may include:

[0138] The image to be cropped module 910 is configured to obtain an image to be cropped, wherein the image to be cropped includes at least one sub-image;

[0139] An image detection module 920 is configured to determine a core prediction map, a distance prediction map, and an angle prediction map for each sub-image based on a pre-trained image detection model.

[0140] A first determining module 930 is configured to determine, for each sub-image, a core pixel set of each sub-image based on probability values ​​of pixels in the core prediction map;

[0141] A second determining module 940 is configured to determine, for each core pixel point in the core pixel set of each sub-image, at least one distance value corresponding to the core pixel point in the distance prediction map and at least one angle value corresponding to the core pixel point in the angle prediction map;

[0142] a third determining module 950, configured to determine a predicted page frame of each sub-image based on the coordinates of each core pixel point in each sub-image, at least one distance value corresponding to each core pixel point, and at least one angle value corresponding to each core pixel point;

[0143] The image cropping module 960 is configured to crop the image to be cropped based on the predicted page frame to obtain a target cropped image.

[0144] An image cropping device according to an embodiment of the present disclosure obtains an image to be cropped, wherein the image to be cropped includes at least one sub-image; based on a pre-trained image detection model, determines a core prediction map, a distance prediction map, and an angle prediction map for each sub-image; for each sub-image, determines a core pixel set for each sub-image based on the probability values ​​of the pixels in the core prediction map; for each core pixel in the core pixel set of each sub-image, determines at least one distance value corresponding to the distance prediction map and at least one angle value corresponding to the angle prediction map; determines a predicted page frame for each sub-image based on the coordinates of each core pixel in each sub-image, the at least one distance value corresponding to each core pixel, and the at least one angle value corresponding to each core pixel; and crops the image to be cropped based on the predicted page frame to obtain a target cropped image. Thus, the device can accurately determine the predicted page frame by identifying the coordinates of the core pixels of the image to be detected, the distance values ​​corresponding to the core pixels, and the angle values ​​corresponding to the core pixels, and perform image cropping based on the predicted page frame. This device can accurately crop images even with high complexity, thereby ensuring an effective cropping effect.

[0145] In some optional embodiments, the image detection module 920 includes:

[0146] a feature extraction unit, configured to extract features of the image to be cropped using a feature extraction network of the image detection model to obtain a feature map of the image to be cropped;

[0147] a core prediction map determining unit, configured to normalize the feature map using a core region prediction network of the image detection model, and perform a convolution process on the normalized feature map to obtain a core prediction map for each sub-image;

[0148] a distance prediction map determining unit, configured to perform at least one convolution process on the feature map using the distance prediction network of the image detection model to obtain a distance prediction map for each sub-image;

[0149] An angle prediction map determining unit is used to perform at least one convolution process on the feature map using the angle prediction network of the image detection model to obtain an angle prediction map for each sub-image.

[0150] In some optional embodiments, the first determining module 930 includes:

[0151] a corner point coordinate determining unit, configured to determine the coordinates of each corner point of each sub-image based on the coordinates of each core pixel point in each sub-image, at least one distance value corresponding to each core pixel point, and at least one angle value corresponding to each core pixel point;

[0152] The predicted page frame generating unit is configured to generate a predicted page frame of each sub-image according to the coordinates of each corner point of each sub-image.

[0153] In some optional embodiments, each sub-image corresponds to a plurality of predicted page frames;

[0154] Accordingly, the image cropping module 960 includes:

[0155] a predicted page frame set forming unit, configured to obtain a predicted probability value of each predicted page frame, and form a predicted page frame set based on the predicted page frames, wherein the predicted probability value is a probability value of a corresponding core pixel point;

[0156] a target page frame selection unit, configured to select a target page frame from the set of prediction page frames based on a prediction probability value of each of the prediction page frames and a degree of overlap between two adjacent prediction page frames in the set of prediction page frames, wherein the degree of overlap is a ratio of an intersection pixel area to a union pixel area of ​​the two adjacent prediction page frames;

[0157] The image cropping unit is configured to crop the image to be cropped based on the target page frame to obtain the target cropped image.

[0158] In some optional embodiments, the target page selection unit is specifically configured to start with a first predicted page frame in the predicted page frame set, take the first predicted page frame as a current page frame, and perform the following operations on the current page frame:

[0159] Calculating the overlap between the current page frame and the next page frame of the current page frame;

[0160] If the overlap is greater than or equal to a preset overlap threshold, the page frame with the largest probability value between the current page frame and the next page frame is retained, and the above operation is repeated until the current page frame is the last predicted page frame, thereby obtaining the target page frame.

[0161] In some optional embodiments, the target page selection unit is further configured to retain the current page frame and the next page frame if the overlap is less than a preset overlap threshold;

[0162] Continue calculating the overlap between the current page frame and the page frame after the next page frame, and repeat the above operation until the current page frame is the last predicted page frame and the overlap between two adjacent predicted page frames is less than the preset overlap threshold, thereby obtaining the target page frame.

[0163] In some optional embodiments, the device further comprises:

[0164] a sample acquisition module, configured to acquire a sample image to be cropped and an annotated image corresponding to the sample image to be cropped, wherein the sample image to be cropped includes at least one sub-image, and the annotated image includes a sample core prediction map corresponding to each sub-image, a sample distance prediction map corresponding to each sub-image, and a sample angle prediction map corresponding to each sub-image;

[0165] The model training module is used to train a preset network using the sample image to be cropped and the annotated image corresponding to the sample image to be cropped to obtain a trained image detection model.

[0166] It should be noted that Figure 9 The image cropping apparatus 900 shown can perform Figures 2 to 8 The various steps in the method embodiment shown are implemented Figures 2 to 8 The various processes and effects in the illustrated method embodiment are not described in detail here.

[0167] Figure 10 A structural schematic diagram of an image cropping device provided by an embodiment of the present disclosure is shown.

[0168] like Figure 10As shown, the image cropping device may include a processor 1001 and a memory 1002 storing computer program instructions.

[0169] Specifically, the processor 1001 may include a central processing unit (CPU), or an application-specific integrated circuit (ASIC), or may be configured to implement one or more integrated circuits of the embodiments of the present application.

[0170] The memory 1002 may include a large capacity memory for information or instructions. By way of example and not limitation, the memory 1002 may include a hard disk drive (HDD), a floppy disk drive, a flash memory, an optical disk, a magneto-optical disk, a magnetic tape, or a universal serial bus (USB) drive, or a combination of two or more of these. Where appropriate, the memory 1002 may include removable or non-removable (or fixed) media. Where appropriate, the memory 1002 may be inside or outside the integrated gateway device. In a specific embodiment, the memory 1002 is a non-volatile solid-state memory. In a specific embodiment, the memory 1002 includes a read-only memory (ROM). Where appropriate, the ROM may be a mask-programmed ROM, a programmable ROM (PROM), an erasable PROM (EPROM), an electrically erasable PROM (EEPROM), an electrically alterable ROM (EAROM) or flash memory, or a combination of two or more of these.

[0171] The processor 1001 reads and executes the computer program instructions stored in the memory 1002 to perform the steps of the image cropping method provided in the embodiment of the present disclosure.

[0172] In one example, the image cropping device may further include a transceiver 1003 and a bus 1004. Figure 10 As shown, the processor 1001 , the memory 1002 and the transceiver 1003 are connected via a bus 1004 and communicate with each other.

[0173] The bus 1004 includes hardware, software, or both. By way of example and not limitation, the bus may include an Accelerated Graphics Port (AGP) or other graphics bus, an Extended Industry Standard Architecture (EISA) bus, a Front Side Bus (FSB), a Hyper Transport (HT) interconnect, an Industrial Standard Architecture (ISA) bus, an InfiniBand interconnect, a Low Pin Count (LPC) bus, a memory bus, a MicroChannel Architecture (MCA) bus, a Peripheral Component Interconnect (PCI) bus, a PCI-Express (PCI-X) bus, a Serial Advanced Technology Attachment (SATA) bus, a Video Electronics Standards Association Local Bus (VLB) bus, or other suitable buses, or a combination of two or more of these. Where appropriate, the bus 1004 may include one or more buses. Although embodiments herein describe and illustrate a particular bus, this application contemplates any suitable bus or interconnect.

[0174] The following is an embodiment of a computer-readable storage medium provided in an embodiment of the present disclosure. The computer-readable storage medium and the image cropping method of the above-mentioned embodiments belong to the same inventive concept. For details not fully described in the embodiment of the computer-readable storage medium, please refer to the embodiment of the above-mentioned image cropping method.

[0175] This embodiment provides a storage medium containing computer-executable instructions. When executed by a computer processor, the computer-executable instructions are used to perform an image cropping method. The method includes:

[0176] Acquire an image to be cropped, wherein the image to be cropped includes at least one sub-image;

[0177] Based on the pre-trained image detection model, determine the core prediction map of each sub-image, the distance prediction map of each sub-image, and the angle prediction map of each sub-image;

[0178] For each sub-image, determining a core pixel set of each sub-image based on probability values ​​of pixels in the core prediction image;

[0179] For each core pixel point in the core pixel set of each sub-image, determining at least one distance value corresponding to the core pixel point in the distance prediction map and at least one angle value corresponding to the core pixel point in the angle prediction map;

[0180] determining a predicted page frame of each sub-image based on the coordinates of each core pixel point in each sub-image, at least one distance value corresponding to each core pixel point, and at least one angle value corresponding to each core pixel point;

[0181] The image to be cropped is cropped based on the predicted page frame to obtain a target cropped image.

[0182] Of course, the storage medium containing computer-executable instructions provided by the embodiment of the present disclosure is not limited to the above method operations, and the computer-executable instructions can also execute related operations in the image cropping method provided by any embodiment of the present disclosure.

[0183] Through the above description of the implementation methods, those skilled in the art can clearly understand that the present disclosure can be implemented with the help of software and necessary general-purpose hardware, and of course it can also be implemented by hardware, but in many cases the former is a better implementation method. Based on this understanding, the technical solution of the present disclosure is essentially or the part that contributes to the prior art can be embodied in the form of a software product, and the computer software product can be stored in a computer-readable storage medium, such as a computer floppy disk, read-only memory (ROM), random access memory (RAM), flash memory (FLASH), hard disk or optical disk, etc., including a number of instructions for enabling a computer cloud platform (which can be a personal computer, server, or network cloud platform, etc.) to execute the image cropping method provided by each embodiment of the present disclosure.

[0184] Note that the above are only preferred embodiments of the present disclosure and the technical principles employed. Those skilled in the art will understand that the present disclosure is not limited to the specific embodiments herein, and that various obvious changes, readjustments, and substitutions can be made by those skilled in the art without departing from the scope of protection of the present disclosure. Therefore, although the present disclosure has been described in more detail through the above embodiments, the present disclosure is not limited to the above embodiments, and may include more other equivalent embodiments without departing from the concept of the present disclosure, and the scope of the present disclosure is determined by the scope of the appended claims.

Claims

1. An image cropping method, comprising: Acquire an image to be cropped, wherein the image to be cropped includes at least one sub-image; Based on the pre-trained image detection model, determine the core prediction map of each sub-image, the distance prediction map of each sub-image, and the angle prediction map of each sub-image; For each sub-image, determining a core pixel set of each sub-image based on probability values ​​of pixels in the core prediction image; For each core pixel point in the core pixel set of each sub-image, determining at least one distance value corresponding to the core pixel point in the distance prediction map and at least one angle value corresponding to the core pixel point in the angle prediction map; determining a predicted page frame of each sub-image based on the coordinates of each core pixel point in each sub-image, at least one distance value corresponding to each core pixel point, and at least one angle value corresponding to each core pixel point; The image to be cropped is cropped based on the predicted page frame to obtain a target cropped image.

2. The method according to claim 1, wherein The method of determining a core prediction map of each sub-image, a distance prediction map of each sub-image, and an angle prediction map of each sub-image based on a pre-trained image detection model includes: Using the feature extraction network of the image detection model, extracting features from the image to be cropped to obtain a feature map of the image to be cropped; Normalizing the feature map using the core region prediction network of the image detection model, and performing a convolution process on the normalized feature map to obtain a core prediction map for each sub-image; Using the distance prediction network of the image detection model, performing at least one convolution process on the feature map to obtain a distance prediction map for each sub-image; The angle prediction network of the image detection model is used to perform at least one convolution process on the feature map to obtain an angle prediction map for each sub-image.

3. The method according to claim 1, wherein The determining of the predicted page frame of each sub-image based on the coordinates of each core pixel point in each sub-image, at least one distance value corresponding to each core pixel point, and at least one angle value corresponding to each core pixel point includes: determining the coordinates of each corner point of each sub-image based on the coordinates of each core pixel point in each sub-image, at least one distance value corresponding to each core pixel point, and at least one angle value corresponding to each core pixel point; A predicted page frame of each sub-image is generated according to the coordinates of each corner point of each sub-image.

4. The method according to any one of claims 1 to 3, wherein each sub-image corresponds to a plurality of predicted page frames; Accordingly, the step of cropping the image to be cropped based on the predicted page frame to obtain a target cropped image includes: Obtaining a predicted probability value of each predicted page frame, and forming a predicted page frame set based on the predicted page frames, wherein the predicted probability value is a probability value of a corresponding core pixel point; selecting a target page frame from the set of predicted page frames based on a predicted probability value of each predicted page frame and a degree of overlap between two adjacent predicted page frames in the set of predicted page frames, wherein the degree of overlap is a ratio of an intersection pixel area to a union pixel area of ​​the two adjacent predicted page frames; The image to be cropped is cropped based on the target page frame to obtain the target cropped image.

5. The method according to claim 4, wherein The selecting a target page frame from the set of predicted page frames based on the predicted probability value of each predicted page frame and the overlap between two adjacent predicted page frames in the set of predicted page frames includes: Starting with the first predicted page frame in the predicted page frame set, taking the first predicted page frame as the current page frame, performing the following operations on the current page frame: Calculating the overlap between the current page frame and the next page frame of the current page frame; If the overlap is greater than or equal to a preset overlap threshold, the page frame with the largest probability value between the current page frame and the next page frame is retained, and the above operation is repeated until the current page frame is the last predicted page frame, thereby obtaining the target page frame.

6. The method according to claim 5, wherein: Also includes: If the overlap is less than a preset overlap threshold, retaining the current page frame and the next page frame; Continue calculating the overlap between the current page frame and the page frame after the next page frame, and repeat the above operation until the current page frame is the last predicted page frame and the overlap between two adjacent predicted page frames is less than the preset overlap threshold, thereby obtaining the target page frame.

7. The method according to claim 1, characterized in that Image detection model training methods, including: Obtaining a sample image to be cropped and an annotated image corresponding to the sample image to be cropped, wherein the sample image to be cropped includes at least one sub-image, and the annotated image includes a sample core prediction map corresponding to each sub-image, a sample distance prediction map corresponding to each sub-image, and a sample angle prediction map corresponding to each sub-image; The sample image to be cropped and the annotated image corresponding to the sample image to be cropped are used to train a preset network to obtain a trained image detection model.

8. An image cropping device, characterized in that: include: An image to be cropped module, configured to obtain an image to be cropped, wherein the image to be cropped includes at least one sub-image; An image detection module is used to determine a core prediction map, a distance prediction map, and an angle prediction map for each sub-image based on a pre-trained image detection model. A first determining module is configured to determine, for each sub-image, a core pixel set of each sub-image based on probability values ​​of pixels in the core prediction image; a second determining module, configured to determine, for each core pixel point in the core pixel set of each sub-image, at least one distance value corresponding to the core pixel point in the distance prediction map and at least one angle value corresponding to the core pixel point in the angle prediction map; a third determining module, configured to determine a predicted page frame of each sub-image based on the coordinates of each core pixel point in each sub-image, at least one distance value corresponding to each core pixel point, and at least one angle value corresponding to each core pixel point; The image cropping module is used to crop the image to be cropped based on the predicted page frame to obtain a target cropped image.

9. An image cropping device, characterized in that: include: processor; a memory for storing executable instructions; The processor is configured to read the executable instructions from the memory and execute the executable instructions to implement the image cropping method according to any one of claims 1 to 7.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that: The storage medium stores a computer program, and when the computer program is executed by a processor, the processor implements the image cropping method according to any one of claims 1 to 7.

Images