Image processing method, readable storage medium and computer program product

Abstract

The present invention relates to the field of image processing technology, and discloses an image processing method, a readable storage medium, and a computer program product. The image processing method comprises: obtaining initial aesthetic scores of multiple candidate sub-images at a target size of the image to be cropped; obtaining depth information of the image to be cropped; predicting target aesthetic scores of the multiple candidate sub-images according to the initial aesthetic scores of the multiple candidate sub-images and the depth information of the image to be cropped; and cropping the image to be cropped according to the target aesthetic scores of the multiple candidate sub-images. The present invention realizes image cropping by comprehensively considering the aesthetic factors and depth information of the sub-images, thereby greatly enhancing the image cropping effect.

Description

Technical Field

[0001] The present invention relates to the technical field of image processing, and in particular to an image processing method, a readable storage medium and a computer program product. Background Art

[0002] In programmatic creativity and other intelligent graphics-related fields, it is often necessary to obtain images with different aspect ratios from the original image. When the aspect ratio of the original image is significantly different from the target aspect ratio, scaling the original image will cause the image to stretch, thus affecting the overall aesthetics of the image. Therefore, in most cases, the original image needs to be cropped.

[0003] The existing automatic image cropping method is an image processing method based on object saliency, which mainly determines the main area of ​​the sub-image according to the object saliency. However, in some scenes, there may be no salient objects or salient objects are scattered. In these cases, the object saliency-based method cannot be effectively cropped. Summary of the invention

[0004] The object of the present invention is to provide an image processing method, a readable storage medium and a computer program product, which realize image cropping by comprehensively considering the aesthetic factors and depth information of the sub-images, thereby greatly enhancing the image cropping effect.

[0005] In order to achieve the above-mentioned purpose, the first aspect of the present invention provides an image processing method, which includes: obtaining initial aesthetic scores of multiple candidate sub-images at a target size of the image to be cropped; obtaining depth information of the image to be cropped; predicting target aesthetic scores of the multiple candidate sub-images based on the initial aesthetic scores of the multiple candidate sub-images and the depth information of the image to be cropped; and cropping the image to be cropped based on the target aesthetic scores of the multiple candidate sub-images.

[0006] Preferably, the obtaining of initial aesthetic scores of multiple candidate sub-images at the target size of the image to be cropped includes: obtaining the initial aesthetic scores of multiple candidate sub-images at the target size through an initial score prediction model: extracting features of the image to be cropped to obtain a feature map of the image to be cropped and a mapping relationship between the feature map of the image to be cropped and the image to be cropped; and obtaining the initial aesthetic scores of multiple candidate sub-images at the target size based on the feature map of the image to be cropped and the mapping relationship, wherein the target size includes a target length ratio and a target width ratio of the multiple candidate sub-images to the image to be cropped.

[0007] Preferably, obtaining the initial aesthetic scores of the multiple candidate sub-images at the target size includes: pre-cropping the feature map of the image to be cropped according to a first preset rule to obtain a plurality of sub-feature maps, wherein the plurality of sub-feature maps at least cover the feature map of the image to be cropped, and the first preset rule includes: the length ratio and the width ratio of the multiple sub-feature maps to the feature map of the image to be cropped are the target length ratio and the target width ratio respectively and the positions of the multiple sub-feature maps are multiple first preset positions; obtaining the initial aesthetic scores of the multiple sub-feature maps; and obtaining the initial aesthetic scores of the multiple candidate sub-images corresponding to the multiple sub-feature maps according to the mapping relationship and the initial aesthetic scores of the multiple sub-feature maps.

[0008] Preferably, the image processing method also includes: a step of training the initial score prediction model through an image training sample set, wherein each of the image training sample set includes: an initial aesthetic score of a first image and multiple first sample sub-images of the first image pre-cropped according to a second preset rule, wherein the multiple first sub-images at least cover the first image, and the second preset rule includes the positions of the multiple first sub-images being multiple second preset positions.

[0009] Preferably, the training of the initial score prediction model through the image training sample set includes: training the initial score prediction model through an image training sample group randomly selected from the image training sample set to obtain initial aesthetic scores of multiple first sub-images of each of the selected image training sample group, wherein the image training sample group includes a preset number of image training samples, and the multiple first sub-images are obtained by pre-cutting each of the image training sample group according to a rule that corresponds one-to-one to the position, length and width of the multiple first sub-images; optimizing parameters in the initial score prediction model according to the initial aesthetic scores of the multiple first sub-images and the initial aesthetic scores of the multiple first sub-images; and iteratively training and optimizing the optimized initial score prediction model through an image training sample group randomly selected from the image training sample set until the training process meets a preset stop condition.

[0010] Preferably, training the initial score prediction model further includes: verifying the optimized initial score prediction model corresponding to multiple preset rounds of training through an image verification sample set to obtain multiple second sub-images and initial aesthetic scores of the multiple second sub-images for each of the image verification sample sets, wherein each of the image verification sample sets includes: a second image and initial aesthetic scores of multiple second sub-images of the second image pre-cut according to a third preset rule, the multiple second sub-images at least covering the second image, the third preset rule including that the positions of the multiple second sample sub-images are multiple third preset positions, and the multiple second sub-images are obtained by pre-cutting each of the image verification sample group according to a rule corresponding to the positions, lengths and widths of the multiple second sub-images; and determining the accuracy of the optimized initial score prediction model corresponding to the multiple preset rounds of training according to the initial aesthetic scores of the multiple second sub-images and the initial aesthetic scores of the multiple second sub-images. Accordingly, the image processing method further includes: selecting the initial score prediction model with the highest accuracy from the multiple optimized initial score prediction models obtained in the iterative training and optimization process as the initial score prediction model that has completed training.

[0011] Preferably, before executing the step of training the initial score prediction model through the image training sample set, the image processing method also includes: obtaining the multiple first sub-images and the multiple second sub-images using a grid anchor point-based method; determining the initial aesthetic score of each first sub-image according to the information richness and composition information of each first sub-image in the multiple first sub-images; and determining the initial aesthetic score of each second sub-image according to the information richness and composition information of each second sub-image in the multiple second sub-images.

[0012] Preferably, the training process meets a preset stop condition including: the number of training times meets a preset number of times; or the difference between the target aesthetic scores of the multiple first sub-images and the initial aesthetic scores of the multiple first sub-images is less than a preset difference.

[0013] Preferably, predicting the target aesthetic scores of multiple candidate sub-images includes: performing the following steps through an initial score prediction model: determining the depth information of the multiple candidate sub-images based on the depth information of the image to be cropped, the position information of the multiple sub-feature images and the mapping relationship; determining the depth factors of the multiple candidate sub-images based on the depth information of the multiple candidate sub-images; and determining the target aesthetic scores of the multiple candidate sub-images based on the depth factors of the multiple candidate sub-images and the initial aesthetic scores of the multiple candidate sub-images.

[0014] Preferably, determining the depth factors of the plurality of candidate sub-images comprises: for a candidate sub-image i among the plurality of candidate sub-images, determining the depth factors of the candidate sub-image i according to the maximum depth value of the candidate sub-image i With minimum depth value And the following formula, determine the depth factor α of the candidate sub-image i i ,

[0015] Preferably, determining the target aesthetic scores of the plurality of candidate sub-images comprises: for a candidate sub-image i among the plurality of candidate sub-images, determining the target aesthetic scores of the candidate sub-image i according to the depth factor α i , the initial aesthetic score of the candidate sub-graph i The target aesthetic score of the candidate sub-image i is determined by the following formula:

[0016] Preferably, cropping the image to be cropped comprises: selecting a candidate sub-image with a highest target aesthetic score from the multiple candidate sub-images; and cropping the candidate sub-image with the highest target aesthetic score from the image to be cropped.

[0017] Through the above technical solution, the present invention creatively first obtains the initial aesthetic scores of multiple candidate sub-images at the target size of the image to be cropped and the depth information of the image to be cropped; then predicts the target aesthetic scores of the multiple candidate sub-images based on the initial aesthetic scores of the multiple candidate sub-images and the depth information of the image to be cropped; finally, the image to be cropped is cropped based on the target aesthetic scores of the multiple candidate sub-images. Therefore, the present invention can realize the cropping of the image by comprehensively considering the aesthetic factors and depth information of the sub-images, thereby greatly enhancing the image cropping effect.

[0018] The second aspect of the present invention also provides an image processing system, which includes: an initial score acquisition device, used to acquire the initial aesthetic scores of multiple candidate sub-images at the target size of the image to be cropped; a depth information acquisition device, used to acquire the depth information of the image to be cropped; a target score acquisition device, used to predict the target aesthetic scores of the multiple candidate sub-images based on the initial aesthetic scores of the multiple candidate sub-images and the depth information of the image to be cropped; an image cropping device, used to crop the image to be cropped based on the target aesthetic scores of the multiple candidate sub-images.

[0019] Preferably, the initial score acquisition device is used to obtain the initial aesthetic scores of multiple candidate sub-images at the target size of the image to be cropped, including: obtaining the initial aesthetic scores of multiple candidate sub-images at the target size of the image to be cropped through an initial score prediction model; extracting the features of the image to be cropped to obtain a feature map of the image to be cropped and a mapping relationship between the feature map of the image to be cropped and the image to be cropped; and obtaining the initial aesthetic scores of the multiple candidate sub-images at the target size based on the feature map of the image to be cropped and the mapping relationship, wherein the target size includes a target length ratio and a target width ratio of the multiple candidate sub-images to the image to be cropped.

[0020] Preferably, the initial score acquisition device is used to obtain the initial aesthetic scores of multiple candidate sub-images at the target size through the initial score prediction model, including: pre-cutting the feature map of the image to be cut according to a first preset rule to obtain multiple sub-feature maps, wherein the multiple sub-feature maps at least cover the feature map of the image to be cut, and the first preset rule includes: the length ratio and width ratio of the multiple sub-feature maps to the feature map of the image to be cut are the target length ratio and the target width ratio respectively, and the positions of the multiple sub-feature maps are multiple first preset positions; obtaining the initial aesthetic scores of the multiple sub-feature maps; and obtaining the initial aesthetic scores of the multiple candidate sub-images corresponding to the multiple sub-feature maps according to the mapping relationship and the initial aesthetic scores of the multiple sub-feature maps.

[0021] Preferably, the target score acquisition device for predicting the target aesthetic scores of the multiple candidate sub-images includes: a depth information determination module, used to determine the depth information of the multiple candidate sub-images according to the depth information of the image to be cropped, the position information of the multiple sub-feature maps and the mapping relationship; a depth factor determination module, used to determine the depth factors of the multiple candidate sub-images according to the depth information of the multiple candidate sub-images; and a target score determination module, used to determine the target aesthetic scores of the multiple candidate sub-images according to the depth factors of the multiple candidate sub-images and the initial aesthetic scores of the multiple candidate sub-images.

[0022] Preferably, the depth factor determination module is used to determine the depth factors of the plurality of candidate sub-images, comprising: for a candidate sub-image i among the plurality of candidate sub-images, according to the maximum depth value of the candidate sub-image i With minimum depth value And the following formula, determine the depth factor α of the candidate sub-image i i ,

[0023] Preferably, the target score determination module is used to determine the target aesthetic scores of the plurality of candidate sub-images, including: for a candidate sub-image i among the plurality of candidate sub-images, according to the depth factor α of the candidate sub-image i i , the initial aesthetic score of the candidate sub-graph i The target aesthetic score of the candidate sub-image i is determined by the following formula:

[0024] Preferably, the image cropping device comprises: a screening module for screening out a candidate sub-image with the highest target aesthetic score from the multiple candidate sub-images; and a cropping module for cropping out the candidate sub-image with the highest target aesthetic score from the image to be cropped.

[0025] The specific details and benefits of the image processing system provided by the present invention can be found in the above description of the image processing method, which will not be repeated here.

[0026] A third aspect of the present invention provides a computer-readable storage medium, on which a computer program is stored, and when the computer program is executed by a processor, the steps of the above-mentioned image processing method are implemented.

[0027] A fourth aspect of the present invention provides an electronic device, comprising: a processor; a memory for storing the computer program; the processor is used to read the computer program from the memory and execute the computer program to implement the steps of the above-mentioned image processing method.

[0028] A fifth aspect of the present invention provides a computer program product, which includes a computer program, and when the computer program is executed by a processor, the steps of the above-mentioned image processing method are implemented.

[0029] Other features and advantages of the present invention will be described in detail in the following detailed description. BRIEF DESCRIPTION OF THE DRAWINGS

[0030] The accompanying drawings are used to provide a further understanding of the embodiments of the present invention and constitute a part of the specification. Together with the following specific implementations, they are used to explain the embodiments of the present invention, but do not constitute a limitation on the embodiments of the present invention. In the accompanying drawings:

[0031] Figure 1 is a flow chart of an image processing method provided by an embodiment of the present invention;

[0032] Figure 2 is a flowchart of obtaining initial aesthetic scores of multiple candidate sub-images at a target size of the image to be cropped provided by an embodiment of the present invention;

[0033] Figure 3 is a schematic diagram of the principle of a target aesthetic score prediction framework provided by an embodiment of the present invention;

[0034] Figure 4 is a flowchart of obtaining initial aesthetic scores of multiple candidate sub-images at the target size provided by an embodiment of the present invention;

[0035] Figure 5 is a schematic diagram of multiple sub-feature graphs of a feature graph of an image to be cropped provided by an embodiment of the present invention;

[0036] Figure 6 is a flowchart of predicting target aesthetic scores of multiple candidate sub-images provided by an embodiment of the present invention;

[0037] Figure 7 is a schematic diagram of multiple sample sub-images of a first image provided by an embodiment of the present invention; and

[0038] Figure 8 It is a structural diagram of an image processing system provided by one embodiment of the present invention. DETAILED DESCRIPTION

[0039] The specific implementation of the present invention is described in detail below in conjunction with the accompanying drawings. It should be understood that the specific implementation described here is only used to illustrate and explain the present invention, and is not used to limit the present invention.

[0040] Figure 1 FIG. 1 is a flow chart of an image processing method provided by an embodiment of the present invention. Figure 1 As shown, the image processing method may include the following steps S101-S104.

[0041] Step S101, obtaining initial aesthetic scores of a plurality of candidate sub-images at a target size of the image to be cropped.

[0042] For step S101, obtaining the initial aesthetic scores of the plurality of candidate sub-images at the target size of the image to be cropped may include performing the following steps S201-S202 through an initial score prediction model, such as Figure 2 shown.

[0043] Step S201 : extracting features of the image to be cropped to obtain a feature map of the image to be cropped and a mapping relationship between the feature map of the image to be cropped and the image to be cropped.

[0044] Step S201 is performed by the feature extraction model 100 in the initial score prediction model 1. In order to ensure that sufficient spatial resolution can be provided to the subsequent model (i.e., the score prediction model 110) in the initial score prediction model 1, the feature extraction model 100 may use a convolutional neural network with a suitable layer depth (for example, the feature extraction model 100 uses Figure 3 The feature extraction model 100 can be used to extract a feature map of the image to be cut and a mapping relationship between the feature map and the image to be cut. The mapping relationship may include: a position mapping relationship between each point on the feature map and a corresponding point on the image to be cut; and a size mapping relationship between the feature map and the image to be cut.

[0045] Specifically, the mapping relationship includes a position mapping relationship and a size mapping relationship between the feature map of the image to be cut and the image to be cut. For the position mapping relationship, the position coordinates (xi, yi) of the sub-feature map i and the position coordinates (xi', yi') of the corresponding candidate sub-map i satisfy the following relationship: (kxi, kyi) = (xi', yi') (where k is the ratio of the length (or width) of the image to be cut to the length (or width) of its feature map, which can be pre-set before extracting the feature map, and the center of the image to be cut is the origin of the original image coordinate system). For the size mapping relationship, the length ratio Li / L (or width Wi / W) of the sub-feature map i and the length ratio Li' / L' of the corresponding candidate sub-map i, where L'=kL (or width Wi' / W', where W'=kW) satisfy the following relationship: kLi=Li' (or kWi=Wi'), where Li and Wi are the length and width of the sub-feature map i, L and W are the length and width of the feature map of the image to be cut, Li' and Wi' are the length and width of the candidate sub-map i, and L' and W' are the length and width of the image to be cut.

[0046] Step S202: acquiring initial aesthetic scores of a plurality of candidate sub-images at the target size according to the feature map of the image to be cropped and the mapping relationship.

[0047] The target size includes a target length ratio and a target width ratio between the plurality of candidate sub-images and the image to be cropped. The step S202 is performed by the score prediction model 110 in the initial score prediction model 1.

[0048] For step S202, obtaining the initial aesthetic scores of the plurality of candidate sub-images at the target size may include the following steps S401-S403: Figure 4 shown.

[0049] Step S401, pre-cropping the feature map of the image to be cropped according to a first preset rule to obtain a plurality of sub-feature maps.

[0050] Among them, the multiple sub-feature maps at least cover the feature map of the image to be cut, and the first preset rule may include: the length ratio and width ratio of the multiple sub-feature maps and the feature map of the image to be cut are the target length ratio and the target width ratio respectively, and the positions of the multiple sub-feature maps are multiple first preset positions.

[0051] Specifically, Figure 5 As shown, after the feature map of the image to be cropped is obtained by the feature extraction model 100, the following operations are performed by the score prediction model 110: the center of the feature map is taken as the origin of the feature map coordinate system, and the sub-feature map is pre-cropped at position 1 (x1, y1) on the feature map Figure 1 , Pre-cut sub-feature at position 2 (x2, y2) Figure 2 , Pre-cut sub-feature at position 3 (x3, y3) Figure 3 , Pre-cut sub-feature at position 4 (x4, y4) Figure 4 , Pre-cut sub-feature at position 5 (x5, y5) Figure 5 , and pre-cut sub-features at position 6 (x6, y6) Figure 6 , and the sizes of the sub-feature maps are the same (for example, the target length ratio is 0.5; the target width ratio is 0.5, which can be set according to actual needs). The coordinates of each position can also be preset in advance. That is, pre-cropping is performed according to the first preset rule, as long as the area around the corresponding position on the image to be cropped meets the target size, the area is pre-cropped, regardless of whether the area has a significant feature.

[0052] In addition, in various embodiments of the present invention, “pre-cropping” of the feature map of the image to be cropped according to the first preset rule refers to determining multiple sub-feature maps on the feature map of the image to be cropped that satisfy the first preset rule (i.e., determining the mapping relationship between the feature map and the multiple sub-feature maps), and no cropping action is actually performed at this time.

[0053] Step S402: Obtain initial aesthetic scores of the plurality of sub-feature graphs.

[0054] The initial aesthetic scores of the multiple sub-feature graphs can be obtained through the initial score prediction model. Specifically, according to the information richness and composition information of the multiple sub-feature graphs, the initial aesthetic score of each sub-feature graph can be determined (the specific process can be seen in the following description of the initial aesthetic score). In other words, the level of the aesthetic score reflects the information richness and composition information of the graph. However, the initial aesthetic scores of the multiple sub-feature graphs do not contain depth information about the composition.

[0055] Step S403: acquiring initial aesthetic scores of the plurality of candidate subgraphs corresponding to the plurality of sub-feature graphs according to the mapping relationship and the initial aesthetic scores of the plurality of sub-feature graphs.

[0056] The initial aesthetic score is the aesthetic score that only considers the candidate subgraph (i.e., the aesthetic score of the sub-feature graph). The initial score prediction model can obtain the initial aesthetic scores of the multiple candidate subgraphs corresponding to the multiple sub-feature graphs according to the mapping relationship and the initial aesthetic scores of the multiple sub-feature graphs.

[0057] Specifically, the initial aesthetic scores of the plurality of sub-feature graphs can be used as the initial aesthetic scores of the corresponding plurality of candidate sub-graphs through the mapping relationship. Figure 5 Sub-features shown Figure 1 , 2 , 3, 4, 5, and 6, respectively, when the initial aesthetic scores are 1, 2.5, 3.5, 4, 5, and 4.5, the sub-features determined according to the mapping relationship Figure 1 , 2 , 3, 4, 5, 6 corresponding candidates Figure 1 , 2 The initial aesthetic scores of , 3, 4, 5, and 6 are 1, 2.5, 3.5, 4, 5, and 4.5, respectively.

[0058] Thus, the initial aesthetic scores of the plurality of candidate sub-images at the target size can be obtained through the above steps S401-S403 (i.e., the process of implementing step S202). Furthermore, the initial aesthetic scores of the plurality of candidate sub-images at the target size of the image to be cropped can be obtained through the above steps S201-S202 (i.e., the process of implementing step S101).

[0059] Step S102: acquiring depth information of the image to be cropped.

[0060] For step S102, the obtaining of the depth information of the image to be cropped may include: obtaining the depth information of the image to be cropped through a depth prediction model 120. The depth prediction model 120 may be a MiDaS network (which may be pre-trained by mixing multiple depth data sets) for obtaining the depth information of the image to be cropped (e.g., the depth of the composition).

[0061] Step S103 : predicting target aesthetic scores of the plurality of candidate sub-images according to the initial aesthetic scores of the plurality of candidate sub-images and the depth information of the image to be cropped.

[0062] The target aesthetic score is a comprehensive consideration of the aesthetics and depth of the candidate sub-images. For step S103, predicting the target aesthetic scores of the plurality of candidate sub-images may include executing the following steps S601-S603 by the target score acquisition device 30, such as Figure 6 shown.

[0063] Step S601, determining the depth information of the multiple candidate sub-images according to the depth information of the image to be cropped, the position information of the multiple sub-feature images and the mapping relationship.

[0064] Specifically, the depth information of the image to be cropped may be a mapping relationship between the position information and the depth information of each pixel on the image to be cropped. Accordingly, the depth information of the multiple candidate sub-images may be determined based on the mapping relationship between the position information and the depth information of each pixel and the position information of the multiple sub-feature images. For example, the position coordinates corresponding to each sub-feature image (i.e., the offset coordinates of the center coordinates of the sub-feature image relative to the entire feature image) are searched from the mapping relationship, and the depth information corresponding to each pixel within the corresponding area (the size of the corresponding area is the size of each sub-feature image) surrounding the searched position coordinates is determined.

[0065] Step S602: Determine depth factors of the multiple candidate sub-images according to the depth information of the multiple candidate sub-images.

[0066] For step S602, determining the depth factors of the plurality of candidate sub-images may include: for a candidate sub-image i among the plurality of candidate sub-images, determining the depth factors of the plurality of candidate sub-images according to the maximum depth value of the candidate sub-image i With minimum depth value And the following formula (1) determines the depth factor α of the candidate sub-image i i ,

[0067]

[0068] Step S603 : determining target aesthetic scores of the plurality of candidate sub-images according to the depth factors of the plurality of candidate sub-images and the initial aesthetic scores of the plurality of candidate sub-images.

[0069] In step S603, for a candidate sub-image i among the plurality of candidate sub-images, according to the depth factor α of the candidate sub-image i, i , the initial aesthetic score of the candidate sub-graph i The target aesthetic score of the candidate sub-image i can be determined by the following formula (2):

[0070]

[0071] After determining the depth information of each candidate sub-image, the maximum depth value and the minimum depth value of each candidate sub-image can be determined according to the depth information, and then the depth factor of each candidate sub-image can be determined by combining formula (1); then, the target aesthetic factor of each candidate sub-image is determined according to the depth factor and the initial aesthetic factor of each candidate sub-image combined with formula (2). Thus, the target aesthetic factor of each candidate sub-image takes into account the dual factors of the aesthetics and depth of the candidate sub-image, so the target sub-image selected by the target aesthetic factor can significantly enhance the cropping effect.

[0072] After determining the target aesthetic score of each candidate sub-image (which takes into account the dual factors of aesthetics and depth of composition) through steps S601-S603 (i.e., step S103), the sub-image with the maximum target aesthetic score can be selected from multiple candidate sub-images through the subsequent step S104 as the target sub-image. Therefore, the target sub-image that can be cut out through the following embodiment can take into account the dual factors of aesthetics and depth of composition, thereby significantly enhancing the cutting effect.

[0073] Step S104: cropping the image to be cropped according to the target aesthetic scores of the multiple candidate sub-images.

[0074] For step S104, cropping the image to be cropped includes: selecting a candidate sub-image with a highest target aesthetic score from the multiple candidate sub-images; and cropping the candidate sub-image with the highest target aesthetic score from the image to be cropped.

[0075] Specifically, in (for example, by Figure 8After the target score acquisition device 30 shown in the figure obtains the target aesthetic score of each candidate sub-image, the target aesthetic score of each candidate sub-image can be sorted in descending order, and then the candidate sub-image at the first position (i.e., with the highest target aesthetic score) can be screened out. Then, according to the position information of the candidate sub-image with the highest target aesthetic score and the size information of the candidate sub-image, the corresponding candidate sub-image can be cut out from the image to be cut out, and the cut candidate sub-image can be output as the target sub-image. For example, if the target aesthetic score of the screened candidate sub-image s is the highest, the candidate sub-image s is cut out from the image to be cut out according to the position and size of the candidate sub-image s, so as to be output as the target sub-image.

[0076] The initial score prediction model 1 involved in the above step S101 (which includes Figure 3 The feature extraction model 100 and the score prediction model 110 shown in the figure can be trained models or untrained models. Of course, using the trained initial score prediction model 1 to perform the above process can obtain very accurate prediction results of the initial scores of each sub-graph.

[0077] In one embodiment, the initial score prediction model 1 is a trained initial score prediction model 1. The image processing method further comprises: training the initial score prediction model 1 through an image training sample set.

[0078] Each of the image training sample sets includes: a first image and initial aesthetic scores of multiple first sample sub-images of the first image cropped according to a second preset rule, wherein the multiple first sample sub-images at least cover the first image, and the second preset rule includes that the positions of the multiple first sample sub-images are multiple second preset positions.

[0079] In various embodiments, the plurality of second preset positions in the second preset rule may be the same as or different from the plurality of first preset positions in the first preset rule.

[0080] Specifically, the image training sample set is briefly described first. Each image training sample in the image training sample set can be any picture (for example, a picture taken by a camera or a mobile phone, which can specifically include pictures of various categories such as landscapes, portraits, animals, and buildings) and a sub-picture of the picture (the sub-picture is pre-cropped according to the second preset rule).

[0081] The specific situation of the sub-picture of the picture is introduced as follows: Figure 7 As shown, the center of the image is taken as the origin of the coordinate system, and the sub-image is cut at position 1 (x1, y1) on the image. Figure 1 (The length ratio is R L1 , width ratio is RW1 ), cut the child at position 2 (x2, y2) Figure 2 (The length ratio is R L2 , width ratio is R W2 ), cut the child at position 3 (x3, y3) Figure 3 (The length ratio is R L3 , width ratio is R W3 ), cut the child at position 4 (x4, y4) Figure 4 (The length ratio is R L4 , width ratio is R W4 ), cut the child at position 5 (x5, y5) Figure 5 (The length ratio is R L5 , width ratio is R W5 ), and cut the child at position 6 (x6, y6) Figure 6 (The length ratio is R L6 , width ratio is R W6 ). The coordinates of the above-mentioned positions 1 to 6 may be the same or different from the coordinates of the positions 1 to 6 involved in the first preset rule. The length ratio is the ratio of the length of the sample sub-image to the length of the image, and the width ratio is the ratio of the width of the sample sub-image to the width of the image (these two ratios can be preset in advance, and the length ratio or width ratio of each sample sub-image can be the same or different). That is to say, according to the second preset rule, the cropping (the cropping operation can be performed by a method based on a grid anchor), as long as the area around the corresponding position on the image meets the corresponding size, the area is cropped, regardless of whether the area has significant features. Thus, multiple sample sub-images of different preset sizes can be obtained for each image sample. For example, if there are a total of 1,500 pictures taken by a camera or a mobile phone (including pictures of multiple categories such as landscapes, portraits, animals, and buildings), according to the second preset rule, an average of about 86 sample sub-images (standard sub-images) can be obtained for each photo, and a total of 129,000 sample sub-images can be obtained. Then, according to the information richness and composition information of each of the multiple sample sub-images, an initial aesthetic score of each sample sub-image is determined (the specific process of determining the initial aesthetic score can be found in the description below).

[0082] Of course, after obtaining the image training sample set through the Grid anchor method, 11 annotators with photography-related backgrounds can be asked to score the 129,000 sample sub-images respectively (to obtain the initial aesthetic score of each sample sub-image). Specifically, the score can be scored according to the information richness and composition information of each sample sub-image (for example, the higher the information richness, the higher the score, and the more the composition information meets the aesthetic requirements, the higher the score; and the score obtained according to the information richness and the score obtained according to the composition information can also be weighted and fused. For example, the final score can range from 1 to 5 points), and finally the average of the 11 scores annotated by the 11 annotators for each sample sub-image is used as the score of each sample sub-image (i.e., the initial aesthetic score).

[0083] Specifically, the training of the initial score prediction model through the image training sample set may include: training the initial score prediction model through an image training sample group randomly selected from the image training sample set to obtain initial aesthetic scores of multiple first sub-images of each of the selected image training sample group, wherein the image training sample group includes a preset number of image training samples, and the multiple first sub-images are obtained by cropping each of the image training sample group according to a rule that corresponds one-to-one to the position, length and width of the multiple first sample sub-images; optimizing parameters in the initial score prediction model according to the initial aesthetic scores of the multiple first sub-images and the initial aesthetic scores of the multiple first sample sub-images; and iteratively training and optimizing the optimized initial score prediction model through an image training sample group randomly selected from the image training sample set until the training process meets the preset stop condition.

[0084] For ease of description, in various embodiments of the present invention, the subgraph of the image training sample is referred to as the first subgraph, and the subgraph of the image verification sample is referred to as the second subgraph. In addition, the random selection may include random selection with replacement or random selection without replacement.

[0085] The training process meeting a preset stop condition may include: the number of training times meeting a preset number of times; or the difference between the initial aesthetic scores of the multiple first sub-images and the initial aesthetic scores of the multiple first sample sub-images is less than a preset difference.

[0086] In the case of obtaining an image training sample set, the initial score prediction model can be trained by the image training sample set. Specifically, first, a preset number of image samples (for example, image training sample group 1) are randomly selected from the image training sample set, and the image training sample group 1 is input into an untrained initial score prediction model (i.e., the initial initial score prediction model), so that the initial aesthetic scores of multiple sub-graphs of each image sample in the image training sample group 1 (for example, the multiple sub-graphs of image a correspond to the positions, lengths, and widths of the multiple sample sub-graphs of image a one-to-one; the multiple sub-graphs of image b correspond to the positions, lengths, and widths of the multiple sample sub-graphs of image b one-to-one. This can be achieved by setting the preset rules of the score prediction model in the initial score prediction model 1 to the second preset rules) are obtained (this process is called the training process). Then, according to the initial aesthetic scores of the multiple sub-graphs of each image sample in the image training sample group 1 and the initial aesthetic scores of the multiple sample sub-graphs of the corresponding image sample, the parameters in the initial initial score prediction model are optimized (this process is called the optimization process, and the optimized initial score prediction model can be called the initial score prediction model after the first optimization). At this point, it indicates that the first training process (including training and optimization process) is finished.

[0087] Next, a preset number of image samples (for example, image training sample group 2) are randomly selected from the image training sample set, and the image training sample group 2 is input into the initial score prediction model optimized for the first time, so that the initial aesthetic scores of multiple sub-graphs of each image sample in the image training sample group 2 (for example, the multiple sub-graphs of image c correspond to the positions, lengths and widths of the multiple sample sub-graphs of image c one by one; the multiple sub-graphs of image d correspond to the positions, lengths and widths of the multiple sample sub-graphs of image d one by one. This can be achieved by setting the preset rules of the score prediction model in the initial score prediction model 1 to the second preset rules) are obtained (this process is called the training process). Then, according to the initial aesthetic scores of the multiple sub-graphs of each image sample in the image training sample group 2 and the initial aesthetic scores of the multiple sample sub-graphs of the corresponding image sample, the parameters in the initial score prediction model optimized for the first time are optimized (this process is called the optimization process, and the optimized initial score prediction model can be called the initial score prediction model optimized for the second time). At this point, it indicates that the second training process (including the training and optimization processes) is over.

[0088] Furthermore, a preset number of image samples (e.g., image training sample group 3) are randomly selected from the image training sample set, and the above similar process is repeated (except that image training sample group 2 in the above process needs to be replaced by image training sample group 3, and other corresponding contents are adaptively modified), that is, the optimized initial score prediction model corresponding to the second round of training is trained; and so on, the optimized initial score prediction model corresponding to the previous round of training is continuously trained (i.e., iterative training) until the number of trainings meets the preset number of times (or the difference between the initial aesthetic scores of the multiple sub-graphs of each image sample output by the initial score prediction model and the initial aesthetic scores of the corresponding multiple sample sub-graphs are less than the preset difference). In various embodiments, the stochastic gradient descent method (SGD) can be used to train (i.e., optimize) the model.

[0089] In the above embodiment, the initial score prediction model is trained only with the image training sample set. Although the above process can obtain a sub-image result with a smaller output error, since it is only compared with the sample sub-image of the image training sample set itself, once the image to be cropped is cropped by the trained initial score prediction model, the corresponding output result may not meet the corresponding expectation. Therefore, in this embodiment, each initial score prediction model in the training process can be verified by an image verification sample set different from the image training sample set to screen out an initial score prediction model with higher accuracy and reliability.

[0090] After each training and optimization of the optimized initial score prediction model by using the image training sample group randomly selected from the image training sample set, the training of the initial score prediction model by using the image training sample set may further include: verifying the optimized initial score prediction model corresponding to multiple preset rounds of training by using the image verification sample set to obtain multiple second sub-images and initial aesthetic scores of the multiple second sub-images of each of the image verification sample sets, wherein each of the image verification sample sets includes: a second image and initial aesthetic scores of multiple second sample sub-images of the second image cut according to a third preset rule. Moreover, the multiple second sample sub-images at least cover the second image, and the third preset rule may include that the positions of the multiple second sample sub-images are multiple third preset positions, and the multiple second sub-images are obtained by pre-cutting each of the image verification sample groups according to a rule corresponding to the positions, lengths and widths of the multiple second sample sub-images one by one; and determining the accuracy of the optimized initial score prediction model corresponding to the multiple preset rounds of training according to the initial aesthetic scores of the multiple second sub-images and the initial aesthetic scores of the multiple second sample sub-images.

[0091] In addition, in various embodiments of the present invention, "pre-cropping" of an image according to the second (or third) preset rule refers to determining a plurality of sample sub-images on the image that satisfy the second (or third) preset rule (i.e., determining the mapping relationship between the image and the plurality of sample sub-images), and no cropping action is actually performed at this time. Note that in various embodiments of the present invention, the sub-image / sub-feature image obtained by the initial score prediction model is still a component of the entire image / the entire feature image (it is not actually cropped out from the entire image / the entire feature image), and the cropping action is only performed when the image to be cropped is "cropped" according to the target aesthetic score. The following is a brief description of the image verification sample set (which is different from the image training sample set, and the ratio of the number of samples in the training sample set to the verification sample set can be 9:1).

[0092] Take the center of the image as the origin of the coordinate system and cut the sub-image at position 1 (x1', y1') on the image. Figure 1 (The length ratio is R' L1 , width ratio is R' W1 ), cut the child at position 2 (x2', y2') Figure 2 (The length ratio is R' L2 , width ratio is R' W2 ), cut the child at position 3 (x3', y3') Figure 3 (The length ratio is R' L3 , width ratio is R' W3 ), cut the child at position 4 (x4', y4') Figure 4 (The length ratio is R' L4 , width ratio is R' W4 ), cut the child at position 5 (x5', y5') Figure 5 (The length ratio is R' L5 , width ratio is R' W5 ), and cut the child at position 6 (x6', y6') Figure 6 (The length ratio is R' L6 , width ratio is R' W6 ). Among them, the length ratio is the ratio of the sub-image length to the image length, and the width ratio is the ratio of the sub-image width to the image width (these two ratios can be preset in advance, and the length ratio or width ratio of each sub-image can be the same or different). Thus, multiple sample sub-images of different preset sizes can be obtained for each image sample (the number of the multiple sample sub-images can be 1:9 to the number of the multiple sample sub-images in the image training sample set, and of course the ratio can also be other reasonable values). Then, according to the information richness and composition information of each of the multiple sample sub-images, the initial aesthetic score of each sample sub-image is determined (the specific process of determining the initial aesthetic score can be referred to the description below).

[0093] Of course, the third preset rule may be the same as or different from the second preset rule. Specifically, the multiple third preset positions in the third preset rule may be the same as or different from the multiple second preset positions in the second preset rule. Moreover, similar to the case of the above-mentioned image training sample set, the initial aesthetic scores of multiple sample sub-images of the image verification sample set can be obtained by manual annotation. The multiple preset rounds may be the Nth round, the 2Nth round, the 3Nth round... the k*Nth round, where N may be a first preset integer (for example, 16, and of course N may be reasonably set according to actual conditions), and k is a second preset integer (which may be set according to the number of samples in the image training sample set and the N value).

[0094] Specifically, after optimizing the parameters in the initial score prediction model optimized for the first time, the initial score prediction model at this time is the initial score prediction model optimized for the second time (i.e., the optimized initial score prediction model corresponding to the second round of training), and so on, after optimizing the parameters in the initial score prediction model optimized for the N-1th time, the initial score prediction model at this time is the initial score prediction model optimized for the Nth time (i.e., the optimized initial score prediction model corresponding to the Nth round of training). The image verification sample set can be input into the optimized initial score prediction model corresponding to the Nth round of training to output the initial aesthetic scores of multiple sub-graphs of each image sample. Then, according to the target aesthetic scores of the multiple sub-graphs of each image sample and the initial aesthetic scores of the corresponding multiple sample sub-graphs (e.g., the mean square error between the target aesthetic scores of the multiple sub-graphs and the initial aesthetic scores of the corresponding multiple sample sub-graphs), the accuracy of the optimized initial score prediction model corresponding to the Nth round of training is determined. Similarly, after optimizing the parameters in the optimized initial score prediction model corresponding to the k*N-1th round of training, at this time, the initial score prediction model is the initial score prediction model optimized for the k*Nth time (i.e., the optimized initial score prediction model corresponding to the k*Nth round of training). The image verification sample set can be input into the initial score prediction model optimized for the k*Nth time to output the initial aesthetic scores of multiple sub-graphs of each image sample. Then, according to the initial aesthetic scores of the multiple sub-graphs of each image sample and the initial aesthetic scores of the corresponding multiple sample sub-graphs (for example, the mean square error between the initial aesthetic scores of the multiple sub-graphs and the initial aesthetic scores of the corresponding multiple sample sub-graphs), the accuracy of the initial score prediction model optimized for the k*Nth time is determined. Therefore, compared with verifying the optimized initial score prediction model corresponding to each round of training, in this embodiment, only the optimized models corresponding to multiple preset rounds of training are verified, thereby greatly reducing the overhead of the model training process and improving the efficiency of model training and verification.

[0095] Accordingly, the image processing method may further include: selecting an initial score prediction model with the highest accuracy from a plurality of optimized initial score prediction models obtained in the iterative training and optimization process as a trained initial score prediction model.

[0096] Specifically, if the initial score prediction model optimized for the i-th time among multiple optimized initial score prediction models has the highest accuracy, the initial score prediction model optimized for the i-th time is selected as the trained (ie, trained) initial score prediction model.

[0097] Before executing the step of training the initial score prediction model through the image training sample set, the image processing method also includes: obtaining the multiple first sample sub-images and the multiple second sample sub-images using a grid anchor point-based method; and determining the initial aesthetic score of each first sample sub-image according to the information richness and composition information of each first sample sub-image in the multiple first sample sub-images; and determining the initial aesthetic score of each second sample sub-image according to the information richness and composition information of each second sample sub-image in the multiple second sample sub-images.

[0098] The information richness may be the richness of the elements and their color features in the image. The composition information may be the layout and structure of the image (i.e., the connection relationship and combination structure of each element in the image) information, etc. Specifically, the Gridanchor method is used and the second preset rule (or the third preset rule) is used to obtain multiple first sample sub-graphs (or second sample sub-graphs). The second preset rule (or the third preset rule) includes the position information and size information of the multiple first sample sub-graphs. After obtaining multiple first sample sub-graphs (or second sample sub-graphs), the initial aesthetic score of each first sample sub-graph (or second sample sub-graph) is determined according to the information richness and composition information of each first sample sub-graph (or second sample sub-graph). For example, the corresponding first score may be obtained according to the information richness of the sample sub-graph; the corresponding second score may be obtained according to the composition information of the sample sub-graph; and then the first score and the second score are weightedly fused to obtain the initial aesthetic score (i.e., the standard aesthetic score) of the sample sub-graph.

[0099] Of course, in one embodiment, 11 labelers with photography-related backgrounds may be asked to score each sample sub-image respectively, and finally the average of the 11 scores labeled by the 11 labelers for each sample sub-image is used as the score of each sample sub-image (i.e., the initial aesthetic score).

[0100] The present embodiment creatively uses image training samples to train the above-mentioned initial score prediction model. Therefore, the present embodiment can predict the target aesthetic scores of multiple candidate sub-images of the image to be cropped through the trained initial score prediction model, thereby realizing image cropping by comprehensively considering the aesthetic factors and depth information of the sub-images, thereby greatly enhancing the image cropping effect.

[0101] In summary, the present invention creatively first obtains the initial aesthetic scores of multiple candidate sub-images at the target size of the image to be cropped and the depth information of the image to be cropped; then predicts the target aesthetic scores of the multiple candidate sub-images based on the initial aesthetic scores of the multiple candidate sub-images and the depth information of the image to be cropped; and finally crops the image to be cropped based on the target aesthetic scores of the multiple candidate sub-images. Therefore, the present invention can realize the cropping of the image by comprehensively considering the aesthetic factors and depth information of the sub-images, thereby greatly enhancing the cropping effect of the image.

[0102] Figure 8 FIG. 1 is a structural diagram of an image processing system provided by an embodiment of the present invention. Figure 8 As shown, the image processing system includes: an initial score acquisition device 10, used to acquire the initial aesthetic scores of multiple candidate sub-images at the target size of the image to be cropped; a depth information acquisition device 20, used to acquire the depth information of the image to be cropped; a target score acquisition device 30, used to predict the target aesthetic scores of the multiple candidate sub-images based on the initial aesthetic scores of the multiple candidate sub-images and the depth information of the image to be cropped; an image cropping device 40, used to crop the image to be cropped based on the target aesthetic scores of the multiple candidate sub-images.

[0103] Preferably, the initial score acquisition device 10 is used to obtain the initial aesthetic scores of multiple candidate sub-images at the target size of the image to be cropped, including: obtaining the initial aesthetic scores of multiple candidate sub-images at the target size of the image to be cropped through an initial score prediction model: extracting the features of the image to be cropped to obtain a feature map of the image to be cropped and a mapping relationship between the feature map of the image to be cropped and the image to be cropped; according to the feature map of the image to be cropped and the mapping relationship, obtaining the initial aesthetic scores of the multiple candidate sub-images at the target size, wherein the target size includes a target length ratio and a target width ratio of the multiple candidate sub-images to the image to be cropped.

[0104] Preferably, the initial score acquisition device 10 is used to obtain the initial aesthetic scores of multiple candidate sub-images at the target size through the initial score prediction model, including: cutting multiple sub-feature maps of the feature map of the image to be cut according to a first preset rule, wherein the multiple sub-feature maps at least cover the feature map of the image to be cut, and the first preset rule includes: the length ratio and width ratio of the multiple sub-feature maps to the feature map of the image to be cut are the target length ratio and the target width ratio respectively, and the positions of the multiple sub-feature maps are multiple first preset positions; obtaining the initial aesthetic scores of the multiple sub-feature maps; and obtaining the initial aesthetic scores of the multiple candidate sub-images corresponding to the multiple sub-feature maps according to the mapping relationship and the initial aesthetic scores of the multiple sub-feature maps.

[0105] Preferably, the target score prediction device 30 is used to predict the target aesthetic scores of the multiple candidate sub-images, including: a depth information determination module (not shown), used to determine the depth information of the multiple candidate sub-images according to the depth information of the image to be cropped, the position information of the multiple sub-feature images and the mapping relationship; a depth factor determination module (not shown), used to determine the depth factors of the multiple candidate sub-images according to the depth information of the multiple candidate sub-images; and a target score determination module (not shown), used to determine the target aesthetic scores of the multiple candidate sub-images according to the depth factors of the multiple candidate sub-images and the initial aesthetic scores of the multiple candidate sub-images.

[0106] Preferably, the depth factor determination module is used to determine the depth factors of the plurality of candidate sub-images, comprising: for a candidate sub-image i among the plurality of candidate sub-images, according to the maximum depth value of the candidate sub-image i With minimum depth value And the following formula, determine the depth factor α of the candidate sub-image i i ,

[0107] Preferably, the target score determination module is used to determine the target aesthetic scores of the plurality of candidate sub-images, including: for a candidate sub-image i among the plurality of candidate sub-images, according to the depth factor α of the candidate sub-image i i , the initial aesthetic score of the candidate sub-graph i The target aesthetic score of the candidate sub-image i is determined by the following formula:

[0108] Preferably, the image cropping device 40 includes: a screening module (not shown), used to screen out the candidate sub-image with the highest target aesthetic score from the multiple candidate sub-images; and a cropping module (not shown), used to crop out the candidate sub-image with the highest target aesthetic score from the image to be cropped.

[0109] The specific details and benefits of the image processing system provided by the present invention can be found in the above description of the image processing method, which will not be repeated here.

[0110] An embodiment of the present invention further provides a computer-readable storage medium, on which a computer program is stored. When the computer program is executed by a processor, the steps of the above-mentioned image processing method are implemented.

[0111] An embodiment of the present invention further provides an electronic device, comprising: a processor; a memory for storing a computer program; the processor is configured to read the computer program from the memory and execute the computer program to implement the steps of the above-mentioned image processing method.

[0112] An embodiment of the present invention further provides a computer program product, which includes a computer program. When the computer program is executed by a processor, the steps of the above-mentioned image processing method are implemented.

[0113] The optional implementation modes of the embodiments of the present invention are described in detail above in conjunction with the accompanying drawings. However, the embodiments of the present invention are not limited to the specific details in the above implementation modes. Within the technical concept of the embodiments of the present invention, various simple modifications can be made to the technical scheme of the embodiments of the present invention, and these simple modifications all belong to the protection scope of the embodiments of the present invention.

[0114] It should also be noted that the various specific technical features described in the above specific embodiments can be combined in any suitable manner without contradiction. To avoid unnecessary repetition, the embodiments of the present invention will not further describe various possible combinations.

[0115] Those skilled in the art can understand that all or part of the steps in the above-mentioned embodiment method can be completed by instructing the relevant hardware through a program, and the program is stored in a storage medium, including a number of instructions to enable a single-chip microcomputer, a chip or a processor to execute all or part of the steps of the method described in each embodiment of the present application. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (ROM, Read-Only Memory), random access memory (RAM, Random Access Memory), disk or optical disk and other media that can store program codes.

[0116] In addition, various implementation modes of the embodiments of the present invention may be arbitrarily combined, and as long as they do not violate the concept of the embodiments of the present invention, they should also be regarded as the contents disclosed by the embodiments of the present invention.

Claims

1. An image processing method, characterized in that: The image processing method comprises: Obtaining initial aesthetic scores of multiple candidate sub-images at a target size of the image to be cropped through an initial score prediction model; Acquire depth information of the image to be cropped; Predicting target aesthetic scores of the plurality of candidate sub-images according to the initial aesthetic scores of the plurality of candidate sub-images and the depth information of the image to be cropped; and The image to be cropped is cropped according to the target aesthetic scores of the multiple candidate sub-images, The method of obtaining the initial aesthetic scores of the multiple candidate sub-images at the target size of the image to be cropped by using the initial score prediction model includes: extracting the features of the image to be cropped to obtain the feature map of the image to be cropped and the mapping relationship between the feature map of the image to be cropped and the image to be cropped, wherein the mapping relationship includes the position mapping relationship and the size mapping relationship between the multiple sub-feature maps on the feature map and the multiple candidate sub-images on the image to be cropped; obtaining the initial aesthetic scores of the multiple candidate sub-images at the target size according to the feature map of the image to be cropped, the initial aesthetic scores of the multiple sub-feature maps and the mapping relationship, wherein the target size includes the target length ratio and the target width ratio of the multiple candidate sub-images to the image to be cropped; The step of predicting target aesthetic scores of multiple candidate sub-images includes: Determine the depth information of the plurality of candidate sub-images according to the depth information of the image to be cropped, the position information of the plurality of sub-feature images, and the mapping relationship; Determining depth factors of the plurality of candidate sub-images according to the depth information of the plurality of candidate sub-images; and The target aesthetic scores of the plurality of candidate sub-images are determined according to the depth factors of the plurality of candidate sub-images and the initial aesthetic scores of the plurality of candidate sub-images.

2. The image processing method according to claim 1, characterized in that: The obtaining of initial aesthetic scores of the plurality of candidate sub-images at the target size comprises: Pre-cutting the feature map of the image to be cut according to a first preset rule to obtain a plurality of sub-feature maps, wherein the plurality of sub-feature maps at least cover the feature map of the image to be cut, and the first preset rule includes: the length ratio and width ratio of the plurality of sub-feature maps to the feature map of the image to be cut are respectively a target length ratio and a target width ratio, and the positions of the plurality of sub-feature maps are a plurality of first preset positions; Obtaining initial aesthetic scores of the plurality of sub-feature maps; and According to the mapping relationship and the initial aesthetic scores of the multiple sub-feature graphs, initial aesthetic scores of the multiple candidate sub-graphs corresponding to the multiple sub-feature graphs are obtained.

3. The image processing method according to claim 1, characterized in that: The image processing method further comprises: a step of training the initial score prediction model using an image training sample set, Each of the image training sample sets includes: a first image and initial aesthetic scores of multiple first sample sub-images of the first image pre-cropped according to a second preset rule, wherein the multiple first sample sub-images at least cover the first image, and the second preset rule includes that the positions of the multiple first sample sub-images are multiple second preset positions.

4. The image processing method according to claim 3, characterized in that: The training of the initial score prediction model by using the image training sample set includes: The initial score prediction model is trained by randomly selecting an image training sample group from the image training sample set to obtain initial aesthetic scores of a plurality of first sub-images of each of the selected image training sample groups, wherein the image training sample group includes a preset number of image training samples, and the plurality of first sub-images are obtained by pre-cropping each of the image training sample groups according to a rule corresponding one-to-one to the position, length, and width of the plurality of first sample sub-images; Optimizing parameters in the initial score prediction model according to the initial aesthetic scores of the plurality of first sub-images and the initial aesthetic scores of the plurality of first sample sub-images; and The optimized initial score prediction model is iteratively trained and optimized by randomly selecting an image training sample group from the image training sample set until the training process meets a preset stop condition.

5. The image processing method according to claim 4, characterized in that: Training the initial score prediction model further includes: Verifying the optimized initial score prediction models corresponding to multiple preset rounds of training through an image verification sample set to obtain multiple second sub-images and initial aesthetic scores of the multiple second sub-images for each of the image verification sample sets, wherein each of the image verification sample sets includes: a second image and initial aesthetic scores of multiple second sample sub-images of the second image pre-cut according to a third preset rule, the multiple second sample sub-images at least covering the second image, the third preset rule including that position information of the multiple second sample sub-images is multiple third preset positions, and the multiple second sub-images are obtained by pre-cutting each of the image verification sample groups according to a rule corresponding one-to-one to the position, length and width of the multiple second sample sub-images; and Determining the accuracy of the optimized initial score prediction models corresponding to the plurality of preset rounds of training according to the initial aesthetic scores of the plurality of second sub-images and the initial aesthetic scores of the plurality of second sample sub-images, Accordingly, the image processing method further includes: selecting the initial score prediction model with the highest accuracy from the multiple optimized initial score prediction models obtained in the iterative training and optimization process as the trained initial score prediction model.

6. The image processing method according to claim 5, characterized in that: Before executing the step of training the initial score prediction model using the image training sample set, the image processing method further includes: Acquire the plurality of first sample sub-images and the plurality of second sample sub-images by adopting a method based on a grid anchor point; Determining an initial aesthetic score of each first sample sub-image according to the information richness and composition information of each first sample sub-image among the plurality of first sample sub-images; and An initial aesthetic score of each second sample sub-image is determined according to the information richness and composition information of each second sample sub-image among the plurality of second sample sub-images.

7. The image processing method according to claim 1, characterized in that: The determining the depth factors of the plurality of candidate sub-images comprises: For a candidate sub-image i among the multiple candidate sub-images, according to the maximum depth value of the candidate sub-image i With minimum depth value And the following formula, determine the depth factor α of the candidate sub-image i i , 8. The image processing method according to claim 1, characterized in that: Determining target aesthetic scores of the plurality of candidate sub-images comprises: For a candidate sub-image i among the multiple candidate sub-images, according to the depth factor α of the candidate sub-image i i , the initial aesthetic score of the candidate sub-graph i The target aesthetic score of the candidate sub-image i is determined by the following formula:

9. The image processing method according to claim 1, characterized in that: The cutting of the image to be cut comprises: Selecting a candidate sub-graph with the highest target aesthetic score from the plurality of candidate sub-graphs; and A candidate sub-image with the highest target aesthetic score is cropped from the image to be cropped.

10. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a computer program, which, when executed by a processor, implements the steps of the image processing method described in any one of claims 1 to 9.

11. A computer program product, comprising a computer program, characterized in that: When the computer program is executed by a processor, the steps of the image processing method described in any one of claims 1 to 9 are implemented.

Images