Training method and device, image processing method and device, equipment and storage medium

Abstract

The invention provides an image cutting model training method and device, an image processing method and device, equipment and a storage medium, and relates to the technical field of computers, in particular to the technical field of deep learning and image processing. According to the specific implementation scheme, sample data is obtained, wherein the sample data at least comprises a sample image, a first cut image obtained after the sample image is cut in a first mode, and a second cut image obtained after the sample image is cut in a second mode; determining a target loss function which is at least used for constraining a difference value between a first prediction attribute value of the first cut image and a first theoretical attribute value of the first cut image and constraining a difference value between a second prediction attribute value of the second cut image and a second theoretical attribute value of the second cut image; and at least utilizing the sample data and the target loss function to perform model training on a preset image cutting model to obtain a target image cutting model.

Description

Technical Field

[0001] The present disclosure relates to the field of computer processing technology, and in particular to the technical fields of artificial intelligence, big data, big models, etc. Background Art

[0002] Information flow products generate image content including pictures, texts or videos, and these images have different aspect ratios. In order to ensure the aesthetics and consistency of images displayed in different scenes on mobile terminals, these images usually need to be automatically cropped. However, the complexity of image content and the different size requirements of different scenes bring challenges to cropping technology. Summary of the invention

[0003] The present invention provides an image cropping model training method, an image processing method, and an apparatus, a device, and a storage medium thereof.

[0004] According to one aspect of the present disclosure, a method for training an image cropping model is provided, comprising:

[0005] Acquire sample data, wherein the sample data at least includes: a sample image, a first cropped image obtained by cropping the sample image in a first manner, and a second cropped image obtained by cropping the sample image in a second manner;

[0006] Determining a target loss function, wherein the target loss function is used at least to: constrain a difference between a first predicted attribute value of the first cropped image and a first theoretical attribute value of the first cropped image, and to constrain a difference between a second predicted attribute value of the second cropped image and a second theoretical attribute value of the second cropped image;

[0007] The preset image cropping model is trained using at least the sample data and the target loss function to obtain a target image cropping model, and the first predicted attribute value and the second predicted attribute value are obtained using the preset image cropping model.

[0008] According to another aspect of the present disclosure, a training device for an image cropping model is provided, comprising:

[0009] A first acquisition unit is used to acquire sample data, wherein the sample data at least includes: a sample image, a first cropped image obtained by cropping the sample image in a first manner, and a second cropped image obtained by cropping the sample image in a second manner;

[0010] A training unit is used to determine a target loss function, wherein the target loss function is used at least to: constrain the difference between a first predicted attribute value of the first cropped image and a first theoretical attribute value of the first cropped image, and to constrain the difference between a second predicted attribute value of the second cropped image and a second theoretical attribute value of the second cropped image; and to perform model training on a preset image cropping model using at least the sample data and the target loss function to obtain a target image cropping model, wherein the first predicted attribute value and the second predicted attribute value are obtained using the preset image cropping model.

[0011] According to another aspect of the present disclosure, there is provided an image processing method, comprising:

[0012] Get the image to be cropped;

[0013] At least the image to be cropped is input into a target image cropping model to obtain a target cropped image, wherein the target image cropping model is obtained based on the training method of the above image cropping model.

[0014] According to another aspect of the present disclosure, there is provided an image processing apparatus, comprising:

[0015] A second acquisition unit, used to acquire the image to be cropped;

[0016] The model inference unit is used to input at least the image to be cropped into a target image cropping model to obtain a target cropped image, wherein the target image cropping model is obtained based on the training method of the above image cropping model.

[0017] According to another aspect of the present disclosure, there is provided an electronic device, comprising:

[0018] at least one processor; and

[0019] a memory communicatively connected to the at least one processor; wherein,

[0020] The memory stores instructions that can be executed by the at least one processor, and the instructions are executed by the at least one processor to enable the at least one processor to execute any method in the embodiments of the present disclosure.

[0021] According to another aspect of the present disclosure, a non-transitory computer-readable storage medium storing computer instructions is provided, wherein the computer instructions are used to enable the computer to execute any method according to the embodiments of the present disclosure.

[0022] According to another aspect of the present disclosure, a computer program product is provided, including a computer program, which implements any method according to the embodiments of the present disclosure when executed by a processor.

[0023] In this way, the disclosed solution can use the target loss function and the acquired sample data to train the preset image cropping model to obtain a model that can be used for image cropping (that is, corresponding to the target image cropping model described above). In this way, the cropped image can effectively avoid problems such as character truncation and text truncation, thereby improving the cropping accuracy, providing support for meeting the multi-size requirements of different scenes, and effectively improving the user experience.

[0024] It should be understood that the content described in this section is not intended to identify the key or important features of the embodiments of the present disclosure, nor is it intended to limit the scope of the present disclosure. Other features of the present disclosure will become easily understood through the following description. BRIEF DESCRIPTION OF THE DRAWINGS

[0025] The accompanying drawings are used to better understand the present solution and do not constitute a limitation of the present disclosure.

[0026] Figure 1 This is a schematic flow chart of a training method for an image cropping model according to an embodiment of the present application. Figure 1 ;

[0027] Figure 2 This is a schematic flow chart of a training method for an image cropping model according to an embodiment of the present application. Figure 2 ;

[0028] Figure 3 is a schematic diagram of a model training scenario for a preset image cropping model according to an embodiment of the present application;

[0029] Figure 4 is a schematic diagram of a scene of a preset image cropping model according to an embodiment of the present application;

[0030] Figure 5 is a flowchart of a training method for an image cropping model according to an embodiment of the present application in an example;

[0031] Figure 6 It is a schematic diagram of a process for automatically constructing sample data according to an embodiment of the present application;

[0032] Figure 7 is a structural schematic diagram of an image cropping system according to an embodiment of the present application;

[0033] Figure 8 is a schematic flow chart of an image processing method according to an embodiment of the present application;

[0034] Fig. 9 is a structural schematic diagram of a training device for an image cropping model according to an embodiment of the present application;

[0035] Fig.10is a structural schematic diagram of an image processing device according to an embodiment of the present application;

[0036] Fig.11 It is a block diagram of an electronic device used to implement the training method or image processing method of the image cropping model of the embodiment of the present disclosure. DETAILED DESCRIPTION

[0037] The following is a description of exemplary embodiments of the present disclosure in conjunction with the accompanying drawings, including various details of the embodiments of the present disclosure to facilitate understanding, which should be considered as merely exemplary. Therefore, it should be recognized by those of ordinary skill in the art that various changes and modifications may be made to the embodiments described herein without departing from the scope of the present disclosure. Similarly, for the sake of clarity and conciseness, the description of well-known functions and structures is omitted in the following description.

[0038] The term "and / or" in this article is only a description of the association relationship of associated objects, indicating that there may be three relationships. For example, A and / or B can mean: A exists alone, A and B exist at the same time, and B exists alone. The term "at least one" in this article means any combination of at least two of any one or more of a plurality of. For example, including at least one of A, B, and C can mean including any one or more elements selected from the set consisting of A, B, and C. The terms "first" and "second" in this article refer to multiple similar technical terms and distinguish them. They do not mean to limit the order or to limit them to only two. For example, the first feature and the second feature refer to two types / two features. The first feature can be one or more, and the second feature can also be one or more.

[0039] In addition, in order to better illustrate the present disclosure, numerous specific details are given in the specific embodiments below. It should be understood by those skilled in the art that the present disclosure can also be implemented without certain specific details. In some examples, methods, means, components and circuits well known to those skilled in the art are not described in detail in order to highlight the main purpose of the present disclosure.

[0040] The following is a description of the related technologies of the embodiments of the present disclosure. The following related technologies are optional solutions that can be arbitrarily combined with the technical solutions of the embodiments of the present disclosure, and they all belong to the protection scope of the embodiments of the present disclosure.

[0041] The disclosed solution proposes a training method for an image cropping model to improve the accuracy of image cropping, effectively ensuring the beauty and consistency of the image when displayed in different scenes on a mobile terminal, thereby improving the user experience.

[0042] Specifically, Figure 1 This is a schematic flow chart of a training method for an image cropping model according to an embodiment of the present application. Figure 1 The method may be optionally applied to electronic devices, such as personal computers, servers, server clusters, and other electronic devices.

[0043] Further, the method includes at least part of the following contents. Figure 1 As shown, including:

[0044] Step S101: Obtain sample data.

[0045] Here, the sample data at least includes: a sample image, a first cropped image obtained by cropping the sample image in a first manner, and a second cropped image obtained by cropping the sample image in a second manner.

[0046] For example, in one example, M groups of sample data may be obtained; accordingly, each sample data in the M groups of sample data at least includes: a sample image, N1 first cropped images obtained by cropping the sample image in a first manner, and N2 second cropped images obtained by cropping the sample image in a second manner. Here, M, N1, and N2 are all natural numbers greater than or equal to 1.

[0047] Further, M is a natural number greater than or equal to 2. N1 and N2 are natural numbers greater than or equal to 1. Here, it can be understood that the values ​​of N1 and N2 can be the same or different, and the disclosed solution does not limit this.

[0048] It should be noted that the first method and the second method described above are different cropping methods, for example, they may specifically refer to cropping the sample image from different directions; further, in one example, the sample data of the disclosed scheme may also include multiple cropped images obtained by cropping the sample image using other cropping methods. In other words, the disclosed scheme does not limit the number of cropping methods used in the sample data.

[0049] Furthermore, in one example, the sample data may also include theoretical attribute values ​​of each first cropped image and theoretical attribute values ​​of each second cropped image, thereby providing label data for subsequent model training.

[0050] It should be noted that the sample image in this example can be specifically understood as an initial image that has not been cropped or marked with a cropping frame; accordingly, the cropped image (e.g., the first cropped image or the second cropped image) can be understood as an image after the initial image is cropped or marked with a cropping frame; further, the theoretical attribute value of the cropped image (e.g., the first cropped image or the second cropped image) can be specifically based on the attribute value obtained after evaluating the cropped image (e.g., the first cropped image or the second cropped image) based on the optimal cropped image of the sample image. For example, in one example, based on the optimal cropped image of the sample image, the Intersection Over Union (IOU) of the cropped image (e.g., the first cropped image or the second cropped image) is calculated, and at this time, the calculated IOU of the cropped image can be used as the theoretical attribute value of the cropped image.

[0051] Furthermore, in one example, the attribute value described above may be specifically an aesthetic score. In this case, the predicted attribute value described in the disclosed solution may be specifically a predicted aesthetic score, and correspondingly, the theoretical attribute value may be specifically a theoretical aesthetic score.

[0052] For example, in one example, the i-th (i is an integer greater than 0 and less than or equal to M) group of sample data in M ​​groups of sample data may specifically include sample image i, N1 first cropped images obtained by cropping sample image i using the first method, theoretical aesthetic scores of each first cropped image, N2 second cropped images obtained by cropping sample image i using the second method, and theoretical aesthetic scores of each second cropped image. In this way, rich and high-quality training samples are effectively obtained, laying a foundation for improving the training effect of the cropping model in the future.

[0053] Step S102: Determine the target loss function.

[0054] Here, the objective loss function is at least used to: constrain (or represent) the difference between a first predicted attribute value (e.g., a first predicted aesthetic score) of the first cropped image and a first theoretical attribute value (e.g., a first theoretical aesthetic score) of the first cropped image, and to constrain the difference between a second predicted attribute value (e.g., a second predicted aesthetic score) of the second cropped image and a second theoretical attribute value (e.g., the first theoretical aesthetic score) of the second cropped image.

[0055] Step S103: performing model training on a preset image cropping model using at least the sample data and the target loss function to obtain a target image cropping model.

[0056] Here, the first predicted attribute value and the second predicted attribute value are obtained by using the preset image cropping model.

[0057] For example, in one example, with respect to obtaining M groups of sample data, the preset image cropping model may be trained using at least the M groups of sample data and a determined target loss function to obtain a target image cropping model.

[0058] It can be understood that, in one example, the target loss function is used to constrain (or is called to represent) the difference between the first predicted attribute value of the first cropped image and the first theoretical attribute value of the first cropped image; or, in another example, the target loss function is used to constrain (or is called to represent) the difference between the second predicted attribute value of the second cropped image and the second theoretical attribute value of the second cropped image; or, in another example, the target loss function is used to constrain (or is called to represent) the difference between the first predicted attribute value of the first cropped image and the first theoretical attribute value of the first cropped image, and constrain (or is called to represent) the difference between the second predicted attribute value of the second cropped image and the second theoretical attribute value of the second cropped image. In this way, technical support is provided for the trained model to have a better cropping effect.

[0059] For example, in one example, the difference between the predicted attribute value (e.g., predicted aesthetic score) of the cropped image and the theoretical attribute value (e.g., theoretical aesthetic score) of the cropped image can be calculated by a regression loss function; at this time, the target loss function (which can be recorded as Loss) in this example can be based on the regression loss function (which can be recorded as L 1―回归 ) and the regression loss function of the second cropped image (which can be recorded as L 2―回归 ), for example, in one example, the target loss function is the sum of the regression loss function of the first cropped image and the regression loss function of the second cropped image, that is, Loss = L 1―回归 +L 2―回归 .

[0060] In this way, the disclosed solution can use the target loss function and the acquired sample data to train the preset image cropping model to obtain a model that can be used for image cropping (that is, corresponding to the target image cropping model described above). In this way, the cropped image can effectively avoid problems such as character truncation and text truncation, improve the cropping accuracy, provide support for meeting the multi-size requirements of different scenes, and effectively improve the user experience.

[0061] In a specific example, the above-mentioned first method and the second method are different cropping methods, such as horizontal cropping method and vertical cropping method based on sliding window and card threshold scaling, etc., and the disclosed solution does not impose specific restrictions on this. In this way, technical support is provided for the trained model to have a better cropping effect.

[0062] Furthermore, in one example, the first cropped image is obtained by cropping the sample image horizontally; and / or the second cropped image is obtained by cropping the sample image vertically.

[0063] For example, in one example, the first method is a horizontal cropping method, and the second method is a vertical cropping method. At this time, the first cropped image is a horizontal cropped image obtained by cropping the sample image horizontally, and accordingly, the second cropped image is a vertical cropped image obtained by cropping the sample image vertically.

[0064] It is understandable that the above cutting method is only an example description. In actual application, it can be set based on specific scene requirements, and the present disclosure does not limit this.

[0065] In this way, the disclosed solution can use different cropping methods to obtain multiple cropped images (such as the first cropped image and the second cropped image). In this way, the richness of samples required for model training is improved, so that the model can learn diverse features, thereby having stronger generalization capabilities and being suitable for different application scenarios, thereby further improving the user experience.

[0066] In a specific example of the disclosed solution, in order to further improve the model training effect, the target loss function is also used for at least one of the following:

[0067] In the case where the sample data includes two or more first cropped images (i.e., two or more first cropped images are obtained after cropping the sample images in the first manner), constraining (also referred to as expressing) the similarity between the first actual sorting result and the first theoretical sorting result; wherein the first actual sorting result is the result of sorting all the first cropped images (i.e., the N1 first cropped images mentioned above) according to the values ​​of the first predicted attribute values ​​of each first cropped image, and the first theoretical sorting result is the result of sorting all the first cropped images (i.e., the N1 first cropped images mentioned above) according to the values ​​of the first theoretical attribute values ​​of each first cropped image;

[0068] When the sample data includes two or more second cropped images (that is, two or more second cropped images are obtained after cropping the sample images in the first manner), the similarity between the second actual sorting result and the second theoretical sorting result is constrained (also referred to as expressed); wherein the second actual sorting result is the result of sorting all the second cropped images (that is, the N2 second cropped images mentioned above) according to the numerical values ​​of the second predicted attribute values ​​of each second cropped image, and the second theoretical sorting result is the result of sorting all the second cropped images (that is, the N2 second cropped images mentioned above) according to the numerical values ​​of the second theoretical attribute values ​​of each second cropped image.

[0069] For example, continuing to take the obtained M groups of sample data as an example, in one example, the target loss function is used to:

[0070] Constrain the difference between the first predicted attribute value of the first cropped image and the first theoretical attribute value of the first cropped image, and constrain the difference between the second predicted attribute value of the second cropped image and the second theoretical attribute value of the second cropped image, and constrain the similarity between the first actual sorting result and the first theoretical sorting result.

[0071] Alternatively, in another example, the objective loss function is used to:

[0072] Constrain the difference between the first predicted attribute value of the first cropped image and the first theoretical attribute value of the first cropped image, and constrain the difference between the second predicted attribute value of the second cropped image and the second theoretical attribute value of the second cropped image, and constrain the similarity between the second actual sorting result and the second theoretical sorting result.

[0073] Alternatively, in yet another example, the objective loss function is used to:

[0074] Constrain the difference between the first predicted attribute value of the first cropped image and the first theoretical attribute value of the first cropped image, and constrain the difference between the second predicted attribute value of the second cropped image and the second theoretical attribute value of the second cropped image, and constrain the similarity between the first actual ranking result and the first theoretical ranking result, and constrain the similarity between the second actual ranking result and the second theoretical ranking result.

[0075] It should be noted that the similarity between the actual sorting results and the theoretical sorting results of the multiple cropped images included in the sample data can be calculated by the sorting loss function; at this time, the target loss function in this example can also be based on the sorting loss function of the first cropped image (which can be recorded as L 1―排序 ) and / or the ranking loss function of the second cropped image (which can be recorded as L 2―排序 ) income.

[0076] For example, in one example, the target loss function may be obtained based on at least one or more of the following loss functions:

[0077] The regression loss function of the first crop image (L 1―回归 );

[0078] The ranking loss function of the first crop image (L 1―排序 );

[0079] The regression loss function of the second crop image (L 2―回归 );

[0080] The ranking loss function of the second crop image (L 2―排序 ).

[0081] Furthermore, in one example, the overall loss function of the first cropped image (which can be recorded as L 1―总体 ), based on the regression loss function of the second cropped image and the sorting loss function of the second cropped image, the overall loss function of the second cropped image is obtained (which can be recorded as L 2―总体 ).

[0082] At this time, the target loss function can be obtained based on the overall loss function of the first cropped image and the overall loss function of the second cropped image, for example, Loss = L 1―总体 +L 2―总体 , Further, in one example, the overall loss function L of the first cropped image is 1―总体 =L 1―回归 +L 1―排序 , the overall loss function L of the second cropped image 2―总体 =L 2―回归 +L 2―排序 , at this time, the target loss function Loss = L 1―回归 +L 1―排序 +L 2―回归 +L 2―排序 .

[0083] It is understandable that weights can also be added to the target loss function based on actual needs to adapt to different cropping requirements. For example, the target loss function Loss = a 1 L 1―回归 +a 2 L 1―排序 +b 1 L 2―回归 +b 2 L 2―排序 Here, a 1 、a 2 , b 1and b 2 It can be determined based on actual needs.

[0084] In this way, the disclosed scheme provides one or more design schemes for target loss functions, so that the parameters of the preset image cropping model can be effectively optimized, thereby improving the cropping accuracy of the model. Moreover, the constructed loss function can adapt to different types of task requirements, so that the trained model can crop images that meet the size requirements of specific scenes, thereby improving the user experience.

[0085] Specifically, Figure 2 This is a schematic flow chart of a training method for an image cropping model according to an embodiment of the present application. Figure 2 The method can be optionally applied to electronic devices, such as personal computers, servers, server clusters and other electronic devices. It can be understood that the above Figure 1 The relevant contents of the method shown can also be applied to this example, and the relevant contents will not be described in detail in this example.

[0086] Further, the method includes at least part of the following contents. Figure 2 As shown, including:

[0087] Step S201: Obtain sample data.

[0088] Here, the sample data at least includes: a sample image, a first cropped image obtained by cropping the sample image in a first manner, and a second cropped image obtained by cropping the sample image in a second manner.

[0089] Here, the relevant content about the sample data can refer to the above examples, which will not be repeated here.

[0090] Step S202: Determine the target loss function.

[0091] Here, the objective loss function is at least used to: constrain (or represent) the difference between the first predicted attribute value of the first cropped image and the first theoretical attribute value of the first cropped image, and to constrain the difference between the second predicted attribute value of the second cropped image and the second theoretical attribute value of the second cropped image.

[0092] Here, the relevant content about the target loss function can be referred to the above example, which will not be repeated here.

[0093] Step S203: Acquire the target position information of the sample image.

[0094] In one example, the above-mentioned obtaining the target position information of the sample image (for example, the above-mentioned step S203) may specifically include: inputting the sample image into a target detection model to obtain the outputted target position information. Here, the target detection model is used to detect the position of the target in the input image.

[0095] That is to say, in this example, the target detection model can be used to detect the position of the target body in the sample image to obtain the specific position information of each target body in the sample image (that is, corresponding to the target body position information described above). In this way, the model can extract effective features based on the target body position information for use in evaluating the cropped image, thereby providing strong support for improving the cropping accuracy of the model.

[0096] Step S204: inputting the sample image, the target position information, the first cropped image and the second cropped image into a preset image cropping model to obtain a first predicted attribute value and a second predicted attribute value as output.

[0097] For example, continuing with the acquired M groups of sample data as an example, the sample image, the target position information of the sample image, the N1 first cropped images and the N2 second cropped images contained in the sample image are input into the preset image cropping model to obtain the first predicted attribute value of each first cropped image and the second predicted attribute value of each second cropped image.

[0098] Step S205: Based on the first predicted attribute value and the second predicted attribute value, obtain the loss value of the target loss function.

[0099] For example, in one example, based on the obtained first predicted attribute values ​​of each first cropped image and the obtained second predicted attribute values ​​of each second cropped image, the loss value of the target loss function is obtained.

[0100] Step S206: Based on the loss value, the adjustable parameters in the preset image cropping model are adjusted to obtain a target image cropping model while meeting the model training requirements.

[0101] For example, in one example, taking the obtained M groups of sample data as an example, Figure 3As shown, first, the sample image, the target body position information of the sample image, the N1 first cropped images and the N2 second cropped images contained in the sample image are input into the preset image cropping model to obtain the first predicted aesthetic score of each first cropped image and the second predicted aesthetic score of each second cropped image; secondly, based on the difference between the first predicted aesthetic score of each first cropped image and its first theoretical aesthetic score, and the difference between the second predicted aesthetic score of each second cropped image and its second theoretical aesthetic score, the loss value of the target loss function is obtained; finally, when the loss value or the number of iterations of the target loss function does not meet the model training requirements, the adjustable parameters in the preset image cropping model are adjusted based on the loss value of the target loss function, and this cycle is repeated until the loss value of the target loss function meets the model training requirements. In this way, a trained target image cropping model can be obtained.

[0102] In this way, the disclosed solution provides a training solution for a preset image cropping model to efficiently train a target image cropping model, and the cropped image obtained by using the trained target image cropping model can effectively avoid problems such as character truncation and text truncation, effectively improve the cropping accuracy, and thus effectively improve the user experience. Moreover, the target image cropping model of the disclosed solution can also crop images of various sizes, effectively meeting the multi-size requirements of different scenes, thereby improving the user experience.

[0103] Here, it should be noted that the preset image cropping model mentioned above can be specifically a neural network model for image cropping, or it can also be specifically a large model for image cropping, etc., and the present disclosure does not limit this.

[0104] Furthermore, in a specific example of the disclosed solution, the preset image cropping model includes at least a first branch and a second branch.

[0105] For instance, in one example, the first branch may be used at least to process the global features of the sample image and the global features of the first cropped image to obtain a first predicted attribute value of the first cropped image; further, in another example, the second branch may be used at least to process the global features of the sample image and the global features of the second cropped image to obtain a second predicted attribute value of the second cropped image.

[0106] In this way, the disclosed solution provides a specific solution for the model structure of a preset image cropping model, thereby improving the ability to learn different cropping methods and having better interpretation capabilities, thereby improving the accuracy of model cropping. Moreover, for the size requirements of specific scenes, the disclosed solution can crop high-quality cropped images, thereby improving the user experience.

[0107] Here, the first branch and the second branch included in the above-mentioned preset image cropping model are only an example. In addition, when the sample data also includes multiple cropped images obtained by cropping the sample image in other cropping methods, the preset image cropping model may also include branches corresponding to other cropping methods. For example, in one example, the number of branches in the preset image cropping model is related to the number of cropping methods. For example, Figure 4 As shown, the number of branches in the preset image cropping model is the same as the number of cropping methods, that is, each of the n (a positive integer greater than or equal to 2) branches contained in the preset image cropping model corresponds to a cropping method.

[0108] Here, the number of branches included in the preset image cropping model can be set according to the needs of the actual scene, and the present disclosure does not impose any specific restrictions on this.

[0109] Further, in one example, the first branch at least includes a first feature alignment module, a first graph attention network (GAT) and a first multilayer perceptron (MLP). Further, the first feature alignment module is mainly used to perform feature alignment on the global features of the sample image and the global features of the first cropped image. For example, in one example, the global features of the sample image and the global features of the first cropped image can be feature aligned using techniques such as region of interest alignment (RoI Align) and region of difference alignment (RoD Align). Further, the first GAT is mainly used to perform attention processing on the features of the sample image after feature alignment and the features of the first cropped image; the first MLP is mainly used to perform feature matching on the features of the sample image after attention processing and the features of the first cropped image to obtain the first predicted attribute value of the first cropped image.

[0110] Alternatively, in another example, the second branch at least includes a second feature alignment module, a second graph attention network GAT, and a second multi-layer perceptron MLP; further, the second feature alignment module is mainly used to perform feature alignment on the global features of the sample image and the global features of the second cropped image. For example, in one example, the global features of the sample image can be aligned with the global features of each second cropped image using RoIAlign and RoD Align techniques. Further, the second GAT is mainly used to perform attention processing on the features of the sample image after feature alignment and the features of the second cropped image; the second MLP is mainly used to perform feature matching on the features of the sample image after attention processing and the features of the second cropped image to obtain the second predicted attribute value of the second cropped image.

[0111] Alternatively, in yet another example, the first branch includes at least a first feature alignment module, a first GAT, and a first MLP, and the second branch includes at least a second feature alignment module, a second GAT, and a second MLP.

[0112] In this way, the disclosed scheme can utilize different processing branches to process the cropped images (such as the first cropped image or the second cropped image) obtained by different cropping methods to obtain the predicted attribute values ​​of each cropped image. In this way, the model's learning ability for cropped images using different cropping methods can be effectively improved, thereby laying the foundation for meeting the cropping requirements of different scenes and effectively improving the cropping accuracy of the model.

[0113] Furthermore, in a specific example, the preset image cropping model also includes a shared backbone network.

[0114] Here, the output of the shared backbone network can be used as the input of the first branch and the second branch; further, the shared backbone network is used to: obtain the global features of the sample image based on the input sample image and the target body position information of the sample image; obtain the global features of the first cropped image based on the input first cropped image; and obtain the global features of the second cropped image based on the input second cropped image.

[0115] For example, in one example, a shared backbone network, such as a residual network (Residual Network, ResNet), can perform feature extraction on the target body in the sample image based on the target body position information of the sample image to obtain the global features of the sample image; in addition, the residual network also performs feature extraction on the input first cropped image to obtain the global features of the first cropped image, and performs feature extraction on the input second cropped image to obtain the global features of the second cropped image.

[0116] It should be noted that in addition to the residual network, the shared backbone network can also be a pre-trained network such as Mobile Network V2 (MobileNetV2), Efficient Network (EfficientNet), Swin Transformer, etc., and the present disclosure does not impose any specific restrictions on this.

[0117] In this way, the disclosed solution can utilize the shared backbone network to extract effective features from the input image, and input the extracted effective features into the corresponding branches respectively, thereby laying the foundation for subsequently improving the evaluation efficiency of the cropped image in each branch.

[0118] The present disclosure is further described in detail below with reference to specific examples. Specifically, Figure 5 FIG. 1 is a flow chart of an end-to-end image cropping model training method based on deep learning in a specific example of an embodiment of the present disclosure. Figure 5 As shown, the model training method may mainly include:

[0119] Step S501: Input the sample image into the target detection model to detect the subject (corresponding to the target body mentioned above) of the sample image to obtain the subject position information (corresponding to the target body position information mentioned above).

[0120] Step S502: Input the sample image, subject position information, multiple horizontal cropped images (for example, corresponding to the multiple first cropped images described above) and multiple vertical cropped images (for example, corresponding to the multiple second cropped images described above) into the backbone (corresponding to the shared backbone network described above) layer in the preset image cropping model to extract global features of each image.

[0121] Here, in an example, before inputting each horizontal cropped image and each vertical cropped image into a preset image cropping model, each horizontal cropped image and each vertical cropped image may be preprocessed based on actual needs, and the disclosed solution does not impose specific limitations on this.

[0122] Furthermore, the backbone layer may be specifically a residual network (Residual Network, ResNet), or may also be specifically a pre-trained network such as Mobile Network V2 (MobileNetV2), EfficientNetwork (EfficientNet), Swin Transformer, etc., and the present disclosure does not impose any specific restrictions on this.

[0123] Step S503: input the global features of the sample image and the global features of the horizontally cropped image into the feature alignment module (for example, which may correspond to the first feature alignment module described above) in the horizontal cropping branch of the preset image cropping model (for example, which may correspond to the first branch described above) to align the global features of the sample image with the global features of the horizontally cropped image; and input the global features of the sample image and the global features of the vertically cropped image into the feature alignment module (for example, which may correspond to the second feature alignment module described above) in the vertical cropping branch of the preset image cropping model (for example, which may correspond to the second branch described above) to align the global features of the sample image with the global features of the vertically cropped image.

[0124] Here, the feature alignment module in the horizontal cropping branch and the feature alignment module in the vertical cropping branch can both use RoI Align and RoD Align technologies for feature alignment. Further, other feature alignment technologies can also be used, and the present disclosure does not specifically limit the specific technology used in the feature alignment module.

[0125] Step S504: Input the features of the horizontally cropped image and the features of the sample image after feature alignment into the GAT in the horizontal cropping branch (corresponding to the first GAT described above) for attention processing, and obtain the features of the horizontally cropped image and the features of the sample image after attention processing; and input the features of the vertically cropped image and the features of the sample image after feature alignment into the GAT in the vertical cropping branch (corresponding to the second GAT described above) for attention processing, and obtain the features of the vertically cropped image and the features of the sample image after attention processing.

[0126] Step S505: Input the features of the horizontal cropped image after attention processing and the features of the sample image into the MLP network in the horizontal cropping branch (corresponding to the first MLP network described above) to obtain the predicted aesthetic score of each horizontal cropped image; and input the features of the vertical cropped image after attention processing and the features of the sample image into the MLP network in the vertical cropping branch (corresponding to the second MLP described above) to obtain the predicted aesthetic score of each vertical cropped image.

[0127] Step S506: Determine the loss value Loss (hereinafter referred to as the total loss value) of the total loss function (that is, corresponding to the target loss function described above), and use the total loss value to adjust the adjustable parameters in the preset image cropping model to obtain the trained preset image cropping model (corresponding to the target image cropping model described above).

[0128] Here, in one example, the total loss value may specifically include the overall loss value of the horizontal cropped image (which may be recorded as L 横―总体 ) and the overall loss value of the vertically cropped image (which can be recorded as L竖―总体 ) and; the overall loss value of the horizontal cropped image is the regression loss value of the horizontal cropped image (which can be recorded as L 横―回归 ) and the sorting loss value (which can be recorded as L 横―排序 ) and; the overall loss value of the vertical cropped image is the regression loss value of the vertical cropped image (which can be recorded as L 竖―回归 ) and the sorting loss value (which can be recorded as L 竖―排序 ) and.

[0129] Furthermore, in one example, the regression loss value of the horizontally cropped image can be obtained based on the predicted aesthetic score of the horizontally cropped image and its theoretical aesthetic score. Similarly, the regression loss value of the vertically cropped image can be obtained based on the predicted aesthetic score of the vertically cropped image and its theoretical aesthetic score.

[0130] Furthermore, based on the predicted aesthetic score of each horizontally cropped image, sorting is performed to obtain a sorting result of the horizontally cropped images (corresponding to the first actual sorting result described above), and based on the similarity between the sorting result of the horizontally cropped images and the theoretical sorting result (the sorting result obtained based on the theoretical score, that is, corresponding to the first theoretical sorting result described above), the sorting loss value of the horizontally cropped images is obtained; similarly, based on the predicted aesthetic score of each vertically cropped image, sorting is performed to obtain a sorting result of the vertically cropped images (corresponding to the second actual sorting result described above), and based on the similarity between the sorting result of the vertically cropped images and the theoretical sorting result (the sorting result obtained based on the theoretical score, that is, corresponding to the second theoretical sorting result described above), the sorting loss value of the vertically cropped images is obtained.

[0131] Furthermore, in a specific example, the sample data may be constructed in the following manner; specifically, Figure 6 As shown, the disclosed solution provides a method for automatically constructing training samples, and the specific steps include:

[0132] Step S601: Using a sliding window and a card threshold, multiple first cropping frames (such as horizontal cropping frames) are generated on the sample image to obtain multiple first cropped images (such as multiple horizontal cropped images). Similarly, multiple second cropping frames (such as vertical cropping frames) are generated on the sample image to obtain multiple second cropped images (such as multiple vertical cropped images).

[0133] Step S602: Obtain theoretical aesthetic scores of each first cropped image generated based on the sample image, and theoretical aesthetic scores of each second cropped image, so as to construct a sample data set.

[0134] For example, the Intersection Over Union (IOU) of each cropping frame (or each cropping image) may be calculated to serve as a theoretical aesthetic score of each cropping frame (or each cropping image).

[0135] Furthermore, in order to further improve the cropping effect, the disclosed solution can also introduce face truncation detection and / or text truncation detection, and set the theoretical aesthetic score of the cropped image with truncation to a negative score. In this way, the score ranking of such candidate boxes can be effectively reduced during model training, so that the model has the ability to avoid character truncation and text truncation.

[0136] Furthermore, in practical applications, the cropped images produced using the traditional cropping technology described above may have truncation of characters or text (such samples may also be called truncated samples). In this case, the cropped images without truncation (that is, high-quality samples) can be used as sample data, and a small number of truncated samples can be manually annotated as sample data, thereby effectively improving the quality of the sample data set.

[0137] In this way, the disclosed solution can use a large amount of sample data for model training composed of manually labeled thousands of truncated samples and automatically constructed tens of thousands of high-quality samples. In this way, rich and high-quality sample data can be obtained quickly, providing strong support for the trained model to have a better tailoring effect.

[0138] Furthermore, the disclosed solution also provides a complete image cropping system, specifically, Figure 7 As shown, the image cropping system of the disclosed solution is divided into two parts: offline and online:

[0139] 1. Offline part

[0140] The offline part includes the construction of sample data, training of the trimmed model, and packaging.

[0141] Regarding the construction of sample data, compared with the prior art, the disclosed solution can generate hundreds of thousands of sample data that can be used for model training with the same manpower cost.

[0142] For the designed target image cropping model, the disclosed solution can effectively avoid problems such as character truncation and text truncation, and can better adapt to multi-size cropping; in addition, the image cropping model used in the disclosed solution can also be a lightweight model with low deployment cost and can respond quickly online to process a large number of images produced in real time.

[0143] (II) Online part

[0144] First, the image to be cropped published by the target object is received, and the target image cropping model is requested in batches in real time to generate an optimal cropping frame on the image to be cropped; secondly, the image to be cropped containing the optimal cropping frame is post-processed, and face detection and text detection are connected at the same time to fine-tune the position of the cropping frame for possible character and text truncation; finally, a complete and untruncerated cropped image can be obtained, thereby further optimizing the cropping effect.

[0145] It should be pointed out that in post-processing, face detection and text detection are connected to determine whether there is any intersection between the cropping frame and the face detection frame and the text detection frame. If there is no intersection, a cropped image is produced; if there is an intersection, it indicates that the current cropping frame is truncated. At this time, it can be adjusted according to the truncation distance using a sliding window or zooming method to obtain a cropping frame without truncation of people or text. In this way, the cropping effect is further optimized.

[0146] The disclosed solution also provides an image processing method, such as Figure 8 As shown, including:

[0147] Step S801: Obtain an image to be cropped.

[0148] Step S802: at least input the image to be cropped into a target image cropping model to obtain a target cropped image.

[0149] Here, the target image cropping model is obtained based on the training method of the image cropping model described above.

[0150] In this way, by using the target image cropping model of the disclosed solution, a cropped image that meets the size requirements in a specific scene can be obtained, thereby effectively improving the user experience.

[0151] Further, in an example, the above step of at least inputting the image to be cropped into the target image cropping model to obtain the target cropped image (for example, the above step S802) may specifically include:

[0152] Step S802-1: Obtaining the target object position information of the image to be cropped.

[0153] Step S802-2: input the image to be cropped and the target position information into the target image cropping model to obtain the target cropped image.

[0154] The above-mentioned obtaining the target position information of the image to be cropped may specifically include: inputting the image to be cropped into a target detection model to obtain the target position information of the image to be cropped. Here, the target detection model is used to detect the position of the target in the input image.

[0155] That is to say, in this example, the target detection model can be used to detect the position of the target object in the image to be cropped to obtain the specific position information of each target object in the image to be cropped (that is, corresponding to the target object position information described above). In this way, the model can extract effective features based on the target object position information for use in evaluating the cropped image, thereby providing strong support for improving the cropping accuracy of the model.

[0156] The disclosed solution also provides a training device for an image cropping model, such as Fig. 9 As shown, including:

[0157] A first acquisition unit 901 is used to acquire sample data, wherein the sample data at least includes: a sample image, a first cropped image obtained by cropping the sample image in a first manner, and a second cropped image obtained by cropping the sample image in a second manner;

[0158] The training unit 902 is used to determine a target loss function, wherein the target loss function is used at least to: constrain the difference between a first predicted attribute value of the first cropped image and a first theoretical attribute value of the first cropped image, and to constrain the difference between a second predicted attribute value of the second cropped image and a second theoretical attribute value of the second cropped image; and to perform model training on a preset image cropping model using at least the sample data and the target loss function to obtain a target image cropping model, wherein the first predicted attribute value and the second predicted attribute value are obtained using the preset image cropping model.

[0159] In a specific example of the disclosed solution, the objective loss function is also used for at least one of the following:

[0160] In the case that the sample data includes two or more first cropped images, constraining the similarity between the first actual sorting result and the first theoretical sorting result; wherein the first actual sorting result is the result of sorting all the first cropped images according to the numerical values ​​of the first predicted attribute values ​​of each first cropped image, and the first theoretical sorting result is the result of sorting all the first cropped images according to the numerical values ​​of the first theoretical attribute values ​​of each first cropped image;

[0161] When the sample data includes two or more second cropped images, the similarity between the second actual sorting result and the second theoretical sorting result is constrained; wherein the second actual sorting result is the result of sorting all the second cropped images according to the numerical values ​​of the second predicted attribute values ​​of each second cropped image, and the second theoretical sorting result is the result of sorting all the second cropped images according to the numerical values ​​of the second theoretical attribute values ​​of each second cropped image.

[0162] In a specific example of the disclosed solution, the training unit is specifically used to:

[0163] Acquire target body position information of the sample image;

[0164] Inputting the sample image, the target position information, the first cropped image and the second cropped image into the preset image cropping model to obtain the output first predicted attribute value and the second predicted attribute value;

[0165] Based on the first predicted attribute value and the second predicted attribute value, obtaining a loss value of the target loss function;

[0166] Based on the loss value, the adjustable parameters in the preset image cropping model are adjusted to obtain a target image cropping model.

[0167] In a specific example of the disclosed solution, the training unit is specifically used to:

[0168] The sample image is input into the target detection model to obtain the output position information of the target body.

[0169] In a specific example of the disclosed solution, the preset image cropping model includes at least a first branch and a second branch;

[0170] The first branch is at least used to process the global features of the sample image and the global features of the first cropped image to obtain the first predicted attribute value;

[0171] The second branch is at least used to process the global features of the sample image and the global features of the second cropped image to obtain the second predicted attribute value.

[0172] In a specific example of the disclosed solution, the first branch at least includes a first feature alignment module, a first graph attention network GAT and a first multi-layer perceptron MLP; wherein the first feature alignment module is used to perform feature alignment on the global features of the sample image and the global features of the first cropped image; the first GAT is used to perform attention processing on the sample image and the first cropped image after feature alignment; the first MLP is used to perform feature matching on the sample image and the first cropped image after attention processing to obtain the first predicted attribute value;

[0173] and / or,

[0174] The second branch at least includes a second feature alignment module, a second graph attention network GAT and a second multi-layer perceptron MLP; wherein the second feature alignment module is used to perform feature alignment on the global features of the sample image and the global features of the second cropped image; the second GAT is used to perform attention processing on the sample image and the second cropped image after feature alignment; and the second MLP is used to perform feature matching on the sample image and the second cropped image after attention processing to obtain the second predicted attribute value.

[0175] In a specific example of the disclosed solution, the preset image cropping model further includes a shared backbone network, and the output of the shared backbone network is used as the input of the first branch and the second branch; the shared backbone network is used to:

[0176] Based on the input sample image and the target position information of the sample image, a global feature of the sample image is obtained;

[0177] Obtaining a global feature of the first cropped image based on the input first cropped image; and

[0178] A global feature of the second cropped image is obtained based on the input second cropped image.

[0179] In a specific example of the disclosed solution, the first cropped image is obtained by horizontally cropping the sample image;

[0180] and / or,

[0181] The second cropped image is obtained by vertically cropping the sample image.

[0182] For the description of the specific functions and examples of each unit of the device in the embodiment of the present disclosure, reference can be made to the relevant description of the corresponding steps in the above method embodiment, which will not be repeated here.

[0183] The disclosed solution also provides an image processing device, such as Fig.10 As shown, including:

[0184] The second acquisition unit 1001 is used to acquire the image to be cropped;

[0185] The model inference unit 1002 is used to input at least the image to be cropped into a target image cropping model to obtain a target cropped image, wherein the target image cropping model is obtained based on the training method of the above image cropping model.

[0186] In a specific example of the disclosed solution, the model reasoning unit is specifically used to:

[0187] Acquire the target position information of the image to be cropped;

[0188] The image to be cropped and the target position information are input into the target image cropping model to obtain the target cropped image.

[0189] For the description of the specific functions and examples of each unit of the device in the embodiment of the present disclosure, reference can be made to the relevant description of the corresponding steps in the above method embodiment, which will not be repeated here.

[0190] In the technical solution disclosed herein, the acquisition, storage and application of user personal information involved are in compliance with the provisions of relevant laws and regulations and do not violate public order and good morals.

[0191] According to an embodiment of the present disclosure, the present disclosure also provides an electronic device, a readable storage medium and a computer program product.

[0192] Fig.11 A schematic block diagram of an example electronic device 1100 that can be used to implement an embodiment of the present disclosure is shown. The electronic device is intended to represent various forms of digital computers, such as laptop computers, desktop computers, workstations, personal digital assistants, servers, blade servers, mainframe computers, and other suitable computers. The electronic device can also represent various forms of mobile devices, such as personal digital assistants, cellular phones, smart phones, wearable devices, and other similar computing devices. The components shown herein, their connections and relationships, and their functions are merely examples and are not intended to limit the implementation of the present disclosure described and / or required herein.

[0193] like Fig.11 As shown, the device 1100 includes a computing unit 1101, which can perform various appropriate actions and processes according to a computer program stored in a read-only memory (ROM) 1102 or a computer program loaded from a storage unit 1108 into a random access memory (RAM) 1103. In the RAM 1103, various programs and data required for the operation of the device 1100 can also be stored. The computing unit 1101, the ROM 1102, and the RAM 1103 are connected to each other via a bus 1104. An input / output (I / O) interface 1105 is also connected to the bus 1104.

[0194] A number of components in the device 1100 are connected to the I / O interface 1105, including: an input unit 1106, such as a keyboard, a mouse, etc.; an output unit 1107, such as various types of displays, speakers, etc.; a storage unit 1108, such as a disk, an optical disk, etc.; and a communication unit 1109, such as a network card, a modem, a wireless communication transceiver, etc. The communication unit 1109 allows the device 1100 to exchange information / data with other devices through a computer network such as the Internet and / or various telecommunication networks.

[0195] The computing unit 1101 may be a variety of general and / or special processing components with processing and computing capabilities. Some examples of the computing unit 1101 include, but are not limited to, a central processing unit (CPU), a graphics processing unit (GPU), various dedicated artificial intelligence (AI) computing chips, various computing units running machine learning model algorithms, digital signal processors (DSPs), and any appropriate processors, controllers, microcontrollers, etc. The computing unit 1101 performs the various methods and processes described above, such as the training method or image processing method of the image cropping model. For example, in some embodiments, the training method or image processing method of the image cropping model may be implemented as a computer software program, which is tangibly included in a machine-readable medium, such as a storage unit 1108. In some embodiments, part or all of the computer program may be loaded and / or installed on the device 1100 via the ROM 1102 and / or the communication unit 1109. When the computer program is loaded into the RAM 1103 and executed by the computing unit 1101, one or more steps of the training method or image processing method of the image cropping model described above may be performed. Alternatively, in other embodiments, the computing unit 1101 may be configured to execute the training method or the image processing method of the image cropping model in any other appropriate manner (for example, by means of firmware).

[0196] Various implementations of the systems and techniques described above herein can be implemented in digital electronic circuit systems, integrated circuit systems, field programmable gate arrays (FPGAs), application specific integrated circuits (ASICs), application specific standard products (ASSPs), systems on chips (SOCs), load programmable logic devices (CPLDs), computer hardware, firmware, software, and / or combinations thereof. These various implementations can include: being implemented in one or more computer programs that can be executed and / or interpreted on a programmable system including at least one programmable processor, which can be a special purpose or general purpose programmable processor that can receive data and instructions from a storage system, at least one input device, and at least one output device, and transmit data and instructions to the storage system, the at least one input device, and the at least one output device.

[0197] The program code for implementing the method of the present disclosure may be written in any combination of one or more programming languages. These program codes may be provided to a processor or controller of a general-purpose computer, a special-purpose computer, or other programmable data processing device, so that the program code, when executed by the processor or controller, enables the functions / operations specified in the flow chart and / or block diagram to be implemented. The program code may be executed entirely on the machine, partially on the machine, partially on the machine and partially on a remote machine as a stand-alone software package, or entirely on a remote machine or server.

[0198] In the context of the present disclosure, a machine-readable medium may be a tangible medium that may contain or store a program for use by or in conjunction with an instruction execution system, device, or equipment. A machine-readable medium may be a machine-readable signal medium or a machine-readable storage medium. A machine-readable medium may include, but is not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, device, or equipment, or any suitable combination of the foregoing. A more specific example of a machine-readable storage medium may include an electrical connection based on one or more lines, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing.

[0199] To provide interaction with a user, the systems and techniques described herein can be implemented on a computer having: a display device (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor) for displaying information to the user; and a keyboard and pointing device (e.g., a mouse or trackball) through which the user can provide input to the computer. Other types of devices can also be used to provide interaction with the user; for example, the feedback provided to the user can be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user can be received in any form (including acoustic input, voice input, or tactile input).

[0200] The systems and techniques described herein may be implemented in a computing system that includes back-end components (e.g., as a data server), or a computing system that includes middleware components (e.g., an application server), or a computing system that includes front-end components (e.g., a user computer with a graphical user interface or a web browser through which a user can interact with implementations of the systems and techniques described herein), or a computing system that includes any combination of such back-end components, middleware components, or front-end components. The components of the system may be interconnected by any form or medium of digital data communication (e.g., a communication network). Examples of communication networks include: a local area network (LAN), a wide area network (WAN), and the Internet.

[0201] A computer system may include a client and a server. The client and the server are generally remote from each other and usually interact through a communication network. The relationship of client and server is generated by computer programs running on respective computers and having a client-server relationship with each other. The server may be a cloud server, a server of a distributed system, or a server combined with a blockchain.

[0202] It should be understood that the various forms of processes shown above can be used to reorder, add or delete steps. For example, the steps recorded in this disclosure can be executed in parallel, sequentially or in different orders, as long as the desired results of the technical solutions disclosed in this disclosure can be achieved, and this document does not limit this.

[0203] The above specific implementations do not constitute a limitation on the protection scope of the present disclosure. It should be understood by those skilled in the art that various modifications, combinations, sub-combinations and substitutions can be made according to design requirements and other factors. Any modification, equivalent substitution and improvement made within the principles of the present disclosure shall be included in the protection scope of the present disclosure.

Claims

1. A training method for an image cropping model, comprising: Acquire sample data, wherein the sample data at least includes: a sample image, a first cropped image obtained by cropping the sample image in a first manner, and a second cropped image obtained by cropping the sample image in a second manner; Determining a target loss function, wherein the target loss function is used at least to: constrain a difference between a first predicted attribute value of the first cropped image and a first theoretical attribute value of the first cropped image, and to constrain a difference between a second predicted attribute value of the second cropped image and a second theoretical attribute value of the second cropped image; At least the sample data and the target loss function are used to perform model training on a preset image cropping model to obtain a target image cropping model; wherein the first predicted attribute value and the second predicted attribute value are obtained using the preset image cropping model.

2. The method according to claim 1, wherein: The objective loss function is also used for at least one of the following: In the case that the sample data includes two or more first cropped images, constraining the similarity between the first actual sorting result and the first theoretical sorting result; wherein the first actual sorting result is the result of sorting all the first cropped images according to the numerical values ​​of the first predicted attribute values ​​of each first cropped image, and the first theoretical sorting result is the result of sorting all the first cropped images according to the numerical values ​​of the first theoretical attribute values ​​of each first cropped image; When the sample data includes two or more second cropped images, the similarity between the second actual sorting result and the second theoretical sorting result is constrained; wherein the second actual sorting result is the result of sorting all the second cropped images according to the numerical values ​​of the second predicted attribute values ​​of each second cropped image, and the second theoretical sorting result is the result of sorting all the second cropped images according to the numerical values ​​of the second theoretical attribute values ​​of each second cropped image.

3. The method according to claim 1 or 2, wherein: The step of performing model training on a preset image cropping model by at least using the sample data and the target loss function to obtain a target image cropping model includes: Acquire target body position information of the sample image; Inputting the sample image, the target position information, the first cropped image and the second cropped image into the preset image cropping model to obtain the output first predicted attribute value and the second predicted attribute value; Based on the first predicted attribute value and the second predicted attribute value, obtaining a loss value of the target loss function; Based on the loss value, the adjustable parameters in the preset image cropping model are adjusted to obtain a target image cropping model.

4. The method according to claim 3, wherein: The step of obtaining the target position information of the sample image includes: The sample image is input into the target detection model to obtain the output position information of the target body.

5. The method according to claim 1, wherein: The preset image cropping model includes at least a first branch and a second branch; The first branch is at least used to process the global features of the sample image and the global features of the first cropped image to obtain the first predicted attribute value; The second branch is at least used to process the global features of the sample image and the global features of the second cropped image to obtain the second predicted attribute value.

6. The method according to claim 5, wherein: The first branch at least includes a first feature alignment module, a first graph attention network GAT and a first multi-layer perceptron MLP; wherein the first feature alignment module is used to perform feature alignment on the global features of the sample image and the global features of the first cropped image; the first GAT is used to perform attention processing on the sample image and the first cropped image after feature alignment; the first MLP is used to perform feature matching on the sample image and the first cropped image after attention processing to obtain the first predicted attribute value; and / or, The second branch at least includes a second feature alignment module, a second graph attention network GAT and a second multi-layer perceptron MLP; wherein the second feature alignment module is used to perform feature alignment on the global features of the sample image and the global features of the second cropped image; the second GAT is used to perform attention processing on the sample image and the second cropped image after feature alignment; and the second MLP is used to perform feature matching on the sample image and the second cropped image after attention processing to obtain the second predicted attribute value.

7. The method according to claim 5 or 6, wherein: The preset image cropping model further includes a shared backbone network, the output of which is used as the input of the first branch and the second branch; the shared backbone network is used to: Based on the input sample image and the target position information of the sample image, a global feature of the sample image is obtained; Obtaining a global feature of the first cropped image based on the input first cropped image; and A global feature of the second cropped image is obtained based on the input second cropped image.

8. The method according to claim 1 or 2, wherein: The first cropped image is obtained by horizontally cropping the sample image; and / or, The second cropped image is obtained by vertically cropping the sample image.

9. An image processing method, comprising: Get the image to be cropped; At least the image to be cropped is input into a target image cropping model to obtain a target cropped image, wherein the target image cropping model is obtained based on any one of the training methods of claims 1 to 8 above.

10. The method according to claim 9, wherein: The step of at least inputting the image to be cropped into a target image cropping model to obtain a target cropped image comprises: Acquire the target position information of the image to be cropped; The image to be cropped and the target position information are input into the target image cropping model to obtain the target cropped image.

11. A training device for an image cropping model, comprising: A first acquisition unit is used to acquire sample data, wherein the sample data at least includes: a sample image, a first cropped image obtained by cropping the sample image in a first manner, and a second cropped image obtained by cropping the sample image in a second manner; A training unit is used to determine a target loss function, wherein the target loss function is used at least to: constrain the difference between a first predicted attribute value of the first cropped image and a first theoretical attribute value of the first cropped image, and to constrain the difference between a second predicted attribute value of the second cropped image and a second theoretical attribute value of the second cropped image; and is also used to perform model training on a preset image cropping model using at least the sample data and the target loss function to obtain a target image cropping model; wherein the first predicted attribute value and the second predicted attribute value are obtained using the preset image cropping model.

12. The device according to claim 11, wherein The objective loss function is also used for at least one of the following: In the case that the sample data includes two or more first cropped images, constraining the similarity between the first actual sorting result and the first theoretical sorting result; wherein the first actual sorting result is the result of sorting all the first cropped images according to the numerical values ​​of the first predicted attribute values ​​of each first cropped image, and the first theoretical sorting result is the result of sorting all the first cropped images according to the numerical values ​​of the first theoretical attribute values ​​of each first cropped image; When the sample data includes two or more second cropped images, the similarity between the second actual sorting result and the second theoretical sorting result is constrained; wherein the second actual sorting result is the result of sorting all the second cropped images according to the numerical values ​​of the second predicted attribute values ​​of each second cropped image, and the second theoretical sorting result is the result of sorting all the second cropped images according to the numerical values ​​of the second theoretical attribute values ​​of each second cropped image.

13. The device according to claim 11 or 12, wherein: The training unit is specifically used for: Acquire target body position information of the sample image; Inputting the sample image, the target position information, the first cropped image and the second cropped image into the preset image cropping model to obtain the output first predicted attribute value and the second predicted attribute value; Based on the first predicted attribute value and the second predicted attribute value, obtaining a loss value of the target loss function; Based on the loss value, the adjustable parameters in the preset image cropping model are adjusted to obtain a target image cropping model.

14. The device according to claim 13, wherein: The training unit is specifically used for: The sample image is input into the target detection model to obtain the output position information of the target body.

15. The device according to claim 11, wherein The preset image cropping model includes at least a first branch and a second branch; The first branch is at least used to process the global features of the sample image and the global features of the first cropped image to obtain the first predicted attribute value; The second branch is at least used to process the global features of the sample image and the global features of the second cropped image to obtain the second predicted attribute value.

16. The device according to claim 15, wherein: The first branch at least includes a first feature alignment module, a first graph attention network GAT and a first multi-layer perceptron MLP; wherein the first feature alignment module is used to perform feature alignment on the global features of the sample image and the global features of the first cropped image; the first GAT is used to perform attention processing on the sample image and the first cropped image after feature alignment; the first MLP is used to perform feature matching on the sample image and the first cropped image after attention processing to obtain the first predicted attribute value; and / or, The second branch at least includes a second feature alignment module, a second graph attention network GAT and a second multi-layer perceptron MLP; wherein the second feature alignment module is used to perform feature alignment on the global features of the sample image and the global features of the second cropped image; the second GAT is used to perform attention processing on the sample image and the second cropped image after feature alignment; and the second MLP is used to perform feature matching on the sample image and the second cropped image after attention processing to obtain the second predicted attribute value.

17. The device according to claim 15 or 16, wherein: The preset image cropping model further includes a shared backbone network, the output of which is used as the input of the first branch and the second branch; the shared backbone network is used to: Based on the input sample image and the target position information of the sample image, a global feature of the sample image is obtained; Obtaining a global feature of the first cropped image based on the input first cropped image; and A global feature of the second cropped image is obtained based on the input second cropped image.

18. The device according to claim 11 or 12, wherein: The first cropped image is obtained by horizontally cropping the sample image; and / or, The second cropped image is obtained by vertically cropping the sample image.

19. An image processing device, comprising: A second acquisition unit, used to acquire the image to be cropped; A model inference unit is used to input at least the image to be cropped into a target image cropping model to obtain a target cropped image, wherein the target image cropping model is obtained based on the training method of the image cropping model of any one of claims 1 to 8 above.

20. The device according to claim 19, wherein The model reasoning unit is specifically used for: Acquire the target position information of the image to be cropped; The image to be cropped and the target position information are input into the target image cropping model to obtain the target cropped image.

21. An electronic device comprising: at least one processor; as well as a memory communicatively connected to the at least one processor; wherein, The memory stores instructions that can be executed by the at least one processor, and the instructions are executed by the at least one processor to enable the at least one processor to perform the method according to any one of claims 1 to 10.

22. A non-transitory computer-readable storage medium storing computer instructions, wherein: The computer instructions are used to cause the computer to execute the method according to any one of claims 1-10.

23. A computer program product comprising a computer program, which, when executed by a processor, implements the method according to any one of claims 1 to 10.

Images