Image cropping method and device based on semantic content

Abstract

The present invention provides a method and device for image cropping based on semantic content, the method comprising: firstly, performing saliency calculation on an image to be cropped according to a preset algorithm, generating a corresponding saliency map according to the calculation result, then selecting an area with the largest mean of saliency values ​​from the saliency map as a candidate cropping area, and then calculating metadata of semantic information of the image to be cropped, adjusting the cropping boundary of the candidate cropping area based on the metadata of the semantic information, so that the semantic information in the candidate cropping area is complete, and finally cropping the candidate cropping area of ​​the image to be cropped according to the adjusted cropping boundary. When cropping an image, the scheme of the present invention crops the image based on image saliency, so that the area with the highest saliency in the image can be effectively retained, and after the image is saliency cropped, the image is cropped by fully combining the semantic information in the image, so as to ensure that the semantic content in the cropped image is complete.

Description

Technical Field

[0001] The present invention relates to the technical field of image processing, and in particular to a method and device for image cropping based on semantic content. Background Art

[0002] With the continuous development of computer technology, more and more people have begun to rely on mobile terminals (such as mobile phones), computers and other terminal devices to obtain information, work, study, entertainment, etc., but different types of terminal devices often have fixed and different sizes. Therefore, in order to ensure that the image displayed on the terminal device display screen better fits the screen size, when the image data is displayed, it is often necessary to crop the image accordingly based on the display size of the terminal. The current mainstream cropping method is cropping based on saliency, that is, retaining the most significant area in the original image as the cropping target.

[0003] However, existing image cropping methods cannot well guarantee the integrity of image semantic description units (such as text, dynamic target areas, etc.). Summary of the invention

[0004] In view of the above problems, the present invention is proposed to provide a semantic content-based image cropping method and device that overcomes the above problems or at least partially solves the above problems.

[0005] According to one aspect of the present invention, a method for image cropping based on semantic content is provided, comprising:

[0006] Perform saliency calculation on the image to be cropped according to a preset algorithm, and generate a corresponding saliency map based on the calculation result;

[0007] Selecting a region with the largest mean value of saliency values ​​from the saliency map as a candidate cropping region;

[0008] Calculating metadata of the semantic information of the image to be cropped, and adjusting the cropping boundary of the candidate cropping area based on the metadata of the semantic information to make the semantic information in the candidate cropping area complete;

[0009] The candidate cropping area of ​​the image to be cropped is cropped according to the adjusted cropping boundary.

[0010] Optionally, performing saliency calculation on the image to be cropped according to a preset algorithm includes:

[0011] Preprocessing the image to be cropped so that the image to be cropped meets the cropping requirements;

[0012] The saliency of the preprocessed image to be cropped is calculated according to a preset algorithm.

[0013] Optionally, preprocessing the image to be cropped includes:

[0014] Converting the image format of the image to be cropped; and / or

[0015] The size of the image to be cropped is scaled proportionally.

[0016] Optionally, calculating metadata of semantic information of the image to be cropped includes:

[0017] A preset deep neural network model is used to calculate metadata of semantic information in the image to be cropped.

[0018] Optionally, the metadata of the semantic information in the image to be cropped includes at least one of the following:

[0019] Dynamic or static image target data, face data, text data.

[0020] Optionally, the dynamic or static image target data includes: pedestrians, scenery, buildings, and vehicles.

[0021] Optionally, adjusting the cropping boundary of the candidate cropping area based on the metadata of the semantic information includes:

[0022] Set the trimming priority for the metadata of each semantic information;

[0023] The cropping boundary of the candidate cropping area is adjusted according to the cropping priority of the metadata of the semantic information.

[0024] Optionally, set a clipping priority for the metadata of each semantic information, including:

[0025] Set the face data to have the highest priority, dynamic or static image target data to have the second highest priority, and text data to have the lowest priority.

[0026] According to another aspect of the present invention, there is also provided an image cropping device based on semantic content, comprising:

[0027] A calculation module, adapted to perform saliency calculation on the cropped image according to a preset algorithm, and generate a corresponding saliency map according to the calculation result;

[0028] A selection module, adapted to select a region with the largest mean of saliency values ​​from the saliency map as a candidate cropping region;

[0029] an adjustment module, adapted to calculate metadata of the semantic information of the image to be cropped, and adjust the cropping boundary of the candidate cropping area based on the metadata of the semantic information to make the semantic information in the candidate cropping area complete;

[0030] The cropping module is adapted to crop the candidate cropping area of ​​the image to be cropped according to the adjusted cropping boundary.

[0031] Optionally, the calculation module further includes:

[0032] A preprocessing unit, adapted to preprocess the image to be cropped so that the image to be cropped meets the cropping requirements;

[0033] The calculation unit is adapted to perform saliency calculation on the preprocessed image to be cropped according to a preset algorithm.

[0034] Optionally, the pre-processing unit is further adapted to:

[0035] Converting the image format of the image to be cropped; and / or

[0036] The size of the image to be cropped is scaled proportionally.

[0037] Optionally, the adjustment module is further adapted to:

[0038] A preset deep neural network model is used to calculate metadata of semantic information in the image to be cropped.

[0039] Optionally, the metadata of the semantic information in the image to be cropped includes at least one of the following:

[0040] Dynamic or static image target data, face data, text data.

[0041] Optionally, the dynamic or static image target data includes: pedestrians, scenery, buildings, and vehicles.

[0042] Optionally, the adjustment module is further adapted to:

[0043] Set the trimming priority for the metadata of each semantic information;

[0044] The cropping boundary of the candidate cropping area is adjusted according to the cropping priority of the metadata of the semantic information.

[0045] Optionally, the adjustment module is further adapted to:

[0046] Set the face data to have the highest priority, dynamic or static image target data to have the second highest priority, and text data to have the lowest priority.

[0047] According to another aspect of the present invention, a computer storage medium is provided, wherein the computer storage medium stores computer program code, and when the computer program code is executed on a computing device, the computing device executes the semantic content-based image cropping method in any of the above embodiments.

[0048] According to another aspect of the present invention, a computing device is provided, comprising: a processor; a memory storing computer program code; when the computer program code is executed by the processor, the computing device executes the semantic content-based image cropping method in any of the above embodiments.

[0049] In an embodiment of the present invention, first, the saliency calculation is performed on the image to be cropped according to a preset algorithm, and a corresponding saliency map is generated according to the calculation result. Then, the area with the largest mean of saliency values ​​is selected from the saliency map as a candidate cropping area, and then the metadata of the semantic information of the image to be cropped is calculated. The cropping boundary of the candidate cropping area is adjusted based on the metadata of the semantic information to make the semantic information in the candidate cropping area complete, and finally, the candidate cropping area of ​​the image to be cropped is cropped according to the adjusted cropping boundary. Therefore, when the scheme of the present invention crops an image, it can effectively retain the area with the highest saliency in the image by cropping the image based on the image saliency, and after the image is cropped according to the saliency, the scheme also fully combines the semantic information in the image to crop the image, ensuring that the semantic content in the cropped image is complete, and effectively avoiding the damage to the integrity of the image semantic information, that is, the scheme retains the saliency area of ​​the original image and ensures the cropping integrity of the basic semantic unit, thereby greatly improving the accuracy of image cropping.

[0050] The above description is only an overview of the technical solution of the present invention. In order to more clearly understand the technical means of the present invention, it can be implemented according to the contents of the specification. In order to make the above and other purposes, features and advantages of the present invention more obvious and easy to understand, the specific implementation methods of the present invention are listed below.

[0051] Based on the following detailed description of specific embodiments of the present invention in conjunction with the accompanying drawings, those skilled in the art will become more aware of the above and other objects, advantages and features of the present invention. BRIEF DESCRIPTION OF THE DRAWINGS

[0052] Various other advantages and benefits will become apparent to those of ordinary skill in the art by reading the detailed description of the preferred embodiments below. The accompanying drawings are only for the purpose of illustrating the preferred embodiments and are not to be considered as limiting the present invention. Moreover, the same reference symbols are used throughout the accompanying drawings to represent the same components. In the accompanying drawings:

[0053] Figure 1 A schematic diagram of the process of a semantic content-based image cropping method according to an embodiment of the present invention is shown;

[0054] Figure 2 A schematic diagram showing a flow chart of a semantic content-based image cropping method according to another embodiment of the present invention;

[0055] Figure 3 A schematic diagram showing the structure of a semantic content-based image cropping device according to an embodiment of the present invention is shown; and

[0056] Figure 4 A schematic structural diagram of an image cropping device based on semantic content according to another embodiment of the present invention is shown. DETAILED DESCRIPTION

[0057] The exemplary embodiments of the present disclosure will be described in more detail below with reference to the accompanying drawings. Although the exemplary embodiments of the present disclosure are shown in the accompanying drawings, it should be understood that the present disclosure can be implemented in various forms and should not be limited by the embodiments set forth herein. On the contrary, these embodiments are provided to enable a more thorough understanding of the present disclosure and to fully convey the scope of the present disclosure to those skilled in the art.

[0058] To solve the above technical problem, an embodiment of the present invention provides an image cropping method based on semantic content. Figure 1 FIG. 4 is a flow chart showing a method for image cropping based on semantic content according to an embodiment of the present invention. Figure 1 The method at least includes steps S102 to S108.

[0059] Step S102: performing saliency calculation on the image to be cropped according to a preset algorithm, and generating a corresponding saliency map according to the calculation result.

[0060] Step S104: Selecting a region with the largest mean saliency value from the saliency map as a candidate cropping region.

[0061] Step S106 , calculating metadata of the semantic information of the image to be cropped, and adjusting the cropping boundary of the candidate cropping area based on the metadata of the semantic information to make the semantic information in the candidate cropping area complete.

[0062] Step S108 , cropping the candidate cropping area of ​​the image to be cropped according to the adjusted cropping boundary.

[0063] When the embodiment of the present invention crops an image, it crops the image based on image saliency, thereby effectively retaining the most significant area in the image. After the image is saliency cropped, the present solution also fully combines the semantic information in the image to crop the image. The semantic content in the cropped image is complete, and the damage to the integrity of the image semantic information is effectively avoided. That is, the present solution retains the salient area of ​​the original image and ensures the cropping integrity of the basic semantic unit, thereby greatly improving the accuracy of image cropping.

[0064] In one embodiment of the present invention, before calculating the saliency of the image to be cropped according to the preset algorithm, the image to be cropped may be preprocessed to make the image to be cropped meet the subsequent cropping requirements, and then the saliency of the preprocessed image to be cropped is calculated according to the preset algorithm. The preset algorithm in this embodiment refers to an image saliency detection algorithm.

[0065] The present invention introduces two methods for preprocessing the image to be cropped, which are introduced below respectively.

[0066] Method 1

[0067] The image format of the image to be cropped is converted so that the format of the image to be cropped meets the algorithm requirements of image saliency calculation.

[0068] At present, there are many standard image formats, such as JPG, PNG, and RGBA four-channel formats. Usually, most images are grayscale images in RGB three-channel format or single-channel format. For example, if the image to be cropped is a grayscale image, the grayscale image can be directly converted into an RGB three-channel image during image preprocessing. For another example, if the image to be cropped is a four-channel RGBA image, the four-channel RGBA image can be converted into a three-channel RGB image during image preprocessing. Specifically, first calculate the alpha channel in the RGBA image and obtain the mask of the alpha channel, then calculate the product of the mask and the other three-channel RGB and superimpose it with a pure white background to obtain a distortion-free three-channel color image.

[0069] Method 2

[0070] The size of the image to be cropped is scaled proportionally to adjust the size of the image to be cropped to a size suitable for image saliency calculation.

[0071] For example, if the size of the input image to be cropped is too large, the size of the image to be cropped may be proportionally compressed to improve the image processing efficiency. For another example, if the size of the image to be cropped is too small to meet the saliency calculation requirements of the preset algorithm, the size of the image to be cropped may be proportionally enlarged.

[0072] Of course, when preprocessing the image to be cropped, the above two methods can be combined to preprocess the image to be cropped. In addition, other methods of preprocessing the image to be cropped can also be included, which is not specifically limited in the embodiment of the present invention.

[0073] Referring to the above step S104, in order to more clearly reflect the selection process of the candidate cropping area of ​​the present invention, a specific embodiment is now introduced.

[0074] First, the saliency values ​​of each row and column in the image saliency map are accumulated.

[0075] For example, the saliency values ​​of each column in the image saliency map may be accumulated according to Formula 1, where s in Formula 1 is j,i represents the significance value of column i and row j in the significance map, C i is the cumulative sum of the significance values ​​of the i column. Similarly, the sum of the significance values ​​of each column can also be calculated based on a calculation method similar to Formula 1.

[0076]

[0077] Then, a region with the largest mean of saliency values ​​is selected from one side of the saliency map by windowing, and the selected region is used as a candidate cropping region, where the final candidate cropping region is the maximum value of the window region. For example, the maximum value of the window region can be calculated by referring to Formula 2, where C j is the cumulative saliency of row j, and k is the number of columns in the candidate cropping region.

[0078]

[0079] Referring to step S106 above, in one embodiment of the present invention, when calculating the metadata of the semantic information of the image to be cropped, a preset deep neural network model may be specifically used to calculate the metadata of the semantic information in the image to be cropped.

[0080] In this embodiment, the metadata of the semantic information in the image to be cropped includes at least one of dynamic or static image target data, face data, text data, etc. The dynamic or static image target data may include dynamic or static pedestrians, plants, rivers, etc., buildings, vehicles, etc. in the image. The above examples of image target data are merely illustrative, and the embodiments of the present invention do not specifically limit this.

[0081] In an embodiment of the present invention, if the calculated metadata of the semantic information in the image to be cropped is dynamic or static image target data, the preset deep neural network model can use a preset target detection model. The target detection model can effectively calculate and identify where the target exists in the image to be cropped. For example, if the image to be cropped is a landscape image, targets such as pedestrians, scenery, and buildings that appear in the image to be cropped can be effectively identified.

[0082] If the calculated metadata of the semantic information in the image to be cropped is face data, the preset deep neural network model can use a preset face detection model. The face detection model can effectively calculate and identify where the face appears in the image to be cropped, and then in the subsequent image cropping process, it can be ensured that the face in the image is a complete face, avoiding the situation where the face actually contained in the image to be cropped is a complete face, but the image after cropping contains only part of the face.

[0083] If the calculated metadata of the semantic information in the image to be cropped is text data, the preset deep neural network model can use a preset text recognition detection model. The text recognition detection model can effectively calculate and identify where the text data exists in the image to be cropped, and then in the subsequent image cropping process, it can be ensured that the text in the image is incomplete text, and the text in the image after cropping will not be half a text.

[0084] Continuing to refer to step S106 above, in one embodiment of the present invention, when adjusting the cropping boundary of the candidate cropping area based on the metadata of the semantic information of the image to be cropped, if the metadata of the semantic information of the image to be cropped includes multiple items, in order to improve the efficiency of image cropping, the image to be cropped can also be adjusted in sequence according to the specified order set for the metadata of each item of semantic information. For example, first, set the cropping priority for the metadata of each item of semantic information. Then, adjust the cropping boundary of the candidate cropping area according to the cropping priority of the metadata of the image semantic information that has been set.

[0085] In this embodiment, combined with the above embodiment, if the metadata of the semantic information of the image to be cropped contains dynamic or static image target data, face data, and text data. Then, the order of cropping priorities set for the metadata of each semantic information can be to set the face data to have the highest priority, the dynamic or static image target data to have the second highest priority, and the text data to have the lowest priority. Of course, other priority orders can also be set for the metadata, and the embodiment of the present invention does not specifically limit this.

[0086] The embodiment of the present invention also provides another image cropping method based on semantic content. Figure 2 FIG. 4 is a flow chart showing a method for image cropping based on semantic content according to an embodiment of the present invention. Figure 2 The method at least includes steps S202 to S212.

[0087] Step S202, preprocessing the image to be cropped to make the image to be cropped meet the cropping requirements.

[0088] Step S204, performing saliency calculation on the preprocessed image to be cropped according to a preset algorithm, and generating a corresponding saliency map according to the calculation result.

[0089] Step S206: Selecting a region with the largest mean saliency value from the saliency map as a candidate cropping region.

[0090] Step S208, calculating metadata of the semantic information of the image to be cropped, and setting a cropping priority for the metadata of each item of semantic information.

[0091] Step S210 , adjusting the cropping boundary of the candidate cropping area according to the cropping priority of the metadata of the semantic information, so that the semantic information in the candidate cropping area is complete.

[0092] In this embodiment, if the metadata of the semantic information of the image includes dynamic or static image target data, since the dynamic or static image target data includes multiple types of targets, such as pedestrians, scenery, buildings, vehicles, and other different targets. Therefore, after setting the first-level priority for metadata such as image target data, face data, and text data, corresponding second-level priorities can be further set for multiple types of target data, so that in the process of adjusting the cropping boundary of the candidate cropping area based on the target data, the second-level priorities of each type of target can be adjusted in sequence. For example, the priority of pedestrians in the dynamic or static image target data is set to be the highest, the priority of buildings is the second highest, the priority of scenery is the third highest, and so on. The order of priorities is not specifically limited here.

[0093] Step S212, cropping the candidate cropping area of ​​the image to be cropped according to the adjusted cropping boundary.

[0094] In one embodiment of the present invention, the steps of calculating the metadata of the semantic information of the image to be cropped and setting the cropping priority for the metadata of each item of semantic information can also be performed before preprocessing the image to be cropped, so that after subsequently selecting the area with the largest mean of saliency values ​​from the saliency map as the candidate cropping area, the cropping boundary of the candidate cropping area can be adjusted directly based on the cropping priority of the metadata of the semantic information set in advance.

[0095] The solution of the present invention not only effectively retains the significant area in the original image to be cropped, but also effectively ensures the cropping integrity of the basic semantic unit, thereby greatly improving the accuracy of image cropping. In addition, by setting corresponding priorities for the metadata of each semantic information of the image, the integrity of more important metadata can be further guaranteed, and it also helps the cropping process to proceed more orderly, thereby improving the efficiency of image cropping.

[0096] Based on the same inventive concept, an embodiment of the present invention further provides an image cropping device based on semantic content. Figure 3 FIG. 4 is a schematic diagram showing the structure of a semantic content-based image cropping device according to an embodiment of the present invention. Figure 3 The semantic content-based image cropping device 300 includes a calculation module 310 , a selection module 320 , an adjustment module 330 , and a cropping module 340 .

[0097] The functions of the components or devices of the semantic content-based image cropping device 300 according to an embodiment of the present invention and the connection relationship between the components are now introduced:

[0098] A calculation module 310, adapted to perform saliency calculation on the image to be cropped according to a preset algorithm, and generate a corresponding saliency map according to the calculation result;

[0099] A selection module 320, coupled to the calculation module 310, is adapted to select a region with a maximum mean value of saliency values ​​from the saliency map as a candidate cropping region;

[0100] The adjustment module 330 is coupled to the selection module 320 and is adapted to calculate metadata of semantic information of the image to be cropped, and adjust the cropping boundary of the candidate cropping area based on the metadata of the semantic information to make the semantic information in the candidate cropping area complete;

[0101] The cropping module 340 is coupled to the adjustment module 330 and is adapted to crop the candidate cropping area of ​​the image to be cropped according to the adjusted cropping boundary.

[0102] In one embodiment of the present invention, the adjustment module 330 is further adapted to calculate metadata of semantic information in the image to be cropped using a preset deep neural network model.

[0103] In one embodiment of the present invention, the metadata of the semantic information in the image to be cropped includes at least one of dynamic or static image target data, face data, and text data. The dynamic or static image target data may include pedestrians, scenery, buildings, vehicles, and the like.

[0104] In an embodiment of the present invention, the adjustment module 330 is further adapted to set a cropping priority for metadata of each item of semantic information, and adjust the cropping boundary of the candidate cropping area according to the cropping priority of the metadata of the semantic information.

[0105] In one embodiment of the present invention, if the metadata of the semantic information in the image to be cropped includes dynamic or static image target data, face data, and text data, then when setting the cropping priority for the metadata of each piece of semantic information, the face data can be set with the highest priority, the dynamic or static image target data with the second highest priority, and the text data with the lowest priority.

[0106] The embodiment of the present invention also provides an image cropping device based on semantic content. Figure 4 FIG. 2 shows a schematic diagram of a structure of an image cropping device based on semantic content according to another embodiment of the present invention. Figure 4 The computing module 310 of the semantic content-based image cropping device 300 includes a preprocessing unit 311 and a computing unit 312 .

[0107] The preprocessing unit 311 is adapted to preprocess the image to be cropped so that the image to be cropped meets the cropping requirements.

[0108] The calculation unit 312 is coupled to the preprocessing unit 311 and is adapted to perform saliency calculation on the preprocessed image to be cropped according to a preset algorithm.

[0109] In an embodiment of the present invention, the pre-processing unit 311 is further adapted to convert the image format of the image to be cropped and / or scale the size of the image to be cropped.

[0110] An embodiment of the present invention further provides a computer storage medium storing computer program code. When the computer program code runs on a computing device, the computing device is caused to execute the semantic content-based image cropping method in any of the above embodiments.

[0111] An embodiment of the present invention further provides a computing device, comprising: a processor; a memory storing computer program code; when the computer program code is executed by the processor, the computing device executes the semantic content-based image cropping method in any of the above embodiments.

[0112] According to any one of the above preferred embodiments or a combination of multiple preferred embodiments, the embodiments of the present invention can achieve the following beneficial effects:

[0113] In an embodiment of the present invention, first, the saliency calculation is performed on the image to be cropped according to a preset algorithm, and a corresponding saliency map is generated according to the calculation result. Then, the area with the largest mean of saliency values ​​is selected from the saliency map as a candidate cropping area, and then the metadata of the semantic information of the image to be cropped is calculated. The cropping boundary of the candidate cropping area is adjusted based on the metadata of the semantic information to make the semantic information in the candidate cropping area complete, and finally, the candidate cropping area of ​​the image to be cropped is cropped according to the adjusted cropping boundary. Therefore, when the scheme of the present invention crops an image, it can effectively retain the area with the highest saliency in the image by cropping the image based on the image saliency, and after the image is cropped according to the saliency, the scheme also fully combines the semantic information in the image to crop the image, ensuring that the semantic content in the cropped image is complete, and effectively avoiding the damage to the integrity of the image semantic information, that is, the scheme retains the saliency area of ​​the original image and ensures the cropping integrity of the basic semantic unit, thereby greatly improving the accuracy of image cropping.

[0114] Those skilled in the art can clearly understand that the specific working processes of the systems, devices and units described above can refer to the corresponding processes in the aforementioned method embodiments, and for the sake of brevity, they are not further described here.

[0115] In addition, the functional units in various embodiments of the present invention may be physically independent of each other, or two or more functional units may be integrated together, or all functional units may be integrated into one processing unit. The above integrated functional units may be implemented in the form of hardware, or in the form of software or firmware.

[0116] Those skilled in the art can understand that if the integrated functional unit is implemented in the form of software and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention can essentially or all or part of the technical solution can be embodied in the form of a software product. The computer software product is stored in a storage medium, which includes a number of instructions to enable a computing device (such as a personal computer, a server, or a network device, etc.) to execute all or part of the steps of the method described in each embodiment of the present invention when running the instructions. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (ROM), random access memory (RAM), magnetic disk or optical disk and other media that can store program codes.

[0117] Alternatively, all or part of the steps of implementing the aforementioned method embodiments may be accomplished by hardware associated with program instructions (such as a computing device such as a personal computer, a server, or a network device), and the program instructions may be stored in a computer-readable storage medium. When the program instructions are executed by a processor of a computing device, the computing device executes all or part of the steps of the methods described in the embodiments of the present invention.

[0118] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit it. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that within the spirit and principles of the present invention, the technical solutions described in the aforementioned embodiments can still be modified, or some or all of the technical features therein can be replaced by equivalents. However, these modifications or replacements do not deviate from the protection scope of the present invention.

Claims

1. A semantic content-based image cropping method, comprising: Perform saliency calculation on the image to be cropped according to a preset algorithm, and generate a corresponding saliency map based on the calculation result; Selecting a region with the largest mean value of saliency values ​​from the saliency map as a candidate cropping region; Calculating metadata of the semantic information of the image to be cropped, and adjusting the cropping boundary of the candidate cropping area based on the metadata of the semantic information to make the semantic information in the candidate cropping area complete; The candidate cropping area of ​​the image to be cropped is cropped according to the adjusted cropping boundary.

2. The method according to claim 1, wherein: The saliency of the cropped image is calculated according to the preset algorithm, including: Preprocessing the image to be cropped so that the image to be cropped meets the cropping requirements; The saliency of the preprocessed image to be cropped is calculated according to a preset algorithm.

3. The method according to claim 2, wherein: Preprocessing the image to be cropped includes: Converting the image format of the image to be cropped; and / or The size of the image to be cropped is scaled proportionally.

4. The method according to any one of claims 1 to 3, wherein: Calculating metadata of semantic information of the image to be cropped, including: A preset deep neural network model is used to calculate metadata of semantic information in the image to be cropped.

5. The method according to any one of claims 1 to 3, wherein: The metadata of the semantic information in the image to be cropped includes at least one of the following: Dynamic or static image target data, face data, text data.

6. The method according to claim 5, wherein: Dynamic or static image target data includes: pedestrians, scenery, buildings, and vehicles.

7. The method according to claim 5, wherein: Adjusting the cropping boundary of the candidate cropping area based on the metadata of the semantic information includes: Set the trimming priority for the metadata of each semantic information; The cropping boundary of the candidate cropping area is adjusted according to the cropping priority of the metadata of the semantic information.

8. The method according to claim 7, wherein: Set the trimming priority for metadata of each semantic information, including: Set the face data to have the highest priority, dynamic or static image target data to have the second highest priority, and text data to have the lowest priority.

9. A semantic content-based image cropping device, comprising: A calculation module, adapted to perform saliency calculation on the cropped image according to a preset algorithm, and generate a corresponding saliency map according to the calculation result; A selection module, adapted to select a region with the largest mean of saliency values ​​from the saliency map as a candidate cropping region; an adjustment module, adapted to calculate metadata of the semantic information of the image to be cropped, and adjust the cropping boundary of the candidate cropping area based on the metadata of the semantic information to make the semantic information in the candidate cropping area complete; The cropping module is adapted to crop the candidate cropping area of ​​the image to be cropped according to the adjusted cropping boundary.

10. The device according to claim 9, wherein: The calculation module also includes: A preprocessing unit, adapted to preprocess the image to be cropped so that the image to be cropped meets the cropping requirements; The calculation unit is adapted to perform saliency calculation on the preprocessed image to be cropped according to a preset algorithm.

11. The device according to claim 10, wherein: The pre-processing unit is also suitable for: Converting the image format of the image to be cropped; and / or The size of the image to be cropped is scaled proportionally.

12. The device according to any one of claims 9 to 11, wherein: The adjustment module is also suitable for: A preset deep neural network model is used to calculate metadata of semantic information in the image to be cropped.

13. The device according to any one of claims 9 to 11, wherein: The metadata of the semantic information in the image to be cropped includes at least one of the following: Dynamic or static image target data, face data, text data.

14. The device according to claim 13, wherein: Dynamic or static image target data includes: pedestrians, scenery, buildings, and vehicles.

15. The device according to claim 13, wherein: The adjustment module is also suitable for: Set the trimming priority for the metadata of each semantic information; The cropping boundary of the candidate cropping area is adjusted according to the cropping priority of the metadata of the semantic information.

16. The device according to claim 15, wherein: The adjustment module is also suitable for: Set the face data to have the highest priority, dynamic or static image target data to have the second highest priority, and text data to have the lowest priority.

17. A computer storage medium storing a computer program code, which, when executed on a computing device, causes the computing device to execute the semantic content-based image cropping method according to any one of claims 1 to 8.

18. A computing device comprising: processor; a memory storing computer program code; When the computer program code is executed by the processor, it causes the computing device to execute the semantic content-based image cropping method according to any one of claims 1 to 8.

Images