An image processing method, apparatus, device and storage medium

Abstract

The present disclosure provides an image processing method, device, equipment and storage medium, the method comprises: obtaining one or more images including a to-be-processed image, when receiving an image processing operation triggered for the to-be-processed image, generating a processing result image based on the to-be-processed image and an image template. Wherein, the processing result image and the image template have a corresponding relationship, and the image template belongs to an image template set matched for the to-be-processed image according to template feature information. After obtaining the to-be-processed image, the embodiment of the present disclosure can automatically match the to-be-processed image to the image template according to the template feature information, and generate the processing result image based on the matched image template and the to-be-processed image, which enriches the image processing mode, thereby improving the user's image processing interaction experience.

Description

Technical Field

[0001] This disclosure relates to the field of data processing, and more particularly to an image processing method, apparatus, device, and storage medium. Background Technology

[0002] With the continuous development of image processing technology, people's needs for image processing-related functions are becoming more and more diversified.

[0003] To meet people's diverse needs for image processing functions, how to further enrich image processing-related functions is a technical problem that urgently needs to be solved. Summary of the Invention

[0004] To address the aforementioned technical problems, this disclosure provides an image processing method, apparatus, device, and storage medium.

[0005] In a first aspect, this disclosure provides an image processing method, the method comprising:

[0006] Acquire an image to be processed; wherein the image to be processed includes one or more images;

[0007] In response to an image processing operation triggered for the image to be processed, a processing result image is generated based on the image to be processed and an image template; wherein, there is a correspondence between the processing result image and the image template, and the image template belongs to a set of image templates matched for the image to be processed based on template feature information.

[0008] In one optional implementation, the template feature information includes at least one of template image feature information, template text feature information, and template slot feature information. The template image feature information includes image feature information extracted from the image template. The template text feature information includes text feature information extracted from the text of the image template. The template slot feature information includes the number of slots in the image template and / or the image subject position information of each slot. The image subject position information is used to characterize the position information of the main object displayed on the filled image of the corresponding slot in the image template.

[0009] In one optional implementation, generating a processed result image based on the image to be processed and an image template in response to an image processing operation triggered for the image to be processed includes:

[0010] In response to an image processing operation triggered for the image to be processed, the matching feature information of the image to be processed is obtained; wherein, the matching feature information includes at least one of the image feature information of the image to be processed, subject location information, and number of images;

[0011] By matching the feature information to be matched of the image to be processed with the template feature information in the template library, an image template set including at least one image template is determined from the template library for the image to be processed;

[0012] A processed result image is generated based on the image to be processed and the image templates in the image template set.

[0013] The template feature information includes the template image feature information. The step of matching the feature information to be matched of the image to be processed with the template feature information in the template library to determine an image template set including at least one image template from the template library for the image to be processed includes:

[0014] By matching the image feature information of the image to be processed with the template image feature information in the template feature information, a first candidate template set is determined for the image to be processed from the template library; wherein, the first candidate template set includes image templates that successfully match the image feature information of the image to be processed;

[0015] From the first candidate template set, determine an image template set that includes at least one image template for the image to be processed.

[0016] In one optional implementation, the template feature information includes the template text feature information. The step of matching the feature information to be matched of the image to be processed with the template feature information in the template library to determine a set of image templates including at least one image template from the template library for the image to be processed includes:

[0017] By matching the image feature information of the image to be processed with the template text feature information in the template feature information, a second candidate template set is determined for the image to be processed from the template library; wherein, the second candidate template set includes image templates whose template text feature information successfully matches the image feature information of the image to be processed;

[0018] From the second candidate template set, determine an image template set that includes at least one image template for the image to be processed.

[0019] In one optional implementation, the template feature information includes the number of slots. The step of determining an image template set for the image to be processed from the template library by matching the matching feature information of the image to be processed with the template feature information in the template library includes:

[0020] By matching the number of slots in the image template with the number of images in the image to be processed, a third candidate template set is determined from the template library for the image to be processed; wherein, the third candidate template set includes image templates whose number of slots successfully matches the number of images in the image to be processed;

[0021] From the third candidate template set, determine an image template set that includes at least one image template for the image to be processed.

[0022] In an optional implementation, the template feature information further includes image subject position information of the slot, and the step of determining an image template set including at least one image template for the image to be processed from the third candidate template set includes:

[0023] By matching the subject position information of the image to be processed with the subject position information of the slots in the template feature information, a fourth candidate template set is determined for the image to be processed from the third candidate template set; wherein, the fourth candidate template set includes image templates whose number of slots and subject position information are both successfully matched with the number of images and subject position information of the image to be processed.

[0024] From the fourth candidate template set, an image template set including at least one image template is determined for the image to be processed.

[0025] In an optional implementation, before generating the processed result image based on the image to be processed and the image template, the method further includes:

[0026] Text content is generated based on the image to be processed and the image template; wherein, the text content is used to reflect the image features of the image to be processed;

[0027] The step of generating the processed result image based on the image to be processed and the image template includes:

[0028] Based on the image to be processed, the text content, and the image template, a processed result image is generated.

[0029] In an optional implementation, after generating a processed result image based on the image to be processed and an image template in response to an image processing operation triggered for the image to be processed, the method further includes:

[0030] The processed image is displayed within the image editing window;

[0031] Receive editing operations on the text content in the processed result image, and update the text content in the processed result image based on the editing operations.

[0032] In one optional implementation, the method further includes:

[0033] Display a list of processed result images corresponding to the image template set; wherein, the list of processed result images includes processed result images corresponding to each image template in the image template set, and the processed result images are generated based on the corresponding image template and the image to be processed.

[0034] Secondly, this disclosure provides an image processing apparatus, the apparatus comprising:

[0035] An acquisition module is used to acquire an image to be processed; wherein, the image to be processed includes one or more images;

[0036] The first generation module is configured to generate a processing result image based on the image to be processed and an image template in response to an image processing operation triggered for the image to be processed; wherein the processing result image and the image template have a corresponding relationship, and the image template belongs to a set of image templates matched for the image to be processed based on template feature information.

[0037] Thirdly, this disclosure provides a computer-readable storage medium storing instructions that, when executed on a terminal device, cause the terminal device to implement the above-described method.

[0038] Fourthly, this disclosure provides an electronic device, including: a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein when the processor executes the computer program, it implements the method described above.

[0039] Fifthly, this disclosure provides a computer program product comprising a computer program / instruction that, when executed by a processor, implements the method described above.

[0040] The technical solution provided in this disclosure has at least the following advantages compared with the prior art:

[0041] This disclosure provides an image processing method. First, an image to be processed is acquired, wherein the image to be processed includes one or more images. When an image processing operation triggered for the image to be processed is received, a processed result image is generated based on the image to be processed and an image template. The processed result image and the image template have a corresponding relationship, and the image template belongs to a set of image templates matched to the image to be processed based on template feature information.

[0042] After obtaining the image to be processed, the embodiments of this disclosure can automatically match an image template for the image to be processed based on the template feature information, and generate a processed result image based on the matched image template and the image to be processed, thereby enriching the image processing methods and improving the user's image processing interactive experience. Attached Figure Description

[0043] The accompanying drawings, which are incorporated in and form a part of this specification, illustrate embodiments consistent with this disclosure and, together with the description, serve to explain the principles of this disclosure.

[0044] To more clearly illustrate the technical solutions in the embodiments of this disclosure or the prior art, the accompanying drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, for those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0045] Figure 1 A schematic flowchart of an image processing method provided in an embodiment of this disclosure;

[0046] Figure 2 A schematic diagram of page interaction provided for an embodiment of this disclosure;

[0047] Figure 3 Another schematic diagram of page interaction provided for an embodiment of this disclosure;

[0048] Figure 4 This is a schematic diagram of an image processing process provided in an embodiment of the present disclosure;

[0049] Figure 5 This is a schematic diagram of the structure of an image processing apparatus provided in an embodiment of the present disclosure;

[0050] Figure 6 This is a schematic diagram of the structure of an electronic device provided in an embodiment of the present disclosure. Detailed Implementation

[0051] To better understand the above-mentioned objectives, features, and advantages of this disclosure, the solutions disclosed herein will be further described below. It should be noted that, unless otherwise specified, the embodiments and features described herein can be combined with each other.

[0052] Numerous specific details are set forth in the following description in order to provide a full understanding of this disclosure, but this disclosure may also be implemented in other ways different from those described herein; obviously, the embodiments in the specification are only some, and not all, of the embodiments of this disclosure.

[0053] With the continuous development of image processing technology, people's needs for image processing-related functions are becoming more and more diversified.

[0054] To meet people's diverse needs for image processing functions, how to further enrich image processing-related functions is a technical problem that urgently needs to be solved.

[0055] Therefore, this disclosure provides an image processing method. First, an image to be processed is acquired, wherein the image to be processed includes one or more images. When an image processing operation triggered for the image to be processed is received, a processed result image is generated based on the image to be processed and an image template. The processed result image and the image template have a corresponding relationship, and the image template belongs to a set of image templates matched to the image to be processed based on template feature information.

[0056] After obtaining the image to be processed, the embodiments of this disclosure can automatically match an image template for the image to be processed based on the template feature information, and generate a processed result image based on the matched image template and the image to be processed, thereby enriching the image processing methods and improving the user's image processing interactive experience.

[0057] Based on this, the present disclosure provides an image processing method, referring to... Figure 1 The above is a flowchart of an image processing method provided in an embodiment of this disclosure. The method specifically includes:

[0058] S101: Obtain the image to be processed.

[0059] The image to be processed includes one or more images.

[0060] The image processing method provided in this disclosure can be applied to a client, such as a client deployed on a smartphone, a client deployed on a tablet, etc.

[0061] In this embodiment of the disclosure, the image to be processed may include one or more images input by the user.

[0062] In one optional implementation, the image to be processed can be one or more images selected by the user from the image album. Specifically, when a user-triggered image selection operation is received, the image album page is displayed, which shows a collection of image materials. When a selection operation is received for at least one image material in the image material collection, the selected image material is determined as the image to be processed in this embodiment of the disclosure.

[0063] like Figure 2The diagram illustrates a page interaction according to an embodiment of this disclosure. It shows a media album page 201, displaying multiple image materials, including image material 202. Upon receiving a selection operation for image material 202, image material 202 is identified as the image to be processed; that is, the image to be processed includes image material 202.

[0064] It is worth noting that in this embodiment of the disclosure, the user can also select multiple image materials based on the material album page as images to be processed in this embodiment of the disclosure.

[0065] Furthermore, the embodiments disclosed herein do not limit the method of acquiring the image to be processed; for example, the image to be processed can also be acquired by taking a picture.

[0066] S102: In response to an image processing operation triggered for the image to be processed, generate a processing result image based on the image to be processed and the image template.

[0067] The processed image and the image template have a corresponding relationship, and the image template belongs to the set of image templates obtained by matching the image to be processed based on template feature information.

[0068] In one optional implementation, an image template set can be obtained by matching the image to be processed from the template library based on the template feature information. Specifically, the template library stores multiple image templates and the template feature information corresponding to each of the multiple image templates.

[0069] In this embodiment of the disclosure, after acquiring the image to be processed, if an image processing operation triggered for the image to be processed is received, a processed result image is generated based on the image to be processed and the image template. This image processing operation can be implemented by the user triggering an image generation control on the page. One way to generate the processed result image based on the image to be processed and the image template is to fill the image to be processed; specifically, the image to be processed is filled into the slots of the matching image template to generate the processed result image.

[0070] As mentioned above Figure 2 In the illustrated page diagram, after obtaining the image to be processed 202, an image generation control 203 is displayed on the material album page 201. When the trigger operation of the image generation control 203 is received, a processing result image is generated based on the image to be processed 202 and the image template, and an image generation page 204 is displayed. The image generation page 204 displays multiple processing result images, including processing result image 205 and processing result image 206.

[0071] The image template is a specific image style or structure pre-set and stored in a template library. The image template may contain feature information such as images, text, and the number of slots. Each image template has its corresponding template feature information, which is used for template matching and filtering. The template library can be a database storing image templates and their corresponding template feature information, providing image template resources for image processing. In this embodiment, the image template set includes each image template matched to the image to be processed.

[0072] The resulting image is generated by combining the image to be processed and the image template. Each resulting image corresponds to one image template. In other words, for each image template matched with the image to be processed, a corresponding resulting image will be generated. The number of resulting images can represent the number of image templates matched based on the image to be processed.

[0073] In this embodiment of the present disclosure, after generating the processed result image based on the image to be processed and each image template in the image template set, a list of processed result images corresponding to the image template set can be displayed on the page. This list of processed result images includes the processed result images corresponding to each image template in the image template set. That is, for each image template in the image template set, a fill operation is performed between it and the image to be processed to obtain the corresponding processed result image, and all the obtained processed result images constitute the processing result list.

[0074] As mentioned above Figure 2 In the page interaction diagram shown, after generating the processing result image based on the above-mentioned image to be processed 202 and each image template in the image template set, and displaying the image generation page 204, the image generation page displays a list of processing result images containing all the processing result images. This list of processing result images corresponds to the image template set. The processing result image 205 and processing result image 206 in the processing result list correspond to two image templates in the image template set, respectively.

[0075] In one optional implementation, after receiving an image processing operation triggered for the image to be processed, and generating a processed result image based on the image to be processed and the image template, the processed result image can be previewed to allow the user to view the display effect of each processed result image in a timely manner. Specifically, the processed result image can be displayed in the image editing window on the image generation page, and this processed result image is one that the user selects from the list of processed result images.

[0076] As mentioned above Figure 2In the illustrated page interaction diagram, the image editing window 207 on the image generation page 204 displays the processed result image 205.

[0077] During the user's preview of the processed image, editing operations can be triggered on the processed image displayed in the image editing window to optimize the image and improve its visual effect. Specifically, while the processed image is displayed in the image editing window, editing operations on the text content in the processed image are received, and the text content in the processed image is updated based on the editing operations.

[0078] In one application scenario, users can directly trigger editing operations on the text content in the processed image displayed in the image editing window. In another application scenario, editing controls are displayed on the image generation page. During the process of displaying the processed image in the image editing window, when a trigger operation for the editing controls is received, the image editing page is displayed, which shows the processed image and various editing controls.

[0079] like Figure 3 As shown, this is another page interaction diagram provided by an embodiment of the present disclosure. The image editing window 302 of the image generation page 301 displays the processed result image 303 and the editing control 304. When the trigger operation of the editing control 304 is received, the image editing page 305 is displayed, which displays the processed result image 303 and various editing controls.

[0080] In addition, after generating a list of all processed result images based on the image to be processed and each image template in the image template set, the order of the processed result images in the list of processed result images displayed on the image generation page can characterize the degree of matching between the image templates in the image template set and the image to be processed. That is, the earlier the processed result image is in the order, the higher the degree of matching between the image template corresponding to the processed result image and the image to be processed.

[0081] In the image processing method provided in this disclosure, an image to be processed is acquired, wherein the image to be processed includes one or more images. When an image processing operation triggered for the image to be processed is received, a processed result image is generated based on the image to be processed and an image template. The processed result image and the image template have a corresponding relationship, and the image template belongs to a set of image templates matched to the image to be processed based on template feature information.

[0082] After obtaining the image to be processed, the embodiments of this disclosure can automatically match an image template for the image to be processed based on the template feature information, and generate a processed result image based on the matched image template and the image to be processed, thereby enriching the image processing methods and improving the user's image processing interactive experience.

[0083] In one optional implementation, the template feature information of the image template may include at least one of template image feature information, template text feature information, and template slot feature information. The template image feature information includes image feature information extracted from the image template. Specifically, the image feature information may include feature information such as image appearance, tone, and theme category. Extracting template image feature information from the image template can be achieved using an image feature extraction model. Specifically, each image template in the template library is input into the image feature extraction model, which analyzes and processes each image template, outputting an image feature vector for each image template as its template image feature information.

[0084] The template text feature information includes text feature information extracted from the text of the image template. Specifically, this text feature information can include text information such as keywords in the text of the image template, for example, the theme, nouns, etc., in the text of the image template. Extracting text feature information from image templates can be achieved through a text feature extraction model. Specifically, the text of image templates in the template library is extracted, and the text corresponding to each image template is input into the text feature extraction model. The text feature extraction model processes the text and outputs the text feature vector of each image template as the text feature information of the image template.

[0085] The template slot feature information includes the number of slots in the image template and / or the image subject position information for each slot. The slots in the image template are the positions used to fill the image to be processed, and each image template may have a different number of slots. The image subject position information of the slots is used to characterize the position information of the main object displayed on the filled image of the corresponding slot in the image template. Specifically, the image subject position information of the slot can refer to the coordinate information of the main object in the filled image of the slot in the image template, such as the coordinate information of the template image subject at the upper left corner or the coordinate information of the template image subject at the lower right corner. The image subject position information of each slot in the image template can be obtained by detecting the image template using a position detection model. Specifically, each image template in the template library can be input into the position detection model. This model detects the position of each slot in the image template and extracts the main object position of the template image in each slot. Then, it outputs the main object position feature vector of the template image in each slot of each image template, which is used as the template slot feature information of the image template.

[0086] By extracting template feature information from dimensions such as image, text, and slots of the image template, the matching degree between the image to be processed and the image template can be improved, so as to select a suitable image template for the image to be processed.

[0087] In one optional implementation, after extracting the template feature information of each image template and storing the template feature information in the template library, if an image processing operation triggered for the image to be processed is received, the matching feature information of the image to be processed is obtained, and then the matching feature information is matched with the template feature information in the template library to determine an image template set containing at least one image template from the template library. Subsequently, a processing result image is generated based on the image to be processed and each image template in the image template set.

[0088] The matching feature information of the image to be processed may include at least one of the following: image feature information, subject location information, and number of images. This image feature information may include image feature information extracted from one or more images included in the image to be processed. Specifically, this image feature information may include features such as image appearance, tone, and subject category. Extracting image feature information from one or more images included in the image to be processed can be achieved using an image feature extraction model. Specifically, the acquired image to be processed is input into the image feature extraction model, which analyzes and processes the one or more images included in the image to be processed, and outputs image feature vectors for each image as the image feature information for each image.

[0089] The subject position information of the image to be processed is used to characterize the display position information of the main object in each image of the image to be processed. Specifically, it can refer to the coordinate position information of the main object such as people or things in the image to be processed. Subject position information can be obtained from the image to be processed through a position detection model. Specifically, each image included in the image to be processed is input into the position detection model, which detects and extracts the subject position in each image, and then outputs the image subject position feature vector of each image, which is used as the subject position information of each image in the image to be processed. The number of images in the image to be processed can be the number of images uploaded by the user.

[0090] After obtaining the matching feature information of the image to be processed, the image template is determined based on the matching feature information. Specifically, after obtaining the matching feature information of each image included in the image to be processed, the matching feature information of each image is matched with the feature information of each template in the template library to obtain the score of each image template. Then, based on the score of each image template, an image template set including at least one image template is determined from the template library.

[0091] In one optional implementation, if the template feature information of the image template includes template image feature information, then an image template can be matched for the image to be processed from the image dimension. Specifically, the image feature information in the image to be processed is matched with the template image feature information in the template library, thereby filtering out a first candidate template set for the image to be processed from the template library, and then determining an image template set including at least one image template based on the first candidate template set for the image to be processed.

[0092] The matching method between the image feature information of the image to be processed and the template image feature information of each image template in the template library can be as follows: the image feature information of each image included in the image to be processed is matched with the template image feature information of each image template in the template library to obtain a similarity score between each image and each image template. Based on the similarity score, the comprehensive score of each image template is determined, and then a first candidate template set is selected from the template library based on the comprehensive score. The method of determining the comprehensive score of each image template based on the similarity score is not limited in this embodiment. Optionally, the similarity scores between each image in the image to be processed and the image template can be weighted and summed to obtain the comprehensive score of each image template. The weight of each image in the image to be processed can be set according to the importance of the image itself. Alternatively, the average of the similarity scores between each image in the image to be processed and the image template can be taken as the comprehensive score of each image template. The method of selecting the first candidate template set from the template library based on the comprehensive score can be a first candidate template set composed of image templates with a comprehensive score exceeding a preset value. This embodiment is not limited in this respect.

[0093] The first candidate template set includes image templates that successfully match the image feature information of the image to be processed. Specifically, the image template can be an image template with a high degree of matching between the template image feature information and the image feature information of the image to be processed. Optionally, it can be an image template whose matching degree between the template image feature information and the image feature information of the image to be processed exceeds a preset matching threshold.

[0094] After determining the first candidate template set from the template library, in order to improve the matching accuracy, the first candidate template set can be further filtered based on other dimensions, thereby determining an image template set that includes at least one image template for the image to be processed.

[0095] In one optional implementation, if the template feature information of the image template includes template text feature information, then image templates can also be matched for the image to be processed from the text dimension. Specifically, the image feature information in the image to be processed is matched with the template text feature information, and then a second candidate template set is determined for the image to be processed from the template library. Subsequently, an image template set including at least one image template is determined for the image to be processed based on the second candidate template set.

[0096] The matching method between the image feature information of the image to be processed and the template text feature information of each image template in the template library can be as follows: The image feature information of each image included in the image to be processed is matched with the template text feature information of each image template in the template library to obtain a matching score between each image and each image template. Based on each matching score, a comprehensive score for each image template is determined, and then a second candidate template set is selected from the template library based on the comprehensive score. The method of determining the comprehensive score of each image template based on each matching score is not limited in this embodiment. Optionally, the matching scores between each image in the image to be processed and the image template can be weighted and summed to obtain the comprehensive score of each image template. The weight of each image in the image to be processed can be set according to the importance of the image itself. Alternatively, the average of the matching scores between each image in the image to be processed and the image template can be taken as the comprehensive score of each image template. The method of selecting the second candidate template set from the template library based on the comprehensive score can be a second candidate template set composed of image templates with a comprehensive score exceeding a preset value. This embodiment is not limited in this respect.

[0097] The second candidate template set includes image templates whose template text features successfully match the image features of the image to be processed. The image template can be an image template with a high degree of matching between the template text features and the image features of the image to be processed. Optionally, it can be an image template whose degree of matching between the template text features and the image features of the image to be processed exceeds a preset matching threshold.

[0098] After obtaining the second candidate template set, further filtering can be performed from the second candidate template set to determine an image template set including at least one image template for the image to be processed, so as to improve the matching accuracy.

[0099] It is evident that by determining a second set of candidate templates for the image to be processed from the template library based on the text dimension, and then determining at least one image template for the image to be processed based on the second set of candidate templates, the text in the image template can be kept in good consistency with the image to be processed, thereby improving the matching accuracy of the image template.

[0100] In one optional implementation, if the template feature information of the image template includes the number of slots, then image templates can also be matched for the image to be processed from the dimension of the number of slots. Specifically, the number of images of the image to be processed is matched with the number of slots of each image template, and then a third candidate template set is determined for the image to be processed from the template library. Based on the third candidate template set, an image template set including at least one image template is determined for the image to be processed.

[0101] Matching the number of slots in an image template with the number of images to be processed can be achieved by selecting image templates from a template library whose number of slots matches the number of images to be processed, and constructing a third candidate template set based on these selected templates. This third candidate template set includes image templates whose number of slots matches the number of images to be processed; that is, the number of slots in the image templates in the third candidate template set is consistent with the number of images to be processed. For example, if there are 3 images to be processed, then the image templates in the third candidate template set will be image templates with 3 slots.

[0102] After obtaining the third candidate template set, further filtering can be performed from this third candidate template set to create an image template set that includes at least one image template for the image to be processed, thereby further improving the matching accuracy.

[0103] Based on the above embodiments, if the template feature information of the image template also includes the image subject position information of the slot, then the image template can be selected from the third candidate template set for the image to be processed by combining the image subject position information. The image subject position information can characterize the subject position information of the template image on the corresponding slot.

[0104] Specifically, the subject position information of the image to be processed is matched with the subject position information of the slots of the image templates in the third candidate template set. Each image template is scored based on the matching degree. Then, a fourth candidate template set is determined from the third candidate template set based on the scoring results. Subsequently, an image template set including at least one image template is determined from the fourth candidate template set for the image to be processed. The subject position of the slots of the image templates in the fourth candidate template set can be within a preset area, which can be a pre-defined region.

[0105] In one optional implementation, the matching method between the subject position information of the image to be processed and the subject position information of the slot in the template feature information can be as follows: the subject position information of each image included in the image to be processed is matched with the subject position information of the corresponding slot in the image template in the third candidate template set according to the image upload order. If an image template is selected from the third candidate template set whose subject position information of the slot matches the subject position information of the image to be processed, then the successfully matched image template constitutes the fourth template candidate set. The correspondence between each slot in the image template and each image in the image to be processed can be that the image upload order corresponds to the slot sequence number; for example, the first uploaded image corresponds to the first slot in the image template.

[0106] In another optional implementation, the matching method between the subject position information of the image to be processed and the subject position information of the slot of the template feature information can also be as follows: distinguish the slots of each image template in the third candidate template set from the main image slot and the detail image slot, and then analyze and determine the subject position information of the main image and the subject position information of the detail image based on the subject position information of each image template. Then match the subject position information of the main image slot and the subject position information of the detail image slot of each image template with the subject position information of the main image and the subject position information of the detail image, respectively. If a matching image template is selected from the third candidate template, the matching image template constitutes the fourth template candidate set.

[0107] After obtaining the fourth candidate template set, the embodiments of this disclosure can further filter the image templates in the fourth candidate template set to determine an image template set including at least one image template for the image to be processed, so as to improve the matching accuracy.

[0108] In one optional implementation, after matching an image template set including at least one image template to the image to be processed, the image templates in the image template set can be subjected to text readability detection, and each image template can be sorted according to the text readability detection results to obtain an image template list, so as to preferentially generate the processed result image based on the image template with better text readability and the image to be processed.

[0109] Specifically, the images to be processed are pre-filled into the slots of each image template in the image template set. It is then determined whether the contrast between the text in the image template and the image to be processed exceeds a preset contrast threshold. Image templates with contrast exceeding the preset threshold are ranked first, and those with contrast below the threshold are ranked last. An image template list is then constructed based on this ranking. The contrast ratio between the text in the image template and the image to be processed can be determined by calculating the contrast ratio between the text color and the background color of the image template and the image to be processed.

[0110] In one optional implementation, before matching an image template set including at least one image template to the image to be processed, and before generating a processed result image based on the image to be processed and the image template, since the text in the image template has poor compatibility with the image to be processed, the text in the image template can be rewritten before generating the processed result image based on the image to be processed and the image template, so that the newly generated text content can reflect the image features of the image to be processed, such as reflecting the features of the main object on the image to be processed.

[0111] Specifically, text content is generated based on the image to be processed and each image template in the image template set. Then, based on the image to be processed, the text content, and the aforementioned image templates, a processed result image is generated. The text content of the image to be processed and each image template in the image template list can be input into the text generation model, which processes the text content based on the image to be processed to generate new text content.

[0112] In the process of using a text generation model to process the text of each image template in the image template list based on the image to be processed to generate new text content, in order to improve the aesthetics of the generated image, this embodiment of the disclosure can also adjust the newly generated text content based on the text information in the image template. This text information may include information such as the text length and font size in the image template. Specifically, the text generation model can be used to process and adjust the text based on the image to be processed and the text length and font size in each image template in the image template list to generate new text content.

[0113] Building upon the above embodiments, to further enhance the aesthetics of the generated processed image, this embodiment can also adjust the position of the main subject in the image to be processed during the process of generating the processed image based on the image to be processed, the newly generated text content, and the image templates in the image template set, to avoid the problem of the main subject shifting in the image to be processed. Specifically, the center height is aligned according to the position of the main subject in the image to be processed and the position of the main subject in the slot of the image template in the image template set, and the aligned image to be processed is then cropped or enlarged.

[0114] In one application scenario, after centering the image to be processed with the image subject position in the slot of the image template in the image template set, if the image to be processed has an extra part relative to the slot of the image template, the extra part of the image to be processed is cropped; if the image to be processed has a gap relative to the slot of the image template, the image to be processed is enlarged.

[0115] To facilitate understanding of the above embodiments, this disclosure will provide illustrated diagrams of the image processing process from three dimensions: image, text, and slots. For example... Figure 4 The diagram shown is a schematic representation of an image processing procedure provided in an embodiment of this disclosure.

[0116] Specifically, template feature information is extracted from the image template. This template feature information includes at least one of template image feature information, template text feature information, and template slot feature information. The template feature information includes the number of slots in the image template and / or the position information of the main image element in each slot. Then, the image template and its corresponding template feature information are stored in a template library.

[0117] After extracting the template feature information of the image template, the user-input image A is obtained. When an image processing operation triggered for image A is received, the matching feature information of image A is obtained. The matching feature information includes at least one of the image feature information of the image to be processed, the subject position information, and the number of images.

[0118] Then, the feature information to be matched is matched with the template feature information in the template library to determine an image template set including at least one image template for image A from the template library.

[0119] In one optional implementation, firstly, based on the template image feature information in the template library, image templates are matched for image A from the image dimension. Specifically, the image feature information of image A is matched with the template image feature information of each image template in the template library to obtain the similarity score between image A and each image template. Based on each similarity score, the comprehensive score of each image template is determined, and then the first candidate template set is selected from the template library based on the comprehensive score.

[0120] After obtaining the first candidate template set, image templates are matched for the image to be processed from the text dimension based on the template text feature information in the template feature information. Specifically, the image feature information of image A is matched with the template text feature information of each image template in the first candidate template set to obtain the matching score between image A and each image template. Based on each matching score, the comprehensive score of each image template is determined, and then a second candidate template set is selected for image A from the first candidate template set based on the comprehensive score.

[0121] After obtaining the second candidate template set, image templates can be matched for image A based on the slot dimension. Specifically, firstly, image templates with a number of slots matching the number of images in image A are selected from the second candidate template set, and a third candidate template set is constructed based on these selected image templates. Then, the subject position information of image A is matched with the subject position information of the slots in the image templates in the third candidate template set, thereby determining a fourth candidate template set for image A from the third candidate template set.

[0122] In one optional implementation, after obtaining the fourth candidate template set, the fourth candidate template set can be used as an image template set. The image templates in the fourth candidate template set can be subjected to text readability detection, and the determined image templates can be sorted according to the text readability detection results to obtain an image template list, so as to preferentially generate the processed result image based on the image template with better text readability and image A.

[0123] After matching an image template set containing at least one image template to image A and obtaining an image template list based on text readability detection, the text content of the image templates will be processed based on image A and each image template in the image template list. Specifically, based on image A and each image template in the image template list, a text generation model will be used to generate new text content based on image A.

[0124] After obtaining the new text content, the image template will be filled with data to generate a list of processed images. During the generation of the processed images, the position of the main subject in image A can be adjusted to avoid offset. Specifically, the center height of the main subject in image A is aligned with the position of the main subject in the slot of the image template. The aligned image A is then cropped or enlarged to improve the aesthetics of the generated processed images.

[0125] As can be seen, after obtaining the image to be processed, the embodiments of this disclosure can automatically match an image template from the template library according to the template feature information, and generate a processed result image based on the matched image template and the image to be processed, thereby enriching the image processing methods and improving the user's image processing interactive experience.

[0126] Based on the above method embodiments, this disclosure also provides an image processing apparatus, with reference to... Figure 5 This is a schematic diagram of the structure of an image processing apparatus provided in an embodiment of the present disclosure. The apparatus includes:

[0127] The acquisition module 501 is used to acquire an image to be processed; wherein the image to be processed includes one or more images;

[0128] The first generation module 502 is used to generate a processing result image based on the image to be processed and the image template in response to an image processing operation triggered for the image to be processed; wherein the processing result image and the image template have a corresponding relationship, and the image template belongs to a set of image templates matched for the image to be processed according to template feature information.

[0129] In one optional implementation, the template feature information includes at least one of template image feature information, template text feature information, and template slot feature information. The template image feature information includes image feature information extracted from the image template. The template text feature information includes text feature information extracted from the text of the image template. The template slot feature information includes the number of slots in the image template and / or the image subject position information of each slot. The image subject position information is used to characterize the position information of the main object displayed on the filled image of the corresponding slot in the image template.

[0130] In one optional implementation, the first generation module includes:

[0131] The acquisition submodule is used to acquire the matching feature information of the image to be processed in response to an image processing operation triggered for the image to be processed; wherein the matching feature information includes at least one of the image feature information of the image to be processed, subject position information, and number of images;

[0132] The matching submodule is used to determine an image template set including at least one image template from the template library by matching the matching feature information of the image to be processed with the template feature information in the template library; wherein, the template library stores multiple image templates and template feature information corresponding to the multiple image templates respectively;

[0133] The generation submodule is used to generate a processed result image based on the image to be processed and the image templates in the image template set.

[0134] In one optional implementation, the template feature information includes the template image feature information, and the matching submodule includes:

[0135] The first determining submodule is used to determine a first candidate template set for the image to be processed from the template library by matching the image feature information of the image to be processed with the template image feature information in the template feature information; wherein, the first candidate template set includes image templates that successfully match the image feature information of the image to be processed;

[0136] The second determining submodule is used to determine an image template set including at least one image template for the image to be processed from the first candidate template set.

[0137] In one optional implementation, the template feature information includes the template text feature information, and the matching submodule includes:

[0138] The third determining submodule is used to determine a second candidate template set for the image to be processed from the template library by matching the image feature information of the image to be processed with the template text feature information in the template feature information; wherein, the second candidate template set includes image templates whose template text feature information successfully matches the image feature information of the image to be processed;

[0139] The fourth determining submodule is used to determine an image template set including at least one image template for the image to be processed from the second candidate template set.

[0140] In one optional implementation, the template feature information includes the number of slots, and the matching submodule includes:

[0141] The fifth determining submodule is used to determine a third candidate template set for the image to be processed from the template library by matching the number of slots in the image template with the number of images in the image to be processed; wherein the third candidate template set includes image templates whose number of slots successfully matches the number of images in the image to be processed;

[0142] The sixth determining submodule is used to determine an image template set including at least one image template for the image to be processed from the third candidate template set.

[0143] In an optional implementation, the template feature information further includes image subject position information of the slot, and the sixth determining submodule includes:

[0144] The seventh determining submodule is used to determine a fourth candidate template set for the image to be processed from the third candidate template set by matching the subject position information of the image to be processed with the subject position information of the slots in the template feature information; wherein, the fourth candidate template set includes image templates whose number of slots and subject position information are both successfully matched with the number of images and subject position information of the image to be processed.

[0145] The eighth determining submodule is used to determine an image template set including at least one image template for the image to be processed from the fourth candidate template set.

[0146] In one optional embodiment, the apparatus further includes:

[0147] The second generation module is used to generate text content based on the image to be processed and the image template; wherein the text content is used to reflect the image features of the image to be processed;

[0148] The first generation module is specifically used for:

[0149] Based on the image to be processed, the text content, and the image template, a processed result image is generated.

[0150] In one optional embodiment, the apparatus further includes:

[0151] The display module is used to display the processed image within the image editing window;

[0152] An editing module is used to receive editing operations on the text content in the processed result image, and update the text content in the processed result image based on the editing operations.

[0153] In one optional embodiment, the apparatus further includes:

[0154] The display module is used to display a list of processed result images corresponding to the image template set; wherein the list of processed result images includes processed result images corresponding to each image template in the image template set, and the processed result images are generated based on the corresponding image template and the image to be processed.

[0155] In the image processing apparatus provided in this embodiment, firstly, an image to be processed is acquired, wherein the image to be processed includes one or more images. When an image processing operation triggered for the image to be processed is received, a processed result image is generated based on the image to be processed and an image template. The processed result image and the image template have a corresponding relationship, and the image template belongs to a set of image templates matched to the image to be processed based on template feature information.

[0156] After obtaining the image to be processed, the embodiments of this disclosure can automatically match an image template for the image to be processed based on the template feature information, and generate a processed result image based on the matched image template and the image to be processed, thereby enriching the image processing methods and improving the user's image processing interactive experience.

[0157] In addition to the methods and apparatus described above, this disclosure also provides a computer-readable storage medium storing instructions that, when executed on a terminal device, cause the terminal device to implement the image processing method described in this disclosure.

[0158] This disclosure also provides a computer program product, which includes a computer program / instructions that, when executed by a processor, implement the image processing method described in this disclosure.

[0159] In addition, this disclosure also provides an electronic device, see [link to relevant documentation]. Figure 6 As shown, it may include:

[0160] The electronic device includes a processor 601, a memory 602, an input device 603, and an output device 604. The number of processors 601 in the electronic device can be one or more. Figure 6 Taking a processor as an example. In some embodiments of this disclosure, the processor 601, memory 602, input device 603, and output device 604 can be connected via a bus or other means, wherein, Figure 6 Taking the example of a connection between China and Israel via a bus.

[0161] The memory 602 can be used to store software programs and modules. The processor 601 executes various functional applications and data processing of the electronic device by running the software programs and modules stored in the memory 602. The memory 602 may mainly include a program storage area and a data storage area. The program storage area may store the operating system, at least one application program required for a function, etc. In addition, the memory 602 may include high-speed random access memory, and may also include non-volatile memory, such as at least one disk storage device, flash memory device, or other volatile solid-state storage device. The input device 603 can be used to receive input digital or character information, and to generate signal inputs related to user settings and function control of the electronic device.

[0162] Specifically in this embodiment, the processor 601 loads the executable files corresponding to the processes of one or more applications into the memory 602 according to the following instructions, and the processor 601 runs the applications stored in the memory 602 to realize the various functions of the above-mentioned electronic device.

[0163] It should be noted that, in this document, relational terms such as "first" and "second" are used merely to distinguish one entity or operation from another, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Furthermore, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitations, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes said element.

[0164] The above description is merely a specific embodiment of this disclosure, enabling those skilled in the art to understand or implement it. Various modifications to these embodiments will be readily apparent to those skilled in the art, and the general principles defined herein may be implemented in other embodiments without departing from the spirit or scope of this disclosure. Therefore, this disclosure is not to be limited to the embodiments described herein, but is to be accorded the widest scope consistent with the principles and novel features disclosed herein.

Claims

1. An image processing method, characterized in that, The method includes: Acquire an image to be processed; wherein the image to be processed includes one or more images; In response to an image processing operation triggered for the image to be processed, a processing result image is generated based on the image to be processed and an image template; wherein, there is a correspondence between the processing result image and the image template, and the image template belongs to a set of image templates matched for the image to be processed based on template feature information.

2. The image processing method according to claim 1, characterized in that, The template feature information includes at least one of template image feature information, template text feature information, and template slot feature information. The template image feature information includes image feature information extracted from the image template. The template text feature information includes text feature information extracted from the text of the image template. The template slot feature information includes the number of slots in the image template and / or the image subject position information of each slot. The image subject position information is used to characterize the position information of the main object displayed on the filled image of the corresponding slot in the image template.

3. The image processing method according to claim 2, characterized in that, The step of generating a processed result image based on the image to be processed and an image template in response to an image processing operation triggered for the image to be processed includes: In response to an image processing operation triggered for the image to be processed, the matching feature information of the image to be processed is obtained; wherein, the matching feature information includes at least one of the image feature information of the image to be processed, subject location information, and number of images; By matching the feature information to be matched of the image to be processed with the template feature information in the template library, an image template set including at least one image template is determined from the template library for the image to be processed; wherein, the template library stores multiple image templates and template feature information corresponding to the multiple image templates respectively; A processed result image is generated based on the image to be processed and the image templates in the image template set.

4. The image processing method according to claim 3, characterized in that, The template feature information includes the template image feature information. The step of matching the feature information to be matched of the image to be processed with the template feature information in the template library to determine an image template set including at least one image template from the template library for the image to be processed includes: By matching the image feature information of the image to be processed with the template image feature information in the template feature information, a first candidate template set is determined for the image to be processed from the template library; wherein, the first candidate template set includes image templates that successfully match the image feature information of the image to be processed; From the first candidate template set, determine an image template set that includes at least one image template for the image to be processed.

5. The image processing method according to claim 3, characterized in that, The template feature information includes the template text feature information. The step of matching the feature information to be matched of the image to be processed with the template feature information in the template library to determine an image template set including at least one image template from the template library for the image to be processed includes: By matching the image feature information of the image to be processed with the template text feature information in the template feature information, a second candidate template set is determined for the image to be processed from the template library; wherein, the second candidate template set includes image templates whose template text feature information successfully matches the image feature information of the image to be processed; From the second candidate template set, determine an image template set that includes at least one image template for the image to be processed.

6. The image processing method according to claim 3, characterized in that, The template feature information includes the number of slots. The step of matching the feature information to be matched of the image to be processed with the template feature information in the template library to determine at least one image template for the image to be processed from the template library includes: By matching the number of slots in the image template with the number of images in the image to be processed, a third candidate template set is determined from the template library for the image to be processed; wherein, the third candidate template set includes image templates whose number of slots successfully matches the number of images in the image to be processed; From the third candidate template set, determine an image template set that includes at least one image template for the image to be processed.

7. The image processing method according to claim 6, characterized in that, The template feature information also includes the image subject position information of the slot. The step of determining an image template set including at least one image template from the third candidate template set for the image to be processed includes: By matching the subject position information of the image to be processed with the subject position information of the slots in the template feature information, a fourth candidate template set is determined for the image to be processed from the third candidate template set; wherein, the fourth candidate template set includes image templates whose number of slots and subject position information are both successfully matched with the number of images and subject position information of the image to be processed. From the fourth candidate template set, an image template set including at least one image template is determined for the image to be processed.

8. The image processing method according to claim 1, characterized in that, Before generating the processed result image based on the image to be processed and the image template, the method further includes: Text content is generated based on the image to be processed and the image template; wherein, the text content is used to reflect the image features of the image to be processed; The step of generating the processed result image based on the image to be processed and the image template includes: Based on the image to be processed, the text content, and the image template, a processed result image is generated.

9. The image processing method according to claim 1, characterized in that, After generating a processed result image based on the image to be processed and an image template in response to an image processing operation triggered for the image to be processed, the method further includes: The processed image is displayed within the image editing window; Receive editing operations on the text content in the processed result image, and update the text content in the processed result image based on the editing operations.

10. The image processing method according to claim 1, characterized in that, The method further includes: Display a list of processed result images corresponding to the image template set; wherein, the list of processed result images includes processed result images corresponding to each image template in the image template set, and the processed result images are generated based on the corresponding image template and the image to be processed.

11. An image processing apparatus, characterized in that, The device includes: An acquisition module is used to acquire an image to be processed; wherein, the image to be processed includes one or more images; The first generation module is configured to generate a processing result image based on the image to be processed and an image template in response to an image processing operation triggered for the image to be processed; wherein the processing result image and the image template have a corresponding relationship, and the image template belongs to a set of image templates matched for the image to be processed based on template feature information.

12. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores instructions that, when executed on a terminal device, cause the terminal device to perform the method as described in any one of claims 1-10.

13. An electronic device, characterized in that, include: A memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor, when executing the computer program, implements the method as claimed in any one of claims 1-10.

14. A computer program product, characterized in that, The computer program product includes a computer program / instruction that, when executed by a processor, implements the method as described in any one of claims 1-10.

Images