Image processing method and device

Abstract

The invention discloses an image processing method, which comprises the steps of obtaining a to-be-processed image and a target specification parameter, the target specification parameter being used for indicating a specification required to be satisfied by an image after the to-be-processed image is processed; and erasing characters in the to-be-processed image to obtain a first image. Further, edge region expansion is performed on the first image to obtain a second image conforming to the target specification parameter, and the second image comprises the first image and the expanded edge region. And then, inputting the second image into an image expansion model, generating image content for the edge region based on the first image by the image expansion model, and obtaining a third image conforming to the target specification parameter according to the first image and target image content generated by the image expansion model. And restoring the erased characters in the image area where the first image is located in the third image to obtain a target image. According to the scheme, the to-be-processed image is processed, and the visual experience of a user can be improved through the processed image.

Description

Technical Field

[0001] The present application relates to the field of image processing, and in particular to an image processing method and device. Background Art

[0002] In some scenarios, the image needs to be adjusted to a certain fixed specification, such as a certain fixed aspect ratio, or a certain fixed size.

[0003] Currently, the image can be adjusted to the above fixed specifications by directly scaling the image, or by cropping the image first and then scaling it. However, in this way, the adjusted image will be distorted, and accordingly, the visual experience brought to the user is poor.

[0004] Therefore, a solution is urgently needed to solve the above problems. Summary of the invention

[0005] In order to solve or at least partially solve the above technical problems, the present application provides an image processing method and device.

[0006] In a first aspect, the present application provides an image processing method, the method comprising:

[0007] Acquire an image to be processed and a target specification parameter, wherein the target specification parameter is used to indicate a specification that the image to be processed needs to meet after processing the image to be processed;

[0008] Erasing the text in the image to be processed to obtain a first image;

[0009] Expanding the edge region of the first image to obtain a second image that meets the target specification parameters, wherein the second image includes the first image and the expanded edge region;

[0010] Inputting the second image into an image expansion model, generating image content for the edge region based on the first image by the image expansion model, and obtaining a third image matching the target specification parameters according to the first image and the target image content generated by the image expansion model;

[0011] The erased text in the image area of ​​the third image where the first image is located is restored to obtain a target image.

[0012] Optionally, restoring the erased text in the image region of the third image where the first image is located to obtain the target image includes:

[0013] The image content of the image region in the third image where the first image is located is replaced by using the image in the image to be processed that corresponds to the first image, so as to obtain the target image.

[0014] Optionally, obtaining a third image according to the first image and the target image content generated by the image expansion model includes:

[0015] Performing a content check on the target image content to obtain a check result;

[0016] If the inspection result indicates that the target image content meets the first condition, a third image is obtained according to the target image content and the first image; or,

[0017] If the inspection result indicates that the target image content meets the second condition, the target image content is blurred, and a third image is obtained based on the first image and the target image content after the blurred process, wherein the strictness of the first condition is higher than that of the second condition.

[0018] Optionally, the performing content inspection on the target image content includes:

[0019] The target image content is detected using at least one subject detection model to determine whether the target image content includes a subject, and the at least one subject detection model includes at least one of a face detection model, a body detection model, and a text detection model.

[0020] Optional,

[0021] The target image content meets the first condition, including: the target image content does not include a subject;

[0022] The target image content meets the second condition, including: the content of the first area in the target image content and the content of the second area in the first image constitute a main body, and the ratio of the first area to the second area is less than or equal to a first ratio threshold.

[0023] Optionally, the method further includes:

[0024] Determine a subject area in the image to be processed;

[0025] Retain the main area to crop the image to be processed;

[0026] Erasing the text in the image to be processed includes:

[0027] Erase the text in the cropped image to be processed.

[0028] Optionally, the method further includes:

[0029] determining whether the subject area includes a preset subject, and when the subject area includes the preset subject, a possibility that new image content generated based on the subject area includes content that does not meet the requirements is greater than a certain threshold;

[0030] The step of retaining the main body area to crop the image to be processed includes:

[0031] When it is determined that the preset subject is not included in the subject area, the subject area is retained to crop the image to be processed.

[0032] Optionally, the target specification parameters include:

[0033] The ratio of the image length to the image width; or,

[0034] Image length and image width.

[0035] In a second aspect, the present application provides an image processing device, the device comprising:

[0036] A first acquisition unit, used for acquiring an image to be processed and a target specification parameter, wherein the target specification parameter is used for indicating a specification that the image to be processed needs to meet after processing the image to be processed;

[0037] An erasing unit, used for erasing the text in the image to be processed to obtain a first image;

[0038] an edge expansion unit, configured to expand an edge region of the first image to obtain a second image that meets the target specification parameters, wherein the second image includes the first image and the expanded edge region;

[0039] an input unit, configured to input the second image into an image expansion model, so that the image expansion model generates image content for the edge area based on the first image;

[0040] A first determining unit, configured to obtain a third image matching the target specification parameters according to the first image and the target image content generated by the image expansion model;

[0041] The second determining unit is used to restore the erased text in the image area of ​​the third image where the first image is located to obtain a target image.

[0042] Optionally, the second determining unit is configured to:

[0043] The image content of the image region in the third image where the first image is located is replaced by using the image in the image to be processed that corresponds to the first image, so as to obtain the target image.

[0044] Optionally, the first determining unit is configured to:

[0045] Performing a content check on the target image content to obtain a check result;

[0046] If the inspection result indicates that the target image content meets the first condition, a third image is obtained according to the target image content and the first image; or,

[0047] If the inspection result indicates that the target image content meets the second condition, the target image content is blurred, and a third image is obtained based on the first image and the target image content after the blurred process, wherein the strictness of the first condition is higher than that of the second condition.

[0048] Optionally, the performing content inspection on the target image content includes:

[0049] The target image content is detected using at least one subject detection model to determine whether the target image content includes a subject, and the at least one subject detection model includes at least one of a face detection model, a body detection model, and a text detection model.

[0050] Optional,

[0051] The target image content meets the first condition, including: the target image content does not include a subject;

[0052] The target image content meets the second condition, including: the content of the first area in the target image content and the content of the second area in the first image constitute a main body, and the ratio of the first area to the second area is less than or equal to a first ratio threshold.

[0053] Optionally, the device further comprises:

[0054] A third determining unit, used to determine a subject area in the image to be processed;

[0055] A cropping unit, used for cropping the image to be processed while retaining the main body area;

[0056] The erasing unit is used for:

[0057] Erase the text in the cropped image to be processed.

[0058] Optionally, the device further comprises:

[0059] A fourth determining unit, configured to determine whether the subject area includes a preset subject, and when the subject area includes the preset subject, a possibility that the new image content generated based on the subject area includes content that does not meet the requirements is greater than a certain threshold;

[0060] The cutting unit is used for:

[0061] When it is determined that the preset subject is not included in the subject area, the subject area is retained to crop the image to be processed.

[0062] Optionally, the target specification parameters include:

[0063] The ratio of the image length to the image width; or,

[0064] Image length and image width.

[0065] In a third aspect, an embodiment of the present application provides an electronic device, the electronic device comprising a processor and a memory;

[0066] The processor is used to execute instructions stored in the memory so that the electronic device performs the method as described in any one of the first aspects above.

[0067] In a fourth aspect, an embodiment of the present application provides a computer-readable storage medium, comprising instructions, wherein the instructions instruct a device to execute a method as described in any one of the above first aspects.

[0068] In a fifth aspect, an embodiment of the present application provides a computer program product, which, when executed on a computer, enables the computer to execute any of the methods described in the first aspect above.

[0069] Compared with the prior art, the embodiments of the present application have the following advantages:

[0070] The present application provides an image processing method, including: obtaining an image to be processed and a target specification parameter, the target specification parameter being used to indicate the specification that the image to be processed needs to meet after the image to be processed is processed. Considering that the image to be processed is processed, it is often necessary to generate new image content at the edge of the image to be processed in combination with the content of the image to be processed. In order to avoid that the generated new image content does not meet the requirements, after the image to be processed is obtained, the text in the image to be processed can be erased to obtain a first image. Further, the edge area of ​​the first image is expanded to obtain a second image that meets the target specification parameter, and the second image includes the first image and the expanded edge area. Then, the second image is input into an image expansion model, and the image expansion model generates image content for the edge area based on the first image, and a third image that meets the target specification parameter is obtained according to the first image and the target image content generated by the image expansion model. Further, the text erased in the image area where the first image is located in the third image is restored to obtain the target image. It can be seen that, by using this solution, when generating image content for the aforementioned edge area, the text in the image to be processed will no longer be combined, thereby avoiding the generation of image content that does not meet the requirements (such as strange text) for the edge area. Accordingly, by using this solution to process the image to be processed, the processed image can enhance the user's visual experience. BRIEF DESCRIPTION OF THE DRAWINGS

[0071] In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments recorded in the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0072] Figure 1 A flowchart of an image processing method provided in an embodiment of the present application;

[0073] Figure 2 A flowchart of another image processing method provided in an embodiment of the present application;

[0074] Figure 3 A schematic diagram of the structure of an image processing device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0075] In order to enable those skilled in the art to better understand the solution of the present application, the technical solution in the embodiments of the present application will be clearly and completely described below in conjunction with the drawings in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of this application.

[0076] The inventor of the present application has found through research that there are currently some artificial intelligence drawing solutions based on diffusion models, which can redraw a part of an existing image (inpainting). Specifically, they can redraw a part of an existing image in combination with the content of the existing image. When the redrawing area is the outer edge of the image, the outward expansion (outpainting) of the existing image can be achieved.

[0077] However, when the diffusion model is currently used to perform the outpainting task, the visual effect of the generated image still cannot meet the requirements, especially when the existing image includes text, the image content generated by the diffusion model will include strange text.

[0078] In order to solve the above problems, the present application provides an image processing method and device.

[0079] Various non-limiting implementations of the present application are described in detail below in conjunction with the accompanying drawings.

[0080] Exemplary Methods

[0081] See also Figure 1 , which is a flow chart of an image processing method provided in an embodiment of the present application. The method provided in an embodiment of the present application can be applied to a client, and of course, can also be applied to a server, which is not specifically limited in the embodiment of the present application. In the following description, the method is applied to a client as an example for explanation.

[0082] In the embodiment of the present application, the method may include the following steps: S101-S105.

[0083] S101: Acquire an image to be processed and target specification parameters, where the target specification parameters are used to indicate specifications that the image to be processed needs to meet after processing the image to be processed.

[0084] In one example, the target specification parameter may be a ratio of image length to image width, for example, 4:3. In another example, the target specification parameter may be a specific image size, in which case the target specification parameter may include image length and image width.

[0085] In one example, the target specification parameter may be input by a user. For example, the page displayed by the client may include an image specification parameter input area, and the user may input the target specification parameter in the image specification parameter input area.

[0086] In one example, the image to be processed may be an image input (i.e., uploaded) by a user, or an image obtained by processing an image input by a user. As a specific example, in addition to the aforementioned image specification parameter input area, the page displayed by the client may also include an image upload control, through which the user may trigger an image upload operation, thereby uploading the corresponding image.

[0087] In one implementation of the embodiment of the present application, the aforementioned image to be processed may be an image that does not include a border of a specific color (e.g., black). In one example, for an image input by a user, the client may perform border detection on the image. If it is detected that the image input by the user includes a border of a specific color, the client may perform a border cropping operation on the image input by the user, thereby obtaining an image to be processed that does not include a border of the specific color. If it is detected that the image input by the user does not include a border of the specific color, the image input by the user may be directly used as the image to be processed.

[0088] The embodiments of the present application do not specifically limit the specific implementation of border detection of an image. In one example, for example, a border detection model can be used to detect the border of the image. In addition, after detecting that the image includes a border, for example, the color of the border can be determined. If the border is a solid color and the color is the aforementioned specific color, it can be determined that the image includes a border of the aforementioned specific color.

[0089] S102: Identify text in the image to be processed, and erase the text from the image to be processed to obtain a first image.

[0090] After obtaining the image to be processed, the image to be processed is further processed to obtain a target image that meets the target specification parameters. Specifically, the edge area of ​​the image to be processed can be expanded so that the expanded image meets the target specification parameters, and in order to enable the expanded image to bring a better visual experience, new image content can also be generated for the expanded edge area based on the image to be processed.

[0091] In this application, if the image to be processed includes text, the image content generated based on the edge area of ​​the image to be processed may include strange text, which may bring a bad visual experience to the user. Therefore, before expanding the edge area of ​​the image to be processed, the text in the image to be processed can be erased to obtain a first image, so as to facilitate the subsequent expansion of the edge area based on the first image.

[0092] In a specific example, the text recognition model can be used to first identify the text in the image to be processed. Then, the identified text is erased from the image to be processed. For example, the text erasure model can be used to erase the identified text from the image to be processed, thereby obtaining a first image. In another specific example, if the text erasure model also has text recognition capabilities, the image to be processed can be directly input into the text erasure model, and the text erasure model recognizes and erases the text in the image to be processed, thereby obtaining a first image. The text erasure model mentioned here includes but is not limited to a generative adversarial network (GAN) model.

[0093] In one example, in order to ensure that the subject in the image to be processed is still the subject in the target image, before executing S102, the following steps A1-A2 may be further executed.

[0094] Step A1: Determine the main body area in the image to be processed.

[0095] In one example, at least one model may be used to perform subject detection on an image to be processed to determine a subject area in the image to be processed. The at least one model may, for example, include at least one subject detection model and / or a saliency detection model. The saliency detection model is used to detect salient areas in an image. In one example, the at least one subject detection model may, for example, include one or more of a face detection model, a human body detection model, and a text detection model.

[0096] in:

[0097] The face detection model is used to detect whether an image includes a face. It can output the coordinates of the vertices (such as the vertices of a rectangular box) of the area where the detected face is located, and the probability that the area is a face.

[0098] The human body detection model is used to detect whether an image includes a human body. It can output the coordinates of multiple points (such as vertices of a rectangular box) in the area where the detected human body is located, as well as the probability that the area is a human body.

[0099] The text detection model is used to detect whether an image includes text. It can output the coordinates of the vertices (such as the vertices of a rectangular box) of the area where the detected text is located, and the probability that the area contains text.

[0100] In the present application, after performing subject detection on the image to be processed using the aforementioned at least one model, the subject area in the image to be processed can be determined based on the detection result of the at least one model.

[0101] As an example, it may be determined that the main region includes a salient region detected by a saliency detection model.

[0102] As an example, when the face recognition model recognizes that the image to be processed includes a face, it can be determined that the main area includes the area where the face is located.

[0103] As another example, when the human body recognition model recognizes that the image to be processed includes a human body, it can be determined that the main area includes the area where the human body is located.

[0104] As another example, when the text recognition model recognizes that the image to be processed includes text and the number of text reaches a certain number, it can be determined that the main area includes the area where the text is located.

[0105] Step A2: cropping the image to be processed while retaining the main area.

[0106] After determining the main body area, in one example, the image to be processed can be cropped based on the main body area. Specifically, the area other than the main body area in the image to be processed can be cropped, and the main body area in the image to be processed can be retained. In this case, S102 can be implemented by erasing the text in the cropped image to be processed to obtain the first image.

[0107] In another example, considering that if the aforementioned main body area includes a preset subject, when edge expansion is performed based on the main body area, it is possible that when image content is generated for the expanded edge area, the generated image content includes content that does not meet the requirements. In other words, if the main body area includes the preset subject, the possibility that the new image content generated based on the main body area includes content that does not meet the requirements is greater than a certain threshold. Correspondingly, if the main body area includes the preset subject, then: when edge expansion is performed based on the first image obtained from the main body area and image content is generated for the expanded edge area, the possibility that the generated image content includes content that does not meet the requirements is also relatively large, resulting in a poor visual experience brought by the final target image.

[0108] In view of this, in one example, after executing the aforementioned step A1, it is further possible to determine whether the aforementioned main body area includes a preset main body. If the main body area includes the preset main body, it can be determined that the main body area is not suitable for further edge expansion, and at this time, the image processing flow can be exited. Correspondingly, if the main body area does not include the preset main body, step A2 and subsequent steps are further executed.

[0109] The embodiments of the present application do not specifically limit the preset subject. The preset subject may be, for example, an incomplete subject, such as half a face, half a body, etc., which will not be described in detail here.

[0110] S103: Expanding the edge region of the first image to obtain a second image that meets the target specification parameters, where the second image includes the first image and the expanded edge region.

[0111] After executing S102 to obtain the first image, the size of the edge region that needs to be expanded to obtain the first image to meet the target specification parameters can be determined based on the size of the first image and the target specification parameters, and then, based on the determined size of the edge region, the edge region of the first image is expanded to obtain a second image that meets the target specification parameters. It is not difficult to understand that the second image includes the first image and the expanded edge region.

[0112] S104: Inputting the second image into an image expansion model, allowing the image expansion model to generate image content for the edge area based on the first image, and obtaining a third image matching the target specification parameters according to the first image and the target image content generated by the image expansion model.

[0113] After obtaining the second image, in order to make the image content of the expanded edge area included in the second image more coordinated with the image content of the first image, thereby enhancing the visual experience brought to the user by the entire image, the second image can be input into an image expansion model. The image expansion model mentioned here can be a stable diffusion model. The image expansion model can be used to generate new image content based on existing image content. Therefore, after inputting the second image into the image expansion model, the image expansion model can generate new image content for the aforementioned edge area included in the second image based on the image content of the first image. For the convenience of description, the image content generated by the image expansion model for the aforementioned edge area included in the second image is referred to as "target image content". Then:

[0114] After obtaining the target image content, a third image can be obtained based on the first image and the target image content. The specifications of the third image are the same as those of the second image, and the difference between the two is that the content of the edge area in the third image is different from the content of the edge area in the second image. Therefore, the third image is also an image that meets the aforementioned target specification parameters.

[0115] In one example, in order to ensure that the obtained third image can bring a better visual experience to the user, after the second image is input into the image expansion model and the target image content is obtained, the target image content can be checked to obtain the inspection result, and further, based on the inspection result, it is determined how to obtain the third image based on the aforementioned target image content and the first image. In one example, multiple conditions can be set in advance, and after obtaining the inspection result, it is determined how to obtain the third image based on the aforementioned target image content and the first image according to the conditions matched by the inspection result.

[0116] In one example, the aforementioned multiple conditions may include a first condition and a second condition, wherein the strictness of the first condition is higher than the strictness of the second condition. For this case:

[0117] If the aforementioned check result matches the first condition, it means that the target image content meets the requirements, and the new image composed of the target image and the first image will not bring visual impact to the user. In this case, a third image including the first image and the target image content can be directly obtained based on the target image content and the first image. Among them, the aforementioned check result matches the first condition, which can be understood as the check result indicating that the target image content meets the first condition.

[0118] If the aforementioned inspection result matches the second condition, it means that the target image content may not meet the requirements, and the new image composed of the target image and the first image may bring visual impact to the user. In this case, the target image content can be blurred first, for example, Gaussian blurring the target image content. Furthermore, based on the first image and the target image content after blurring, a third image including the first image and the target image content after blurring is obtained. Among them, the aforementioned inspection result matches the second condition, which can be understood as the inspection result indicating that the target image content meets the second condition.

[0119] In one example, if the aforementioned inspection result indicates that the target image content does not meet the second condition, it means that the target image content has content that does not meet the requirements. In this case, the image processing flow can be directly exited.

[0120] The embodiments of the present application do not specifically limit the aforementioned multiple conditions. In one example, considering that if the newly generated image content (i.e., the target image content) includes a new subject, it may bring a bad visual experience to the user. Therefore, the aforementioned multiple conditions may be related to whether the target image content includes a subject. In this case, the aforementioned content check of the target image content may be to detect the target image content using at least one subject detection model to determine whether the target image content includes a subject. In one example, the image composed of the first image and the target image content can be directly input into the aforementioned at least one subject detection model, respectively, to detect whether the target image content includes a subject. The at least one subject detection model mentioned here may be at least one of the aforementioned face detection model, human body detection model, and text detection model, and will not be repeated here.

[0121] When the multiple conditions are related to whether the target image content includes a subject, in one example, the aforementioned first condition, for example, may be that the target image content does not include a subject. Correspondingly, the second condition may be that the target image content includes a small part of a complete subject. Specifically, the second condition may be that the content of the first area in the target image content and the content of the second area in the first image constitute a subject, and the ratio of the first area to the second area is less than or equal to the first ratio threshold, that is, the first area is smaller than the second area. Among them, the first ratio threshold may be a value less than 1, that is, in the present application: in the case where the target image content does not include a subject, the third image can be obtained directly based on the target image content and the first image. In the case where the target image content includes only a small part of the complete subject, the target image content can be blurred, and the third image can be obtained based on the first image and the target image content after the blurred processing, so as to minimize the visual impact on the user.

[0122] S105: Restoring the erased text in the image area of ​​the third image where the first image is located to obtain a target image.

[0123] As described above, the first image is a part of the third image, and the first image is obtained by erasing the text in the image to be processed. Therefore, after obtaining the third image, the erased text in the image area of ​​the third image where the first image is located can be restored, so that the obtained target image can retain the text in the image to be processed.

[0124] In a specific example, when S105 is implemented, based on the third image and the image to be processed, the erased text may be regenerated for the image area in the third image where the first image is located.

[0125] In another specific example, when S105 is implemented, the image corresponding to the first image in the image to be processed can be used to replace the image content of the image area in the third image where the first image is located, so as to obtain the target image. For example, when the image to be processed is not cropped (that is, the client has not executed the aforementioned steps A1-A2), the image to be processed can be directly used to replace the image content of the image area in the third image where the first image is located, so as to obtain the target image. For another example, when the image to be processed is cropped (that is, the client has executed the aforementioned steps A1-A2), the image to be processed after cropping can be used to replace the image content of the image area in the third image where the first image is located, so as to obtain the target image. In this way, the generated target content can retain the content of the image input by the user as much as possible, and the image resolution can be kept consistent with the image resolution input by the user.

[0126] From the above description, it can be seen that, using the solution provided in the embodiment of the present application, when the image to be processed is processed into a target image that meets the target specification parameters, the image content of the edge area will not be generated in combination with the text in the image to be processed, thereby avoiding the generation of image content that does not meet the requirements (such as strange text) for the edge area. Accordingly, when the image to be processed is processed using this solution, the processed image can enhance the user's visual experience.

[0127] The above is an introduction to the image processing method provided in the embodiment of the present application. Figure 2 , introduces a possible implementation method of the embodiment of the present application. Figure 2 A flowchart of another image processing method provided in an embodiment of the present application. Figure 2 The method shown includes the following steps S201-S215.

[0128] S201: Acquire an image input by a user.

[0129] S202: Detect black borders and crop the black borders.

[0130] The image obtained after executing S202 may correspond to the image to be processed in the above embodiment.

[0131] S203: Determine whether the target specification parameters are met.

[0132] After executing S202 to obtain the image to be processed, further execute S203 based on the image to be processed. If the image to be processed meets the target specification parameters, directly output the image obtained by executing S202. If the image to be processed does not meet the target specification parameters, continue to execute S204.

[0133] S204: Main body area detection.

[0134] Regarding S204, please refer to the description of step A1 above, and no repeated description is given here.

[0135] S205: Determine whether the preset subject is included.

[0136] S205 refers to determining whether the subject area detected in S204 includes a preset subject.

[0137] If the subject area detected in S204 includes the preset subject, the image processing flow is exited.

[0138] If the subject area detected in S204 does not include the preset subject, then continue to execute S206.

[0139] S206: retaining the main area for cropping.

[0140] S207: Determine whether the target specification parameters are met.

[0141] S207 refers to determining whether the image obtained by executing S206 meets the target specification parameters. If it meets the target specification parameters, the image obtained by executing S206 is directly output. If it does not meet the target specification parameters, S208 is continued.

[0142] S208: Erase the image text.

[0143] S208 refers to erasing the text in the image obtained in S206, and the obtained image may correspond to the first image in the above embodiment.

[0144] S209: Edge extension, input image expansion model.

[0145] S209 may correspond to S103 and S104 in the above embodiment. For specific implementation, reference may be made to the relevant description of S103 and S104 in the above text, and no repeated description is given here.

[0146] S210: Check the expanded image content.

[0147] The expanded image content mentioned here may correspond to the “target image content” in the above embodiment. Accordingly, the expanded image content check mentioned here may correspond to the “content check of the target image content” in the above embodiment.

[0148] S211: Check the result and match it with the preset conditions.

[0149] The inspection result mentioned here refers to the result obtained by the expanded image inspection in S210.

[0150] If the check result matches the first condition, S212 is executed, and if the check result matches the second condition, S213 is executed.

[0151] S212: Directly adopt the expanded image content.

[0152] S213: Blurring the expanded image content.

[0153] S212 refers to obtaining a third image directly based on the target image content and the first image, and S213 refers to performing blur processing on the target image content and obtaining a third image based on the first image and the target image content after blur processing.

[0154] S214: Paste the original image back to obtain the target image.

[0155] S214 refers to replacing the image content of the image region in the third image where the first image is located with the image in the image to be processed corresponding to the first image, so as to obtain the target image.

[0156] S215: Output the target image.

[0157] Using this solution, users only need to input image and target specification parameters to obtain the corresponding target image.

[0158] Exemplary Devices

[0159] Based on the method provided in the above embodiment, the embodiment of the present application further provides a device, which is described below in conjunction with the accompanying drawings.

[0160] See also Figure 3 , which is a structural schematic diagram of an image processing device provided in an embodiment of the present application. Figure 3 The image processing device shown in FIG. 1 can be used to execute the image processing method provided by the above method embodiment. Figure 3 As shown, the device 300 may specifically include, for example: a first acquisition unit 301 , an erasing unit 302 , an edge expansion unit 303 , an input unit 304 , a first determination unit 305 , and a second determination unit 306 .

[0161] A first acquisition unit 301 is used to acquire an image to be processed and a target specification parameter, wherein the target specification parameter is used to indicate a specification that the image to be processed needs to meet after processing the image to be processed;

[0162] An erasing unit 302, configured to erase the text in the image to be processed to obtain a first image;

[0163] an edge expansion unit, configured to expand an edge region of the first image to obtain a second image that meets the target specification parameters, wherein the second image includes the first image and the expanded edge region;

[0164] An input unit 303, configured to input the second image into an image expansion model, so that the image expansion model generates image content for the edge region based on the first image;

[0165] A first determining unit 304 is configured to obtain a third image matching the target specification parameters according to the first image and the target image content generated by the image expansion model;

[0166] The second determining unit 305 is used to restore the erased text in the image area of ​​the third image where the first image is located to obtain a target image.

[0167] Optionally, the second determining unit 305 is configured to:

[0168] The image content of the image region in the third image where the first image is located is replaced by using the image in the image to be processed that corresponds to the first image, so as to obtain the target image.

[0169] Optionally, the first determining unit 304 is configured to:

[0170] Performing a content check on the target image content to obtain a check result;

[0171] If the inspection result indicates that the target image content meets the first condition, a third image is obtained according to the target image content and the first image; or,

[0172] If the inspection result indicates that the target image content meets the second condition, the target image content is blurred, and a third image is obtained based on the first image and the target image content after the blurred process, wherein the strictness of the first condition is higher than that of the second condition.

[0173] Optionally, the performing content inspection on the target image content includes:

[0174] The target image content is detected using at least one subject detection model to determine whether the target image content includes a subject, and the at least one subject detection model includes at least one of a face detection model, a body detection model, and a text detection model.

[0175] Optional,

[0176] The target image content meets the first condition, including: the target image content does not include a subject;

[0177] The target image content meets the second condition, including: the content of the first area in the target image content and the content of the second area in the first image constitute a main body, and the ratio of the first area to the second area is less than or equal to a first ratio threshold.

[0178] Optionally, the device further comprises:

[0179] A third determining unit, used to determine a subject area in the image to be processed;

[0180] A cropping unit, used for cropping the image to be processed while retaining the main body area;

[0181] The erasing unit 302 is used for:

[0182] Erase the text in the cropped image to be processed.

[0183] Optionally, the device further comprises:

[0184] A fourth determining unit, configured to determine whether the subject area includes a preset subject, and when the subject area includes the preset subject, a possibility that the new image content generated based on the subject area includes content that does not meet the requirements is greater than a certain threshold;

[0185] The cutting unit is used for:

[0186] When it is determined that the preset subject is not included in the subject area, the subject area is retained to crop the image to be processed.

[0187] Optionally, the target specification parameters include:

[0188] The ratio of the image length to the image width; or,

[0189] Image length and image width.

[0190] Since the device 300 is a device corresponding to the image processing method provided by the above method embodiment, the specific implementation of each unit of the device 300 is based on the same concept as the above method embodiment. Therefore, regarding the specific implementation of each unit of the device 300, please refer to the relevant description part of the above method embodiment, which will not be repeated here.

[0191] The embodiment of the present application also provides an electronic device, the electronic device comprising a processor and a memory;

[0192] The processor is used to execute the instructions stored in the memory, so that the electronic device executes the image processing method provided by the above method embodiment.

[0193] An embodiment of the present application provides a computer-readable storage medium, including instructions, wherein the instructions instruct a device to execute the image processing method provided by the above method embodiment.

[0194] The embodiment of the present application also provides a computer program product. When the computer program product is run on a computer, the computer is enabled to execute the image processing method provided by the above method embodiment.

[0195] Those skilled in the art will readily appreciate other embodiments of the present application after considering the specification and practicing the invention disclosed herein. The present application is intended to cover any modification, use or adaptation of the present application, which follows the general principles of the present application and includes common knowledge or customary techniques in the art that are not disclosed in the present disclosure. The specification and examples are intended to be exemplary only, and the true scope and spirit of the present application are indicated by the following claims.

[0196] It should be understood that the present application is not limited to the precise structures that have been described above and shown in the drawings, and that various modifications and changes may be made without departing from the scope thereof. The scope of the present application is limited only by the appended claims.

[0197] The above description is only a preferred embodiment of the present application and is not intended to limit the present application. Any modifications, equivalent substitutions, improvements, etc. made within the spirit and principles of the present application should be included in the protection scope of the present application.

Claims

1. An image processing method, characterized in that: The method comprises: Acquire an image to be processed and a target specification parameter, wherein the target specification parameter is used to indicate a specification that the image to be processed needs to meet after processing the image to be processed; Erasing the text in the image to be processed to obtain a first image; Expanding the edge region of the first image to obtain a second image that meets the target specification parameters, wherein the second image includes the first image and the expanded edge region; Inputting the second image into an image expansion model, generating image content for the edge region based on the first image by the image expansion model, and obtaining a third image matching the target specification parameters according to the first image and the target image content generated by the image expansion model; The erased text in the image area of ​​the third image where the first image is located is restored to obtain a target image.

2. The method according to claim 1, characterized in that The step of restoring the erased text in the image region of the third image where the first image is located to obtain a target image includes: The target image is obtained by using an image in the image to be processed that corresponds to the first image to replace the image content of the image region in the third image where the first image is located.

3. The method according to claim 1, characterized in that: The step of obtaining a third image according to the first image and the target image content generated by the image expansion model comprises: Performing a content check on the target image content to obtain a check result; If the inspection result indicates that the target image content meets the first condition, a third image is obtained according to the target image content and the first image; or, If the inspection result indicates that the target image content meets the second condition, the target image content is blurred, and a third image is obtained based on the first image and the target image content after the blurred process, wherein the strictness of the first condition is higher than that of the second condition.

4. The method according to claim 3, characterized in that The checking the content of the target image includes: The target image content is detected using at least one subject detection model to determine whether the target image content includes a subject, and the at least one subject detection model includes at least one of a face detection model, a body detection model, and a text detection model.

5. The method according to claim 3 or 4, characterized in that: The target image content meets the first condition, including: the target image content does not include a subject; The target image content meets the second condition, including: the content of the first area in the target image content and the content of the second area in the first image constitute a main body, and the ratio of the first area to the second area is less than or equal to a first ratio threshold.

6. The method according to claim 1, characterized in that The method further comprises: Determine a subject area in the image to be processed; Retain the main area to crop the image to be processed; Erasing the text in the image to be processed includes: Erase the text in the cropped image to be processed.

7. The method according to claim 6, characterized in that The method further comprises: determining whether the subject area includes a preset subject, and when the subject area includes the preset subject, a possibility that new image content generated based on the subject area includes content that does not meet the requirements is greater than a certain threshold; The step of retaining the main body area to crop the image to be processed includes: When it is determined that the preset subject is not included in the subject area, the subject area is retained to crop the image to be processed.

8. The method according to claim 1, characterized in that The target specification parameters include: The ratio of the image length to the image width; or, Image length and image width.

9. An image processing device, characterized in that: The device comprises: A first acquisition unit, used for acquiring an image to be processed and a target specification parameter, wherein the target specification parameter is used for indicating a specification that the image to be processed needs to meet after processing the image to be processed; An erasing unit, used for erasing the text in the image to be processed to obtain a first image; an edge expansion unit, configured to expand an edge region of the first image to obtain a second image that meets the target specification parameters, wherein the second image includes the first image and the expanded edge region; an input unit, configured to input the second image into an image expansion model, so that the image expansion model generates image content for the edge area based on the first image; A first determining unit, configured to obtain a third image matching the target specification parameters according to the first image and the target image content generated by the image expansion model; The second determining unit is used to restore the erased text in the image area of ​​the third image where the first image is located to obtain a target image.

10. An electronic device, characterized in that: The electronic device comprises a processor and a memory; The processor is used to execute the instructions stored in the memory so that the electronic device executes the method according to any one of claims 1 to 8.

11. A computer-readable storage medium, characterized in that: The method comprises instructions, wherein the instructions instruct the device to execute the method according to any one of claims 1 to 8.

Images