Image processing method, storage medium and electronic device based on large model

Abstract

The present application discloses an image processing method, storage medium and electronic device based on a large model, the method comprising: obtaining a segmented image including a target element and a background element according to the segmentation processing of the image to be processed; wherein the background element comprises a first pixel and the target element comprises a second pixel; determining whether there is a defective area at the edge position of the segmented image according to the first pixel at the edge position of the segmented image and / or the second pixel at the edge position of the target element; wherein the edge position of the segmented image corresponds to the edge position of the target element; if so, performing image restoration on the defective area in the image to be processed by using the large model to generate a target image corresponding to the image to be processed. In this way, when the target element in the image to be processed is defective, it can be automatically restored, and the restored image can clearly and completely express the target element.

Description

Technical Field

[0001] The present application relates to the field of artificial intelligence, and more particularly to a method and device for image processing based on a large model. The present application also relates to a method for acquiring a target image, a method for processing a commodity image based on a large model, a computer storage medium and electronic device, and a computer program product. Background Art

[0002] Image processing refers to the technical process of analyzing, processing and manipulating digital images, aiming to improve the visual effect of the image or extract some useful information from it to adapt and use it in different scenarios. Image processing can improve the visual effect of the image and extract useful information. The image processing process can include image segmentation, feature extraction and recognition, restoration, enhancement and other aspects.

[0003] Image segmentation is the process of dividing an image into multiple non-overlapping regions, each of which satisfies certain specific properties, in order to identify and distinguish different objects or regions in the image, providing a basis for subsequent feature extraction and recognition.

[0004] Image feature extraction and recognition is to extract useful information from images, such as edges, corners, textures, etc., or to identify specific objects or scenes in images.

[0005] Image restoration is the process of repairing and reconstructing images that may be damaged or distorted during the acquisition or transmission process.

[0006] Image enhancement is an important part of image processing, which focuses on improving the visual effect of images, making them clearer, easier to observe and understand. This is achieved by adjusting the image's contrast, brightness, color and other attributes, or by using more complex algorithms to remove noise, sharpen edges, etc.

[0007] With the rapid development of computer vision and artificial intelligence technology, image processing technology is also constantly improving and innovating. Advanced technologies such as deep learning and convolutional neural networks are widely used in the field of image processing, greatly improving the efficiency and accuracy of image processing. Summary of the invention

[0008] The present application provides an image processing method based on a large model to solve the problems in the prior art of distorted generated images and poor user experience.

[0009] The present application provides an image processing method based on a large model, comprising:

[0010] According to the segmentation process of the image to be processed, a segmented image including a target element and a background element is obtained; wherein the background element includes a first pixel, and the target element includes a second pixel;

[0011] Determine whether there is a defective area of ​​the target element at the edge position of the segmented image according to the first pixel at the edge position of the segmented image and / or the second pixel at the edge position of the target element; wherein the edge position of the segmented image corresponds to the edge position of the target element;

[0012] If so, the incomplete area in the image to be processed is repaired using the large model to generate a target image corresponding to the image to be processed.

[0013] In some embodiments, determining whether the target element has a defective area at the edge position of the segmented image according to the first pixel at the edge position of the segmented image and / or the second pixel at the edge position of the target element includes:

[0014] Acquire the distribution data of the first pixel at the edge position of the segmented image or the distribution data of the second pixel at the edge position of the target element;

[0015] According to the distribution data of the first pixel or the distribution data of the second pixel, it is determined whether there is a defective area of ​​the target element at the edge position of the segmented image.

[0016] In some embodiments, determining whether there is a defective area of ​​the target element at the edge position of the segmented image according to the distribution of the first pixels at the edge position or the distribution of the second pixels at the edge position includes:

[0017] Determining whether there is a defective area of ​​the target element at the edge position of the segmented image according to the proportion of the first pixel at the edge position of the segmented image;

[0018] or,

[0019] According to the proportion of the second pixels at the edge position of the segmented image, it is determined whether there is a defective area of ​​the target element at the edge position of the segmented image.

[0020] In some embodiments, determining whether the target element has a defective area at the edge position of the segmented image according to the first pixel at the edge position of the segmented image and / or the second pixel at the edge position of the target element includes:

[0021] Acquire a pixel point of the first pixel and a pixel point of the second pixel; wherein the pixel point of the first pixel and the pixel point of the second pixel correspond to each other;

[0022] It is determined whether there is a defective area of ​​the target element at an edge position of the segmented image according to the distance from the pixel point of the first pixel to the pixel point of the second pixel.

[0023] In some embodiments, performing image restoration on the incomplete area in the image to be processed by using a large model to generate a target image corresponding to the image to be processed includes:

[0024] Determining a defect direction in the image to be processed according to the defect area;

[0025] The large model uses the defective direction as a designated repair direction, and expands the image to be processed according to a designated size along the designated repair direction to generate an expanded image;

[0026] Filling the extended image according to the image to be processed to generate a filled image;

[0027] According to the image to be processed, repairing the filled image to generate a repaired image;

[0028] The repaired image and the image to be processed are fused to generate a target image corresponding to the image to be processed.

[0029] In some embodiments, the large model uses the defective direction as a designated repair direction, and expands the image to be processed according to a designated size along the designated repair direction to generate an expanded image, including:

[0030] The large model expands the image to be processed along the specified repair direction according to a specified step size or a specified extension ratio to generate an expanded image.

[0031] In some embodiments, filling the extended image according to the image to be processed to generate a filled image includes:

[0032] According to the selection method, a target image is selected at the edge of the image to be processed;

[0033] The extended image is filled according to the target image to generate the filled image.

[0034] In some embodiments, it also includes:

[0035] Performing restoration result verification on the target image;

[0036] When the verification result meets the storage requirement, the target image is stored.

[0037] The present application also provides a target image acquisition method, comprising:

[0038] Select the image to be processed;

[0039] Obtain a target image corresponding to the image to be processed, which is determined according to the above-mentioned large model-based image processing method.

[0040] The present application also provides a commodity image processing method based on a large model, comprising:

[0041] According to the segmentation processing of the commodity image to be processed, a segmented image including a target commodity element and a background element is obtained; wherein the background element includes a first pixel, and the target commodity element includes a second pixel;

[0042] Determine whether there is a defective area of ​​the target product element at the edge position of the segmented image according to the first pixel at the edge position of the segmented image and / or the second pixel at the edge position of the target product element; wherein the edge position of the segmented image corresponds to the edge position of the target product element;

[0043] If so, the incomplete area in the product image to be processed is repaired using the large model to generate a target product image corresponding to the product image to be processed.

[0044] The present application also provides a computer storage medium for storing data generated by a network platform, and a program for processing the data generated by the network platform;

[0045] When the program is read and executed by the processor, it executes the image processing method based on the large model as described above, or executes the target image acquisition method as described above, or executes the commodity image processing method based on the large model as described above.

[0046] The present application also provides an electronic device, comprising:

[0047] processor;

[0048] The memory is used to store a program for processing data generated by a network platform. When the program is read and executed by the processor, it executes the image processing method based on the large model as mentioned above, or executes the target image acquisition method as mentioned above, or executes the commodity image processing method based on the large model as mentioned above.

[0049] Compared with the prior art, this application has the following advantages:

[0050] The present application provides an image processing method based on a large model, which can perform batch processing and automatically complete the processing process during the image processing process, that is, through the input image to be processed, according to the segmentation processing of the image to be processed, a segmented image including a target element and a background element is obtained; wherein the background element includes a first pixel, and the target element includes a second pixel; according to the first pixel at the edge position of the segmented image and / or the second pixel at the edge position of the target element, it is determined whether the target element has a defective area at the edge position of the segmented image; wherein the edge position of the segmented image corresponds to the edge position of the target element; if so, the defective area in the image to be processed is image-repaired through the large model to generate a target image corresponding to the image to be processed. Thus, the image repair is automatically completed, so that when the target element in the image to be processed is defective, it can be automatically repaired, and the repaired image can clearly and completely express the target element.

[0051] The entire processing chain does not require users to perceive, choose, judge, or participate in verification and review. While improving image processing efficiency, it can also improve image restoration quality and user experience.

[0052] In addition, the image processing nodes can be backtracked for the entire image processing chain, so that the processing nodes can be accurately located and the large model can be adjusted. In addition, by scoring the quality of the target image, the target image can be directly synchronized to the online scene for use if the quality meets the requirements, avoiding the problem of reduced user experience caused by the target image being detected as poor by other image quality detection methods outside the image processing chain after repair processing. BRIEF DESCRIPTION OF THE DRAWINGS

[0053] Figure 1 This is a flowchart of an image processing method based on a large model provided in this application.

[0054] Figure 2 It is a schematic diagram of a segmentation processing embodiment in a large model-based image processing method provided in the present application.

[0055] Figure 3 It is a schematic diagram of an embodiment of a defective area in an image processing method based on a large model provided in the present application.

[0056] Figure 4 It is a schematic diagram of an embodiment of a restored target image in an image processing method based on a large model provided in the present application.

[0057] Figure 5 It is a schematic diagram of repairing and verifying a target image in a large model-based image processing method provided in the present application.

[0058] Figure 6 It is a structural schematic diagram of an image processing device based on a large model provided in this application.

[0059] Figure 7 This is a flowchart of a commodity image processing method based on a large model provided in this application.

[0060] Figure 8 It is a structural schematic diagram of a commodity image processing device based on a large model provided in this application.

[0061] Fig. 9 This is a flow chart of a target image acquisition method provided by the present application.

[0062] Fig.10 It is a structural schematic diagram of a target image acquisition device provided in this application.

[0063] Fig.11 It is a structural schematic diagram of an electronic device provided by this application. DETAILED DESCRIPTION

[0064] Many specific details are described in the following description to facilitate a full understanding of the present application. However, the present application can be implemented in many other ways than those described herein, and those skilled in the art can make similar generalizations without violating the connotation of the present application, so the present application is not limited by the specific implementation disclosed below.

[0065] The terms used in this application are only for the purpose of describing specific embodiments and are not intended to limit this application. The descriptions used in this application and the appended claims, such as "a", "first", and "second", are not limitations on quantity or sequence, but are used to distinguish the same type of information from each other.

[0066] Based on the above background technology, it can be known that the invention concept of this application is derived from the image processing scenario. The specific application scenario can be based on the production scenario of a series of product pictures required by merchants on an online life application service platform, or the production scenario of a series of product pictures required by offline merchants. The commonly used method is to first cut out a product picture, and then fill it with the same background picture, and display the product image through the store product list, thereby improving the user's food selection experience. However, based on the processing of the above-mentioned product pictures, it is found in actual application scenarios that when the main body of the original product picture itself is incomplete, the product picture obtained by cutting out + filling the background is not beautiful, and the picture with incomplete main body is better than the optimized picture under the original background, which makes the image processing more harmful than beneficial, and in actual application scenarios, it will also cause the merchant's product pictures to still have various styles of dishes, making the visual effect messy.

[0067] In view of this, the present application provides an image processing method based on a large model, which can integrate the recognition, incompleteness, and restoration of the main elements of the image into one processing link through automated intelligent image processing, thereby improving the speed and quantity of image processing while improving the quality of image processing.

[0068] like Figure 1 As shown, Figure 1 This is a flowchart of an image processing method based on a large model provided by the present application, the method comprising:

[0069] Step S101: acquiring a segmented image including a target element and a background element according to the segmentation process of the image to be processed; wherein the background element includes a first pixel and the target element includes a second pixel;

[0070] Step S102: determining whether there is a defective region at the edge position of the segmented image of the target element according to the first pixel at the edge position of the segmented image and / or the second pixel at the edge position of the target element, or determining whether there is a defective region at the edge position of the target element in the segmented image; wherein the edge position of the segmented image corresponds to the edge position of the target element;

[0071] Step S103: If yes, the incomplete area in the image to be processed is repaired by using the large model to generate a target image corresponding to the image to be processed.

[0072] The execution of the above steps S101 to S103 is based on the large model, that is, the process of executing the above steps by the input image to be processed is used to obtain the target image output after processing based on the large model. In this embodiment, the large model refers to a neural network model with a large number of parameters and complex structure in the field of deep learning and artificial intelligence. Compared with traditional small models, large models have stronger learning and generalization capabilities and can handle more complex tasks and data sets. Large models are usually trained on huge data sets to acquire more extensive knowledge and capabilities. The large model in this embodiment is a large model for processing images. The type of large model is not specifically limited, and can implement the functions and processing processes involved in this embodiment.

[0073] The above steps S101 to S103 are described in detail below.

[0074] Regarding step S101: according to the segmentation processing of the image to be processed, a segmented image including a target element and a background element is obtained; wherein the background element includes a first pixel, and the target element includes a second pixel.

[0075] The target element refers to the image element included in the image to be processed. For example, for a product image, the target element can be determined based on the type of product in the product image and the attributes to be expressed by the product. The target element of a food image is the element in the image that represents the characteristics of the food attribute, such as Figure 2 As shown, the dish container carries the elements of the dish; the target elements of the daily necessities image are the elements in the image that characterize the attributes of the daily necessities, such as clothing, shoes, socks, etc. Of course, it can also be images in other application scenarios. In this embodiment, the dish image is mainly used as an example for description, and the target element can be the dish element in the dish image.

[0076] The specific implementation process of step S101 may include:

[0077] Step S101-1: performing segmentation processing on the image to be processed to obtain a transparent image including target elements and background elements;

[0078] Step S101 - 2 : determining the transparent image as a segmented image.

[0079] The segmentation process in step S101-1 may include a threshold-based segmentation method, an edge-based segmentation method, a region-based segmentation method, a graph-theory-based segmentation method, a deep learning-based segmentation method, etc.; the threshold segmentation method may be based on an image with obvious differences in color or grayscale values, and adopts a threshold segmentation method. By setting one or more thresholds, pixels are divided into two categories: target pixels and background pixels. The edge segmentation method relies on detecting edge information in the image. The edge is where the grayscale, color or texture characteristics of the image change dramatically, usually corresponding to the boundary between the target image and the background image. The region segmentation method relies on the similarity between pixels to perform region segmentation. The graph-theory segmentation method regards the image as a graph, where the nodes in the graph correspond to pixels and the edges correspond to the similarity measure between pixels, and then uses a graph cut algorithm (such as GrabCut, Normalized Cut, etc.) to segment the target image and the background image. The deep learning method uses models such as convolutional neural networks (CNN) and generative adversarial networks (GAN) to perform segmentation based on image depth maps.

[0080] like Figure 2 As shown, Figure 2It is a schematic diagram of a segmentation processing embodiment in an image processing method based on a large model provided by the present application. In this embodiment, the segmented image after the segmentation processing of the image to be processed can be a mask image (binarized image), for example: a transparent image with background pixels as transparent pixels, that is: the background element includes the first pixel as a transparent pixel, and the target element includes the second pixel as a non-transparent pixel. Of course, the first pixel can also be other pixels, and the second pixel can be a pixel different from the first pixel, and it is not only distinguished by transparent pixels and non-transparent pixels.

[0081] In order to improve the quality of images in subsequent repair, verification and other processing links, the segmented images can also be edge processed, such as smoothing, noise processing, etc., to improve the segmentation quality of the segmented images.

[0082] In order to improve the efficiency of the image processing, avoid waste of resources, and improve the accuracy of image processing, after the execution of step S101, it may also include: judging the image quality of the segmented image to avoid affecting the subsequent image restoration results due to poor image quality, thereby wasting computing resources. In this embodiment, the judgment of the quality of the segmented image may include judging the image quality in terms of whether there is a watermark in the segmented image, whether the target element is blurred and / or over-exposed, whether the target element and the background element overlap, etc. If the image quality meets the quality threshold (or subsequent processing requirements), the subsequent steps are executed, otherwise a prompt may be output. Of course, the judgment of the image quality may also be made before step S101, that is, the image to be processed is obtained, and the image quality of the image to be processed is judged.

[0083] Regarding step S102: determine whether there is a defective area of ​​the target element at the edge position of the segmented image based on the first pixel at the edge position of the segmented image and / or the second pixel at the edge position of the target element; wherein the edge position of the segmented image corresponds to the edge position of the target element.

[0084] like Figure 3 As shown, Figure 3 Schematic diagram of an incomplete region embodiment in a large model-based image processing method provided by the present application. The purpose of step S102 is to determine whether the target element in the image to be processed is incomplete, that is, whether the expression of the target element in the image to be processed is complete. Therefore, the specific implementation process of step S102 can be as follows.

[0085] Method 1 includes:

[0086] Step S102-11: acquiring distribution data of the first pixel at the edge position of the segmented image or distribution data of the second pixel at the edge position of the target element;

[0087] Step S102-12: Determine whether there is a defective area at the edge position of the segmented image for the target element according to the distribution data of the first pixel or the distribution data of the second pixel. The distribution data of the first pixel or the distribution data of the second pixel may be based on the pixel distribution ratio of the first pixel at the edge position to determine whether there is a defective area at the edge position of the segmented image for the target element. That is, when the ratio of the first pixel at the edge position is not 100%, it means that there is a second pixel at the edge position and the target element is defective. Of course, the smaller the ratio of the first pixel at the edge position, the higher the degree of defect of the target element. When the ratio of the second pixel at the edge position is greater than 0, it means that there is a defective target element, and the larger the ratio, the higher the degree of defect. In this embodiment, the ratio may be based on the pixel ratio of the first pixel or the second pixel in the height and / or length direction of the edge position. Therefore, the acquisition of the first pixel or the second pixel may be selected along the height and / or length direction of the edge position. Using the above example, if the first pixel is a transparent pixel, that is, the segmented image is a transparent image, the distribution ratio is the ratio of transparent pixels to the total pixels on each side of the transparent image, and the incomplete area is determined. Alternatively, the incomplete area is determined by the ratio of non-transparent pixels to the total pixels on each side of the transparent image. It can then be determined whether there are incomplete target elements in each direction of the segmented image.

[0088] Therefore, the step S102-12 includes:

[0089] Step S102-12-11: determining whether there is a defective area of ​​the target element at the edge position of the segmented image according to the proportion of the first pixel at the edge position of the segmented image;

[0090] or,

[0091] Step S102-12-12: Determine whether there is a defective area of ​​the target element at the edge position of the segmented image according to the proportion of the second pixel at the edge position of the segmented image.

[0092] Method 2 includes:

[0093] Step S102-21: acquiring a pixel point of the first pixel and a pixel point of the second pixel; wherein the pixel point of the first pixel and the pixel point of the second pixel correspond to each other;

[0094] Step S102-22: Determine whether there is a defective area of ​​the target element at the edge position of the segmented image according to the distance from the pixel point of the first pixel to the pixel point of the second pixel.

[0095] When the distance from the first pixel to the second pixel is equal to 0, it means that the pixel of the second pixel at the edge of the target element is located at the edge of the segmented image, and thus the target element is incomplete. Alternatively, it is determined whether the pixel of the second pixel can be obtained at the edge of the segmented image, and if so, it also means that the target element is incomplete.

[0096] Therefore, in this embodiment, the pixel point of the first pixel can be acquired by acquiring it at the edge position of the segmented image along the image height and / or length direction, and the pixel point of the second pixel can be acquired by acquiring it at the edge position of the target element along the image height and / or length direction, or by acquiring it at the edge position of the segmented image along the image height and / or length direction.

[0097] In this embodiment, when the incomplete area is determined by distance, the pixel point of the first pixel can be selected by selecting the pixel point in the middle area of ​​the height and / or length direction of the edge position of the segmented image, and the pixel point of the second pixel can be selected by selecting the pixel point in the middle area of ​​the height and / or length direction of the edge position of the target element, and then the distance is calculated in sequence. If there is a distance of zero, it means that there is an incomplete area of ​​the target element in the segmented image. When the incomplete area is determined by the pixel value of the target element, it can also be selected along the height and / or length direction of the edge position of the segmented image. If the pixel value of the target element exists in the process of selecting the edge position of the segmented image, it means that there is an incomplete area of ​​the target element.

[0098] The target element edge position can determine the target element edge position area according to the set selection range. The segmented image edge position can be determined according to the image contour edge. The pixel point of the second pixel can be obtained from a pixel point at any point in the selection range. In this embodiment, the pixel point in the middle area of ​​the selection range or the pixel point in the selection range close to the center area of ​​the target element. For example: a×3, where a represents the maximum height of the target element, and 3 represents the width value selected from the edge of the target element to the center. Of course, only 3 is used as an example for explanation here.

[0099] In this embodiment, the incomplete area may be determined by performing the above-mentioned processing on each edge of the segmented image, and the first pixel and the second pixel are pixels with corresponding positions.

[0100] Regarding the step S103: performing image restoration on the incomplete area in the image to be processed by using a large model to generate a target image corresponding to the image to be processed.

[0101] like Figure 4 As shown, Figure 4 1 is a schematic diagram of an embodiment of a target image after restoration in an image processing method based on a large model provided by the present application. The purpose of step S103 is to restore the incomplete area so that the target element is fully expressed in the image to be processed. Specifically, it may include:

[0102] Step S103-1: determining the defect direction in the image to be processed according to the defect area; of course, the defect direction may also be determined according to the position of the first pixel and / or the second pixel, and specifically, the defect direction may be determined according to the coordinates of the pixels.

[0103] Step S103-2: the large model uses the incomplete direction as the designated repair direction, expands the image to be processed according to the specified size along the designated repair direction, and generates an extended image; in this embodiment, the expansion or extension can specifically be to expand the image to be processed along the designated repair direction according to the specified step size or the specified extension ratio to generate an extended image, for example: the image to be processed is 800×800, and the segmented image obtained after segmentation is also 800×800, the incomplete direction is the left side, and the image to be processed is expanded to 800×1200, based on the expanded image with an expansion height of 400 of the image to be processed, that is: 800×400, the extended image is a white background image, so that the subsequent repair can reduce the amount of calculation, and also because by extending the specified direction according to the specified size, the image distortion problem caused by excessive extension can be effectively avoided.

[0104] Step S103-3: Fill the extended image according to the image to be processed to generate a filled image. For filling the extended image, the filling content can be selected from the image to be processed, and then the extended image is filled. Following the above example, a 800×5 strip image is selected from the edge position of the image to be processed, and the strip images are arranged in sequence and filled into the extended image. When the extended image is completely filled, a 800×400 filled image is generated. The filled image is used as the image prompt of the large model to perform step S103-4. Therefore, the specific implementation of step S103-3 may include: step S103-31: select a target image at the edge of the image to be processed according to the selection method; step S103-32: fill the extended image according to the target image to generate the filled image. In this embodiment, an 800×5 strip image is used as an example of the target image for explanation, and how to select the target image from the edge of the image to be processed is not limited to the above example.

[0105] Step S103-4: According to the image to be processed, the filling image is repaired by calling the large model to generate a repaired image. In this embodiment, the filling image can be repaired by calling a large model (for example, the SD large model can also be called the StableDiffusion Inpaint large model). According to the above example, the large model repairs the 800×400 filling image. For example: the filling image is used as the image prompt of the large model, the image optimization module of the large model iteratively optimizes the image features of the filling image, and the iteratively optimized image features are input into the image decoder to reconstruct the image, thereby obtaining the repaired image.

[0106] Step S103-5: The repaired image and the image to be processed are fused to generate a target image corresponding to the image to be processed. In this embodiment, Poisson Blending can be used to fuse the repaired image and the image to be processed. The so-called Poisson Blending is an image processing technology, and its core goal is to seamlessly blend a specific area of ​​an image into another image while maintaining the consistency of illumination, color and texture in the fused area. This technology is based on the Poisson equation, uses the properties of the Laplace equation, takes the pixel values ​​of the two images as boundary conditions, and obtains the fused image by solving the Laplace equation. In the process of Poisson fusion, the continuity of color and brightness will be maintained in the fusion area, while minimizing the discontinuity at the boundary, so that the fused image is more natural. In addition, weighted averaging method, multi-resolution fusion method, etc. can also be used. This embodiment is described by taking the Poisson fusion method as an example.

[0107] In order to avoid special situations such as poor aesthetics due to extension or extension of multiple target elements, which may affect the success rate and accuracy of image restoration after extension, the target image can also be verified, such as Figure 5 As shown, Figure 5 It is a schematic diagram of performing restoration verification on a target image in an image processing method based on a large model provided by the present application, which specifically includes:

[0108] Step S104: verifying the restoration result of the target image;

[0109] Step S105: When the verification result meets the storage requirement, the target image is stored.

[0110] The specific implementation process of step S104 may include: performing segmentation processing on the target image to obtain a target segmented image, determining whether there is a defective area of ​​the target element in the target segmented image, and if not, verifying that the result meets the storage requirement. Further, when there is no defective area of ​​the target element in the target segmented image, the target image may also be scored for image attributes, for example, scoring the light, clarity, and clarity of the target element of the target image, so as to optimize and iterate the large model according to the score, thereby improving the optimization and iteration efficiency of the model.

[0111] Similarly, in the process of verifying the target image, the image quality of the target segmented image can also be judged. If the image meets the quality requirements, the defective area is determined. If it does not meet the quality requirements, the image to be processed is reprocessed or a prompt is output.

[0112] The above is a detailed description of an embodiment of an image processing method based on a large model provided by the present application. For the above method, batch processing can be performed and the processing process can be completed automatically during the image processing process, that is, the image to be processed is input, and the image repair is automatically completed according to the above steps S101 to S103, so that the target element in the image to be processed is incomplete, and it can be automatically repaired, so that the repaired image can clearly and completely express the target element. The entire processing link does not require perception, selection, judgment, and participation in verification and review for the user, which can improve the image repair quality and user experience while improving the image processing efficiency. In addition, the image processing node can be traced back to the overall link of the image processing, so that the processing node can be accurately located and the adjustment of the large model can be realized. And, by scoring the quality of the target image, the target image can be directly synchronized to the online scene for use when the quality meets the requirements, avoiding the problem of reduced user experience caused by the target image after the repair process is completed and being detected as poor by other image quality detection methods outside the image processing link.

[0113] The present application provides an embodiment of an image processing method based on a large model, which ensures the repairability of the image to be processed by judging the image quality after segmentation, and avoids the poor user experience caused by the need to remind the image quality requirements multiple times in the prior art. In addition, it also avoids the problem of increased review costs caused by the randomness of the image-generated image or text-generated image method in the prior art, as well as the problem of increased batch generation costs, increased error rates, and generated image distortion caused by the need for user subjective operations during batch processing.

[0114] The above is a specific description of an embodiment of an image processing method based on a large model provided by the present application. Corresponding to the embodiment of an image processing method based on a large model provided above, the present application also discloses an embodiment of an image processing device based on a large model, please refer to Figure 6 Since the device embodiment is basically similar to the method embodiment, the description is relatively simple, and the relevant parts can be referred to the partial description of the method embodiment. The device embodiment described below is only illustrative.

[0115] like Figure 6 As shown, Figure 6 It is a structural schematic diagram of a large model-based image processing device provided in the present application, which includes: an acquisition unit 601, a determination unit 602 and a generation unit 603.

[0116] The acquisition unit 601 is used to acquire a segmented image including a target element and a background element according to the segmentation process of the image to be processed; wherein the background element includes a first pixel and the target element includes a second pixel;

[0117] The determining unit 602 is configured to determine whether there is a defective region of the target element at the edge position of the segmented image according to the first pixel at the edge position of the segmented image and / or the second pixel at the edge position of the target element; wherein the edge position of the segmented image corresponds to the edge position of the target element;

[0118] The generating unit 603 is used to perform image restoration on the incomplete area in the image to be processed by using a large model when the determination result of the determining unit 602 is yes, so as to generate a target image corresponding to the image to be processed.

[0119] The acquisition unit 601 includes: an acquisition subunit and a determination subunit; the segmentation subunit is used to segment the image to be processed to obtain a transparent image including target elements and background elements; and the determination subunit is used to determine the transparent image as a segmented image.

[0120] For the specific content of the acquisition unit 601, reference may be made to the content of the above step S101, which will not be described in detail here.

[0121] Mode 1 of the determination unit 602 includes: an acquisition subunit and a determination subunit; the acquisition subunit is used to acquire the distribution data of the first pixel at the edge position of the segmented image or the distribution data of the second pixel at the edge position of the target element; the determination subunit is used to determine whether there is a defective area at the edge position of the segmented image of the target element according to the distribution data of the first pixel or the distribution data of the second pixel. The determination subunit is specifically used to determine whether there is a defective area at the edge position of the segmented image of the target element according to the proportion of the first pixel at the edge position of the segmented image; or determine whether there is a defective area at the edge position of the segmented image of the target element according to the proportion of the second pixel at the edge position of the segmented image.

[0122] Mode 2 of the determination unit 602 includes: an acquisition subunit and a determination subunit, the acquisition subunit is used to acquire the pixel point of the first pixel and the pixel point of the second pixel; wherein the pixel point of the first pixel corresponds to the pixel point of the second pixel; the determination subunit is used to determine whether there is a defective area of ​​the target element at the edge position of the segmented image based on the distance from the pixel point of the first pixel to the pixel point of the second pixel.

[0123] For the specific content of the determining unit 602, reference may be made to the content of the above step S102, which will not be described in detail here.

[0124] The generation unit 603 includes: a determination subunit, a first generation subunit, a second generation subunit, a third generation subunit and a fourth generation subunit; the determination subunit is used to determine the defect direction in the image to be processed according to the defective area; the first generation subunit is used for the large model to use the defect direction as a designated repair direction, and to expand the image to be processed along the designated repair direction according to a designated size to generate an extended image; the second generation subunit is used to fill the extended image according to the image to be processed to generate a filled image; the third generation subunit is used to repair the filled image according to the image to be processed to generate a repaired image; the fourth generation subunit is used to fuse the repaired image and the image to be processed to generate a target image corresponding to the image to be processed.

[0125] The first generating subunit is specifically used for expanding the image to be processed along the specified repair direction according to a specified step size or a specified extension ratio by the large model to generate an extended image.

[0126] The second generating subunit includes: a selecting subunit and a generating subunit, wherein the selecting subunit is used to select a target image at the edge of the image to be processed according to a selecting method; and the generating subunit is used to fill the extended image according to the target image to generate the filled image.

[0127] It also includes: a verification unit and a storage unit, wherein the verification unit is used to verify the restoration result of the target image; and the storage unit is used to store the target image when the verification result meets the storage requirement.

[0128] For the specific content of the generating unit 603, reference may be made to the specific content of the above step S103, which will not be described in detail here.

[0129] The above is a description of an image processing device based on a large model provided in this application. For the specific content of the device, please refer to the description of the above method. This is only a summary description.

[0130] Based on the above content, this application also provides a commodity image processing method based on a large model, such as Figure 7 As shown, Figure 7 This is a flowchart of a commodity image processing method based on a large model provided by the present application, the method comprising:

[0131] Step S701: acquiring a segmented image including a target product element and a background element according to the segmentation process of the product image to be processed; wherein the background element includes a first pixel, and the target product element includes a second pixel;

[0132] Step S702: determining whether there is a defective area of ​​the target product element at the edge position of the segmented image according to the first pixel at the edge position of the segmented image and / or the second pixel at the edge position of the target product element; wherein the edge position of the segmented image corresponds to the edge position of the target product element;

[0133] Step S703: If yes, the incomplete area in the product image to be processed is repaired by using the large model to generate a target product image corresponding to the product image to be processed.

[0134] The commodity images to be processed may be images of food, clothing, accessories, etc., that is, the items traded based on the application platform are not limited to physical items but also include virtual items.

[0135] It may also include:

[0136] Step S704: verifying the restoration result of the target product image;

[0137] Step S705: When the verification result meets the storage requirement, the target product image is stored.

[0138] For the contents of the above steps S701 to S705, reference may be made to the above steps S101 to S105, which will not be described in detail here.

[0139] Accordingly, the present application also provides a commodity image processing device based on a large model, such as Figure 8 As shown, Figure 8 This is a schematic diagram of the structure of a commodity image processing device based on a large model provided by the present application. The device includes:

[0140] The acquisition unit 801 is used to acquire a segmented image including a target product element and a background element according to the segmentation process of the product image to be processed; wherein the background element includes a first pixel, and the target product element includes a second pixel;

[0141] A determining unit 802 is configured to determine whether there is a defective area of ​​the target commodity element at the edge position of the segmented image according to the first pixel at the edge position of the segmented image and / or the second pixel at the edge position of the target commodity element; wherein the edge position of the segmented image corresponds to the edge position of the target commodity element;

[0142] The generating unit 803 is used to perform image restoration on the incomplete area in the commodity image to be processed by using a large model when the determination result of the determining unit 802 is yes, so as to generate a target commodity image corresponding to the commodity image to be processed.

[0143] Based on the above content, the present application also provides a method for acquiring a target image, such as Fig. 9 As shown, Fig. 9 is a flow chart of a target image acquisition method provided by the present application. The method embodiment may include:

[0144] Step S901: Select an image to be processed;

[0145] Step S902: Acquire a target image corresponding to the image to be processed, which is determined according to the large model-based image processing method.

[0146] In this embodiment, the selection of the image to be processed and the acquisition of the target image can be implemented on the client, and the client can be determined in combination with a specific application scenario. For example, during the processing of the product image, the client can be a client on the merchant side.

[0147] After obtaining the target image, the client can output, display, modify, or store it locally according to its own needs, and of course remote output display or modification operations are not excluded. That is, after obtaining the target image, the client can process the target image accordingly according to the required scenario.

[0148] Accordingly, the present application also provides a target image acquisition device, such as Fig.10 As shown, Fig.10 : is a schematic diagram of a target image acquisition device provided by the present application. The device embodiment may include:

[0149] Selection unit 1001: selects an image to be processed;

[0150] Acquisition unit 1002: Acquisition of a target image corresponding to the image to be processed, which is determined according to the large model-based image processing method.

[0151] Regarding the above-mentioned target image acquisition method and device, reference may be made to the above-mentioned large model-based image processing method and large model commodity image processing method and other related descriptions, which will not be described in detail here.

[0152] Based on the above content, the present application also provides a computer storage medium for storing data generated by a network platform, and a program for processing the data generated by the network platform;

[0153] When the program is read and executed by the processor, it executes the image processing method based on the large model as described above, or executes the commodity image processing method based on the large model as described above.

[0154] Based on the above content, the present application also provides an electronic device, such as Fig.11 As shown, Fig.11This is a schematic diagram of the structure of an electronic device provided by the present application. The electronic device includes:

[0155] Processor 1101;

[0156] The memory 1102 is used to store a program for processing data generated by a network platform. When the program is read and executed by the processor, it executes the image processing method based on the large model as described above, or executes the commodity image processing method based on the large model as described above.

[0157] It should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, stored data, displayed data, etc.) involved in this application are all information and data authorized by the user or fully authorized by all parties, and the collection, use and processing of relevant data must comply with the relevant laws, regulations and standards of relevant countries and regions, and provide corresponding operation entrances for users to choose to authorize or refuse.

[0158] In a typical configuration, a computing device includes one or more processors (CPU), input / output interfaces, network interfaces, and memory.

[0159] Memory may include non-permanent storage in a computer-readable medium, in the form of random access memory (RAM) and / or non-volatile memory, such as read-only memory (ROM) or flash RAM. Memory is an example of a computer-readable medium.

[0160] 1. Computer-readable media includes permanent and non-permanent, removable and non-removable media that can be used to store information by any method or technology. Information can be computer-readable instructions, data structures, program modules or other data. Examples of computer storage media include, but are not limited to, phase change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technology, compact disc read-only memory (CD-ROM), digital versatile disc (DVD) or other optical storage, magnetic cassettes, magnetic tape magnetic disk storage or other magnetic storage devices or any other non-transmission media that can be used to store information that can be accessed by a computing device. As defined herein, computer-readable media does not include non-transitory media such as modulated data signals and carrier waves.

[0161] 2. Those skilled in the art should understand that the embodiments of the present application can be provided as methods, systems or computer program products. Therefore, the present application can take the form of a complete hardware embodiment, a complete software embodiment or an embodiment combining software and hardware. Moreover, the present application can take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program codes.

[0162] Although the present application is disclosed as above in the form of a preferred embodiment, it is not intended to limit the present application. Any technical personnel in this field may make possible changes and modifications without departing from the spirit and scope of the present application. Therefore, the scope of protection of the present application shall be based on the scope defined by the claims of the present application.

Claims

1. A large model-based image processing method, characterized in that: include: According to the segmentation process of the image to be processed, a segmented image including a target element and a background element is obtained; wherein the background element includes a first pixel, and the target element includes a second pixel; Determine whether there is a defective area of ​​the target element at the edge position of the segmented image according to the first pixel at the edge position of the segmented image and / or the second pixel at the edge position of the target element; wherein the edge position of the segmented image corresponds to the edge position of the target element; If yes, then performing image restoration on the incomplete area in the image to be processed by using the large model to generate a target image corresponding to the image to be processed, including: Determining a defect direction in the image to be processed according to the defect area; The large model uses the defective direction as a designated repair direction, and expands the image to be processed according to a designated size along the designated repair direction to generate an expanded image; Filling the extended image according to the image to be processed to generate a filled image; According to the image to be processed, repairing the filled image to generate a repaired image; The repaired image and the image to be processed are fused to generate a target image corresponding to the image to be processed.

2. The image processing method based on a large model according to claim 1, characterized in that: The determining, based on the first pixel at the edge position of the segmented image and / or the second pixel at the edge position of the target element, whether there is a defective area of ​​the target element at the edge position of the segmented image comprises: Acquire the distribution data of the first pixel at the edge position of the segmented image or the distribution data of the second pixel at the edge position of the target element; According to the distribution data of the first pixel or the distribution data of the second pixel, it is determined whether there is a defective area of ​​the target element at the edge position of the segmented image.

3. The image processing method based on a large model according to claim 2, characterized in that: The determining whether there is a defective area of ​​the target element at the edge position of the segmented image according to the distribution of the first pixels at the edge position or the distribution of the second pixels at the edge position includes: Determining whether there is a defective area of ​​the target element at the edge position of the segmented image according to the proportion of the first pixel at the edge position of the segmented image; or, According to the proportion of the second pixels at the edge position of the segmented image, it is determined whether there is a defective area of ​​the target element at the edge position of the segmented image.

4. The image processing method based on a large model according to claim 1, characterized in that: The determining, based on the first pixel at the edge position of the segmented image and / or the second pixel at the edge position of the target element, whether there is a defective area of ​​the target element at the edge position of the segmented image comprises: Acquire a pixel point of the first pixel and a pixel point of the second pixel; wherein the pixel point of the first pixel and the pixel point of the second pixel correspond to each other; It is determined whether there is a defective area of ​​the target element at an edge position of the segmented image according to the distance from the pixel point of the first pixel to the pixel point of the second pixel.

5. The image processing method based on a large model according to claim 1, characterized in that: The large model uses the defective direction as a designated repair direction, and expands the image to be processed according to a designated size along the designated repair direction to generate an expanded image, including: The large model expands the image to be processed along the specified repair direction according to a specified step size or a specified extension ratio to generate an expanded image.

6. The image processing method based on a large model according to claim 1, characterized in that: The step of filling the extended image according to the image to be processed to generate a filled image includes: According to the selection method, a target image is selected at the edge of the image to be processed; The extended image is filled according to the target image to generate the filled image.

7. The image processing method based on a large model according to claim 1, characterized in that: Also includes: Performing restoration result verification on the target image; When the verification result meets the storage requirement, the target image is stored.

8. A method for acquiring a target image, characterized in that: include: Select the image to be processed; Obtain a target image corresponding to the image to be processed determined by the large model-based image processing method described in any one of claims 1 to 7.

9. A commodity image processing method based on a large model, characterized in that: include: According to the segmentation processing of the commodity image to be processed, a segmented image including a target commodity element and a background element is obtained; wherein the background element includes a first pixel, and the target commodity element includes a second pixel; Determine whether there is a defective area of ​​the target product element at the edge position of the segmented image according to the first pixel at the edge position of the segmented image and / or the second pixel at the edge position of the target product element; wherein the edge position of the segmented image corresponds to the edge position of the target product element; If yes, then the incomplete area in the product image to be processed is repaired by using the large model to generate a target product image corresponding to the product image to be processed, including: Determining a defect direction in the product image to be processed according to the defect area; The large model uses the defective direction as a designated repair direction, and expands the commodity image to be processed according to a designated size along the designated repair direction to generate an expanded image; Filling the extended image according to the commodity image to be processed to generate a filled image; According to the commodity image to be processed, the filled image is repaired to generate a repaired image; The repaired image and the product image to be processed are fused to generate a target product image corresponding to the product image to be processed.

10. A computer storage medium for storing data generated by a network platform and a program for processing the data generated by the network platform; When the program is read and executed by a processor, the program executes the method according to any one of claims 1 to 7, or the method according to claim 8, or the method according to claim 9.

11. An electronic device, comprising: processor; A memory for storing a program for processing data generated by a network platform, wherein when the program is read and executed by the processor, the program executes the method according to any one of claims 1 to 7, or the method according to claim 8, or the method according to claim 9.

Images